diff --git a/README.md b/README.md index ed251cf..58ee154 100644 --- a/README.md +++ b/README.md @@ -13,7 +13,7 @@ --- -Scrob syncs your libraries from **Jellyfin**, **Plex**, and **Emby**, tracks your watch history, ratings, and personal lists, and lets you push your activity back to your media server - all from a clean, app-like web interface that installs as a PWA on any device. +Scrob syncs your libraries from **Jellyfin**, **Plex**, **Emby**, and **Nuvio**, tracks your watch history, ratings, and personal lists, and can push watched activity back to connected providers - all from a clean, app-like web interface that installs as a PWA on any device. ## Table of Contents @@ -26,7 +26,11 @@ Scrob syncs your libraries from **Jellyfin**, **Plex**, and **Emby**, tracks you - [First Setup](#first-setup) - [Updating](#updating) - [Configuration](#configuration) -- [Development](#development) +- [Nuvio Cloud Synchronization](#nuvio-cloud-synchronization) + - [Connect Nuvio](#connect-nuvio) + - [Synchronization Directions](#synchronization-directions) + - [Scheduling and Limitations](#scheduling-and-limitations) +- [MDBList Synchronization](#mdblist-synchronization) - [Webhooks](#webhooks-real-time-scrobbling) - [Jellyfin](#jellyfin) - [Plex](#plex) @@ -36,16 +40,18 @@ Scrob syncs your libraries from **Jellyfin**, **Plex**, and **Emby**, tracks you - [Email Validation & SMTP](#email-validation--smtp) - [Contributing](#contributing) - [Contributors](#contributors) +- [Development](#development) - [License](#license) ## Features -- **Multi-source sync**: Import your full library, watch history, and ratings from Jellyfin, Plex, and Emby. Incremental syncs keep everything up to date. -- **Keep all servers in sync**: Keep your watched status in sync between all your servers. Supports multiple instances. +- **Multi-source sync**: Import libraries and watch history from Jellyfin, Plex, Emby, and Nuvio. Nuvio also imports playback progress for Continue Watching. +- **Keep providers in sync**: Keep watched status synchronized between your media servers and Nuvio. Supports multiple instances and Nuvio profiles. - **Real-time scrobbling**: Webhooks from Jellyfin, Plex, Emby, and Kodi update your watch state as you play - no manual sync needed. - **Manual scrobble**: Start a watching session directly from any movie or episode page. Pause, resume, stop, or mark as watched - session progress shows live on the home screen. - **Trakt integration**: Sync your watched history and ratings from Trakt, and push Scrob activity back to Trakt automatically. - **Simkl integration**: Sync your watched history and ratings from Simkl, and push Scrob activity back to Simkl automatically. +- **MDBList integration**: Pull watched history, ratings, and watchlist items from MDBList, and optionally push Scrob changes back using an MDBList API key. - **Watch history & ratings**: Track every movie and episode you've watched, including multiple plays with individual timestamps. Log plays manually with a custom date, or remove individual entries — all from the watched button on any movie or episode page. Rate them on a 10-point scale with optional reviews. - **Season ratings**: Rate individual seasons separately from the overall show. - **Personal lists**: Create and curate lists of movies and shows. Mark them public to share with other users on the same instance. @@ -111,7 +117,7 @@ Scrob syncs your libraries from **Jellyfin**, **Plex**, and **Emby**, tracks you ### Prerequisites - [Docker](https://docs.docker.com/get-docker/) and [Docker Compose](https://docs.docker.com/compose/install/) -- A [TMDB API key](https://www.themoviedb.org/settings/api) (free) - used for metadata, search, and images +- A [TMDB Read Access Token](https://www.themoviedb.org/settings/api) (free) - used for metadata, search, and images ### Docker Compose @@ -249,8 +255,10 @@ docker run -d \ ### First Setup 1. Open `http://localhost:7330` and create your account. -2. Go to **Settings → Integrations** to add your TMDB API key and connect Jellyfin, Plex, or Emby. -3. Select which libraries to sync, then trigger your first sync from **Settings → Sync**. +2. Go to **Settings → Integrations** to add your TMDB Read Access Token and connect Jellyfin, Plex, Emby, or Nuvio. +3. Select which media-server libraries to sync, or select a Nuvio profile, then trigger your first sync. + +For Nuvio, choose **Nuvio** as the provider, sign in, and select one of the returned profiles. See [Nuvio Cloud Synchronization](#nuvio-cloud-synchronization) for credential handling, sync directions, scheduling, and current limitations. ### Updating @@ -296,6 +304,50 @@ Remove the `scrob-db` service and set `DATABASE_URL` to your existing instance: DATABASE_URL: postgresql+asyncpg://user:password@your-db-host:5432/scrob ``` +## Nuvio Cloud Synchronization + +Scrob connects to the [Nuvio public Cloud API](https://nuvio.tv/docs) at `https://api.nuvio.tv`. A TMDB Read Access Token must be configured in Scrob so Nuvio content identifiers can be matched to movies and shows. + +### Connect Nuvio + +1. Open **Settings → Media & Cloud Connections** and select **Add Connection**. +2. Choose **Nuvio**, then enter a connection name, your Nuvio email, and your Nuvio password. +3. Select **Test** to authenticate and load the profiles attached to the account. +4. Select the Nuvio profile to synchronize, choose the pull and push options, then select **Add**. + +Scrob exchanges the email and password for a refresh token. The password is never persisted. Refresh-token rotation is handled automatically during connection checks and synchronization. + +Each connection targets one Nuvio profile. Add another connection if you need to synchronize another profile from the same account. + +### Synchronization Directions + +| Direction | Setting | Behavior | +|---|---|---| +| Nuvio → Scrob | **Collection status** | Imports the profile's library movies and series. | +| Nuvio → Scrob | **Watched status** | Imports watched movies and episodes with their latest watch timestamps. | +| Nuvio → Scrob | **Playback progress** | Imports position and duration into Continue Watching. | +| Scrob → Nuvio | **Watched status** | Pushes watched and unwatched changes made in Scrob or imported from another connected provider. | +| Scrob → Nuvio | **Playback progress** | Pushes current playback positions into Nuvio's Continue Watching state as non-destructive upserts. | + +**Sync now** runs an inbound synchronization using the enabled Nuvio → Scrob settings. **Push** sends the enabled watched-history and playback-progress data from Scrob to Nuvio. Both operations use non-destructive upserts; items absent from Scrob are not removed from Nuvio. + +Library membership is currently pull-only. Ratings are not synchronized with Nuvio. + +### Scheduling and Limitations + +**Auto Pull** repeats the enabled inbound synchronization every 15 minutes, 30 minutes, 1 hour, 3 hours, 6 hours, 12 hours, 24 hours, or 48 hours. Nuvio synchronization is polling-based; Nuvio does not use the media-server webhook URLs documented below. + +Inbound Nuvio identifiers are normalized to TMDB for Scrob's internal matching. Before an outbound push, Scrob resolves those TMDB identifiers to Nuvio-compatible bare IMDb identifiers (`tt...`) and caches the mapping. Unsupported identifiers are skipped rather than attached to the wrong title. + +## MDBList Synchronization + +1. Open **Settings → Connections → MDBList**. +2. Copy the API key from [MDBList Preferences](https://mdblist.com/preferences), paste it into Scrob, and select **Save Changes**. +3. Choose the data to import under **MDBList → Scrob**, then select **Pull**. MDBList pulls run only when this button is selected. +4. To send changes back, enable the required **Scrob → MDBList** options. Watched-state and rating edits are pushed as they happen; edits to the managed **MDBList - Watchlist** are pushed to the MDBList watchlist. + +The manual **Push** action sends the complete enabled watched, ratings, or managed-watchlist snapshot. MDBList pagination follows `next_cursor` and requests the documented maximum of 1,000 items per page. + ## Webhooks (Real-time Scrobbling) Webhooks update your watch history and Continue Watching in real time. Each user's webhook URL is shown in **Settings** next to the relevant integration. diff --git a/backend/core/enrichment.py b/backend/core/enrichment.py index aeb4331..00d4f4d 100644 --- a/backend/core/enrichment.py +++ b/backend/core/enrichment.py @@ -36,6 +36,7 @@ async def enrich_media(media: Media, api_key: str = None, series_tmdb_id: int = media.tmdb_data = { "runtime": data.get("runtime"), "genres": [g["name"] for g in data.get("genres", [])], + "external_ids": data.get("external_ids", {}), "cast": [ {"name": c["name"], "character": c["character"], "profile_path": tmdb.poster_url(c.get("profile_path"), size="w185")} for c in data.get("credits", {}).get("cast", [])[:10] diff --git a/backend/core/image_cache.py b/backend/core/image_cache.py index e1f25e8..fed2399 100644 --- a/backend/core/image_cache.py +++ b/backend/core/image_cache.py @@ -24,6 +24,9 @@ _last_prune_at: datetime = datetime.min.replace(tzinfo=timezone.utc) _PRUNE_INTERVAL = timedelta(minutes=5) +# asyncpg hard limit is 32767 parameters per query; stay well under it +_MAX_IN_PARAMS = 30_000 + def parse_tmdb_url(url: str) -> tuple[Optional[str], Optional[str]]: """Parse TMDB URL or path into (size, path) tuple.""" @@ -120,8 +123,9 @@ async def _prune_type(db, image_type: str, total_size: int, limit_bytes: int) -> total_size -= row.file_size ids_to_delete.append(row.id) - if ids_to_delete: - await db.execute(sa_delete(ImageCache).where(ImageCache.id.in_(ids_to_delete))) + for i in range(0, len(ids_to_delete), _MAX_IN_PARAMS): + chunk = ids_to_delete[i : i + _MAX_IN_PARAMS] + await db.execute(sa_delete(ImageCache).where(ImageCache.id.in_(chunk))) return total_size diff --git a/backend/core/mdblist.py b/backend/core/mdblist.py new file mode 100644 index 0000000..2a388b2 --- /dev/null +++ b/backend/core/mdblist.py @@ -0,0 +1,167 @@ +"""Async client for MDBList's user synchronization API.""" + +from __future__ import annotations + +from collections.abc import Iterable +from typing import Any + +import httpx + +MDBLIST_BASE = "https://api.mdblist.com" +PAGE_SIZE = 1000 +PUSH_BATCH_SIZE = 500 + + +class MDBListAPIError(RuntimeError): + """Raised when MDBList rejects or cannot complete a request.""" + + +async def _request( + method: str, + path: str, + api_key: str, + *, + params: dict[str, Any] | None = None, + payload: dict[str, Any] | None = None, +) -> dict[str, Any]: + query = dict(params or {}) + query["apikey"] = api_key + try: + async with httpx.AsyncClient(timeout=60.0) as client: + response = await client.request( + method, + f"{MDBLIST_BASE}{path}", + params=query, + json=payload, + ) + response.raise_for_status() + except httpx.HTTPStatusError as exc: + detail = exc.response.text.strip()[:500] + suffix = f": {detail}" if detail else "" + raise MDBListAPIError( + f"MDBList {method} {path} failed ({exc.response.status_code}){suffix}" + ) from exc + except httpx.HTTPError as exc: + raise MDBListAPIError(f"MDBList {method} {path} failed: {exc}") from exc + + if response.status_code == 204 or not response.content: + return {} + data = response.json() + if not isinstance(data, dict): + raise MDBListAPIError(f"MDBList {method} {path} returned an invalid response") + return data + + +async def validate_api_key(api_key: str) -> bool: + if not api_key: + return False + try: + await _request("GET", "/sync/last_activities", api_key) + return True + except MDBListAPIError: + return False + + +async def _get_all(api_key: str, path: str) -> dict[str, Any]: + merged: dict[str, Any] = { + "movies": [], + "shows": [], + "seasons": [], + "episodes": [], + } + cursor: str | None = None + total_seen = 0 + seen_cursors: set[str] = set() + + while True: + params: dict[str, Any] = {"limit": PAGE_SIZE} + if cursor: + params["cursor"] = cursor + elif total_seen: + params["offset"] = total_seen + + page = await _request("GET", path, api_key, params=params) + page_count = 0 + for key in ("movies", "shows", "seasons", "episodes"): + values = page.get(key) + if isinstance(values, list): + merged[key].extend(values) + page_count += len(values) + total_seen += page_count + + pagination = page.get("pagination") + pagination = pagination if isinstance(pagination, dict) else {} + next_cursor = pagination.get("next_cursor") + if next_cursor: + next_cursor = str(next_cursor) + if next_cursor in seen_cursors: + raise MDBListAPIError(f"MDBList {path} returned a repeated pagination cursor") + seen_cursors.add(next_cursor) + cursor = next_cursor + continue + + if pagination.get("has_more"): + if page_count == 0: + raise MDBListAPIError(f"MDBList {path} reported more pages without returning items") + cursor = None + continue + break + + return merged + + +async def get_watched(api_key: str) -> dict[str, Any]: + return await _get_all(api_key, "/sync/watched") + + +async def get_ratings(api_key: str) -> dict[str, Any]: + return await _get_all(api_key, "/sync/ratings") + + +async def get_watchlist(api_key: str) -> dict[str, Any]: + return await _get_all(api_key, "/watchlist/items") + + +def _batched_payloads(payload: dict[str, list[dict[str, Any]]]) -> Iterable[dict[str, list[dict[str, Any]]]]: + for key in ("movies", "shows", "seasons", "episodes"): + values = payload.get(key, []) + for offset in range(0, len(values), PUSH_BATCH_SIZE): + yield {key: values[offset : offset + PUSH_BATCH_SIZE]} + + +async def _push(path: str, api_key: str, payload: dict[str, list[dict[str, Any]]]) -> dict[str, int]: + stats = {"submitted": 0, "batches": 0, "not_found": 0} + for batch in _batched_payloads(payload): + stats["batches"] += 1 + result = await _request("POST", path, api_key, payload=batch) + stats["submitted"] += sum(len(values) for values in batch.values()) + not_found = result.get("not_found") + if isinstance(not_found, dict): + stats["not_found"] += sum( + len(values) for values in not_found.values() if isinstance(values, list) + ) + return stats + + +async def push_watched(api_key: str, payload: dict[str, list[dict[str, Any]]]) -> dict[str, int]: + return await _push("/sync/watched", api_key, payload) + + +async def remove_watched(api_key: str, payload: dict[str, list[dict[str, Any]]]) -> dict[str, int]: + return await _push("/sync/watched/remove", api_key, payload) + + +async def push_ratings(api_key: str, payload: dict[str, list[dict[str, Any]]]) -> dict[str, int]: + return await _push("/sync/ratings", api_key, payload) + + +async def remove_ratings(api_key: str, payload: dict[str, list[dict[str, Any]]]) -> dict[str, int]: + return await _push("/sync/ratings/remove", api_key, payload) + + +async def push_watchlist(api_key: str, payload: dict[str, list[dict[str, Any]]]) -> dict[str, int]: + return await _push("/watchlist/items/add", api_key, payload) + + +async def remove_watchlist(api_key: str, payload: dict[str, list[dict[str, Any]]]) -> dict[str, int]: + return await _push("/watchlist/items/remove", api_key, payload) diff --git a/backend/core/nuvio.py b/backend/core/nuvio.py new file mode 100644 index 0000000..0a23cdc --- /dev/null +++ b/backend/core/nuvio.py @@ -0,0 +1,315 @@ +import asyncio +from dataclasses import dataclass +from typing import Any + +import httpx + + +DEFAULT_URL = "https://api.nuvio.tv" +PUBLISHABLE_KEY = "sb_publishable_1Clq8rlTVACkdcZuqr6_AD__xUUC_EN" +_PAGE_SIZE = 500 + + +class NuvioAPIError(RuntimeError): + pass + + +def parse_profile_id(value: str | None) -> int: + try: + profile_id = int(value or "") + except (TypeError, ValueError): + raise NuvioAPIError("Nuvio profile must be an integer from 1 to 6") + if profile_id < 1 or profile_id > 6: + raise NuvioAPIError("Nuvio profile must be an integer from 1 to 6") + return profile_id + + +@dataclass(frozen=True) +class NuvioSession: + access_token: str + refresh_token: str + expires_in: int + + +def _base_url(url: str) -> str: + return url.rstrip("/") + + +def _public_headers() -> dict[str, str]: + return {"apikey": PUBLISHABLE_KEY, "Content-Type": "application/json"} + + +def _auth_headers(access_token: str) -> dict[str, str]: + return {**_public_headers(), "Authorization": f"Bearer {access_token}"} + + +async def _raise_api_error(response: httpx.Response, operation: str) -> None: + if response.is_success: + return + try: + payload = response.json() + detail = payload.get("message") or payload.get("error_description") or payload.get("error") + except (ValueError, AttributeError): + detail = None + suffix = f": {detail}" if detail else "" + raise NuvioAPIError(f"Nuvio {operation} failed ({response.status_code}){suffix}") + + +def _parse_session(payload: dict[str, Any]) -> NuvioSession: + access_token = payload.get("access_token") + refresh_token = payload.get("refresh_token") + if not access_token or not refresh_token: + raise NuvioAPIError("Nuvio authentication returned an incomplete session") + return NuvioSession( + access_token=str(access_token), + refresh_token=str(refresh_token), + expires_in=int(payload.get("expires_in") or 0), + ) + + +async def sign_in(url: str, email: str, password: str) -> NuvioSession: + async with httpx.AsyncClient(timeout=20.0, follow_redirects=False) as client: + response = await client.post( + f"{_base_url(url)}/auth/v1/token", + params={"grant_type": "password"}, + headers=_public_headers(), + json={"email": email, "password": password}, + ) + await _raise_api_error(response, "sign-in") + return _parse_session(response.json()) + + +async def refresh_session( + url: str, + refresh_token: str, + client: httpx.AsyncClient | None = None, +) -> NuvioSession: + owns_client = client is None + if client is None: + client = httpx.AsyncClient(timeout=20.0, follow_redirects=False) + try: + response = await client.post( + f"{_base_url(url)}/auth/v1/token", + params={"grant_type": "refresh_token"}, + headers=_public_headers(), + json={"refresh_token": refresh_token}, + ) + await _raise_api_error(response, "token refresh") + return _parse_session(response.json()) + finally: + if owns_client: + await client.aclose() + + +async def _rpc( + client: httpx.AsyncClient, + url: str, + access_token: str, + function_name: str, + payload: dict[str, Any] | None = None, +) -> Any: + response = await client.post( + f"{_base_url(url)}/rest/v1/rpc/{function_name}", + headers=_auth_headers(access_token), + json=payload, + ) + await _raise_api_error(response, function_name) + if response.status_code == 204 or not response.content: + return None + return response.json() + + +async def get_profiles( + url: str, + access_token: str, + client: httpx.AsyncClient | None = None, +) -> list[dict[str, Any]]: + owns_client = client is None + if client is None: + client = httpx.AsyncClient(timeout=20.0, follow_redirects=False) + try: + profiles = await _rpc(client, url, access_token, "sync_pull_profiles") + return profiles if isinstance(profiles, list) else [] + finally: + if owns_client: + await client.aclose() + + +async def authenticate(url: str, email: str, password: str) -> tuple[NuvioSession, list[dict[str, Any]]]: + session = await sign_in(url, email, password) + profiles = await get_profiles(url, session.access_token) + return session, profiles + + +async def validate_connection( + url: str, + refresh_token: str, + profile_id: int | None = None, +) -> tuple[NuvioSession, list[dict[str, Any]]]: + async with httpx.AsyncClient(timeout=20.0, follow_redirects=False) as client: + session = await refresh_session(url, refresh_token, client=client) + profiles = await get_profiles(url, session.access_token, client=client) + if profile_id is not None and not any(int(profile.get("profile_index") or 0) == profile_id for profile in profiles): + raise NuvioAPIError(f"Nuvio profile {profile_id} was not found") + return session, profiles + + +async def _pull_library( + client: httpx.AsyncClient, + url: str, + access_token: str, + profile_id: int, +) -> list[dict[str, Any]]: + items: list[dict[str, Any]] = [] + offset = 0 + while True: + page = await _rpc( + client, + url, + access_token, + "sync_pull_library", + {"p_profile_id": profile_id, "p_limit": _PAGE_SIZE, "p_offset": offset}, + ) + page = page if isinstance(page, list) else [] + items.extend(page) + if len(page) < _PAGE_SIZE: + return items + offset += _PAGE_SIZE + + +async def _pull_watched_items( + client: httpx.AsyncClient, + url: str, + access_token: str, + profile_id: int, +) -> list[dict[str, Any]]: + items: list[dict[str, Any]] = [] + page_number = 1 + while True: + page = await _rpc( + client, + url, + access_token, + "sync_pull_watched_items", + {"p_profile_id": profile_id, "p_page": page_number, "p_page_size": _PAGE_SIZE}, + ) + page = page if isinstance(page, list) else [] + items.extend(page) + if len(page) < _PAGE_SIZE: + return items + page_number += 1 + + +async def _pull_watch_progress( + client: httpx.AsyncClient, + url: str, + access_token: str, + profile_id: int, +) -> list[dict[str, Any]]: + # sync_pull_watch_progress does not accept a p_offset parameter — passing + # one makes PostgREST 404 with "could not find the function" because no + # matching signature exists. The RPC always returns the full in-progress + # list (bounded, unlike watch history) in a single call. + rows = await _rpc( + client, + url, + access_token, + "sync_pull_watch_progress", + {"p_profile_id": profile_id, "p_limit": 200}, + ) + return rows if isinstance(rows, list) else [] + + +async def pull_sync_data( + url: str, + refresh_token: str, + profile_id: int, +) -> tuple[NuvioSession, dict[str, list[dict[str, Any]]]]: + async with httpx.AsyncClient(timeout=30.0, follow_redirects=False) as client: + session = await refresh_session(url, refresh_token, client=client) + profiles = await get_profiles(url, session.access_token, client=client) + if not any(int(profile.get("profile_index") or 0) == profile_id for profile in profiles): + raise NuvioAPIError(f"Nuvio profile {profile_id} was not found") + library, watched, progress = await asyncio.gather( + _pull_library(client, url, session.access_token, profile_id), + _pull_watched_items(client, url, session.access_token, profile_id), + _pull_watch_progress(client, url, session.access_token, profile_id), + ) + return session, {"library": library, "watched": watched, "progress": progress} + + +async def _push_sync_items( + url: str, + refresh_token: str, + profile_id: int, + watched_items: list[dict[str, Any]] | None = None, + progress_items: list[dict[str, Any]] | None = None, +) -> NuvioSession: + async with httpx.AsyncClient(timeout=30.0, follow_redirects=False) as client: + session = await refresh_session(url, refresh_token, client=client) + for function_name, items_key, items in ( + ("sync_push_watched_items", "p_items", watched_items or []), + ("sync_push_watch_progress", "p_entries", progress_items or []), + ): + for offset in range(0, len(items), _PAGE_SIZE): + await _rpc( + client, + url, + session.access_token, + function_name, + {"p_profile_id": profile_id, items_key: items[offset : offset + _PAGE_SIZE]}, + ) + return session + + +async def push_watched_items( + url: str, + refresh_token: str, + profile_id: int, + items: list[dict[str, Any]], +) -> NuvioSession: + return await _push_sync_items(url, refresh_token, profile_id, watched_items=items) + + +async def push_watch_progress( + url: str, + refresh_token: str, + profile_id: int, + items: list[dict[str, Any]], +) -> NuvioSession: + return await _push_sync_items(url, refresh_token, profile_id, progress_items=items) + + +async def push_sync_items( + url: str, + refresh_token: str, + profile_id: int, + watched_items: list[dict[str, Any]], + progress_items: list[dict[str, Any]], +) -> NuvioSession: + return await _push_sync_items( + url, + refresh_token, + profile_id, + watched_items=watched_items, + progress_items=progress_items, + ) + + +async def delete_watched_items( + url: str, + refresh_token: str, + profile_id: int, + keys: list[dict[str, Any]], +) -> NuvioSession: + async with httpx.AsyncClient(timeout=30.0, follow_redirects=False) as client: + session = await refresh_session(url, refresh_token, client=client) + for offset in range(0, len(keys), _PAGE_SIZE): + await _rpc( + client, + url, + session.access_token, + "sync_delete_watched_items", + {"p_profile_id": profile_id, "p_keys": keys[offset : offset + _PAGE_SIZE]}, + ) + return session diff --git a/backend/core/tmdb.py b/backend/core/tmdb.py index 1c61408..b66981c 100644 --- a/backend/core/tmdb.py +++ b/backend/core/tmdb.py @@ -52,7 +52,7 @@ async def validate_api_key(api_key: str) -> bool: async def get_movie(tmdb_id: int, api_key: str = None, language: str | None = None) -> dict: - params: dict = {"append_to_response": "credits,release_dates,recommendations"} + params: dict = {"append_to_response": "credits,release_dates,recommendations,external_ids"} if language: params["language"] = language return await _get( diff --git a/backend/core/trakt.py b/backend/core/trakt.py index 4b3c091..83bfcf6 100644 --- a/backend/core/trakt.py +++ b/backend/core/trakt.py @@ -16,6 +16,40 @@ TRAKT_BASE = "https://api.trakt.tv" TIMEOUT = 30.0 +PAGE_SIZE = 250 + + +async def _get_all_pages( + client: httpx.AsyncClient, + path: str, + headers: dict, + extra_params: dict[str, str] | None = None, +) -> list[dict]: + items: list[dict] = [] + page = 1 + while True: + params = dict(extra_params or {}) + params.update({"page": page, "limit": PAGE_SIZE}) + response = await client.get( + f"{TRAKT_BASE}{path}", + headers=headers, + params=params, + ) + response.raise_for_status() + page_items = response.json() + if not isinstance(page_items, list): + raise TypeError(f"Trakt {path} returned a non-list response") + items.extend(page_items) + + if not page_items: + return items + try: + page_count = int(response.headers.get("X-Pagination-Page-Count", page)) + except (TypeError, ValueError): + page_count = page + if page >= page_count: + return items + page += 1 def _headers(client_id: str, access_token: Optional[str] = None) -> dict: @@ -127,12 +161,11 @@ async def get_watched_movies(client_id: str, access_token: str) -> list[dict]: Returns list of: {plays, last_watched_at, movie: {title, ids: {tmdb, ...}}} """ async with httpx.AsyncClient(timeout=60.0) as client: - resp = await client.get( - f"{TRAKT_BASE}/sync/watched/movies", - headers=_headers(client_id, access_token), + return await _get_all_pages( + client, + "/sync/watched/movies", + _headers(client_id, access_token), ) - resp.raise_for_status() - return resp.json() async def get_watched_shows(client_id: str, access_token: str) -> list[dict]: @@ -141,12 +174,12 @@ async def get_watched_shows(client_id: str, access_token: str) -> list[dict]: Returns list of: {show: {title, ids: {tmdb, ...}}, seasons: [{number, episodes: [{number, plays, last_watched_at}]}]} """ async with httpx.AsyncClient(timeout=120.0) as client: - resp = await client.get( - f"{TRAKT_BASE}/sync/watched/shows", - headers=_headers(client_id, access_token), + return await _get_all_pages( + client, + "/sync/watched/shows", + _headers(client_id, access_token), + extra_params={"extended": "progress"}, ) - resp.raise_for_status() - return resp.json() async def get_ratings(client_id: str, access_token: str) -> dict: diff --git a/backend/main.py b/backend/main.py index ea12fbe..231f112 100644 --- a/backend/main.py +++ b/backend/main.py @@ -6,7 +6,7 @@ from sqlalchemy.ext.asyncio import AsyncSession from db import engine, Base import models # noqa: F401 -from routers import webhooks, media, history, ratings, sync, shows, auth, lists, oidc, profile, trakt, simkl, comments, admin, compat +from routers import webhooks, media, history, ratings, sync, shows, auth, lists, oidc, profile, trakt, simkl, mdblist, comments, admin, compat from slowapi import _rate_limit_exceeded_handler from slowapi.errors import RateLimitExceeded @@ -22,13 +22,13 @@ async def _auto_sync_scheduler(): from db import async_sessionmaker from models.connections import MediaServerConnection - from routers.sync import run_jellyfin_sync, run_emby_sync, run_plex_sync + from routers.sync import run_jellyfin_sync, run_emby_sync, run_plex_sync, run_nuvio_sync from datetime import datetime, timezone CHECK_INTERVAL = 300 # seconds between scheduler ticks - source_map = {"jellyfin": CollectionSource.jellyfin, "emby": CollectionSource.emby, "plex": CollectionSource.plex} - runner_map = {"jellyfin": run_jellyfin_sync, "emby": run_emby_sync, "plex": run_plex_sync} + source_map = {"jellyfin": CollectionSource.jellyfin, "emby": CollectionSource.emby, "plex": CollectionSource.plex, "nuvio": CollectionSource.nuvio} + runner_map = {"jellyfin": run_jellyfin_sync, "emby": run_emby_sync, "plex": run_plex_sync, "nuvio": run_nuvio_sync} while True: await asyncio.sleep(CHECK_INTERVAL) @@ -51,11 +51,12 @@ async def _auto_sync_scheduler(): if not source or not run_fn: continue - # Skip if a sync is already pending or running for this user+source + # Skip if a sync is already pending or running for this connection active_q = await db.execute( select(SyncJob).where( SyncJob.user_id == user_id, SyncJob.source == source, + SyncJob.connection_id == conn.id, SyncJob.status.in_([SyncStatus.pending, SyncStatus.running]), ) ) @@ -67,6 +68,7 @@ async def _auto_sync_scheduler(): select(SyncJob).where( SyncJob.user_id == user_id, SyncJob.source == source, + SyncJob.connection_id == conn.id, SyncJob.status.in_([SyncStatus.completed, SyncStatus.failed]), ).order_by(SyncJob.updated_at.desc()).limit(1) ) @@ -77,7 +79,7 @@ async def _auto_sync_scheduler(): if elapsed_hours < conn.auto_sync_interval: continue - job = SyncJob(user_id=user_id, source=source, status=SyncStatus.pending) + job = SyncJob(user_id=user_id, source=source, status=SyncStatus.pending, connection_id=conn.id) db.add(job) await db.flush() job_id = job.id @@ -315,6 +317,7 @@ async def lifespan(app: FastAPI): app.include_router(profile.router, prefix="/profile", tags=["profile"]) app.include_router(trakt.router, prefix="/trakt", tags=["trakt"]) app.include_router(simkl.router, prefix="/simkl", tags=["simkl"]) +app.include_router(mdblist.router, prefix="/mdblist", tags=["mdblist"]) app.include_router(comments.router, prefix="/comments", tags=["comments"]) app.include_router(admin.router, prefix="/admin", tags=["admin"]) app.include_router(compat.router, tags=["compat"]) diff --git a/backend/migrations/versions/s3t4u5v6w7x8_add_nuvio_provider.py b/backend/migrations/versions/s3t4u5v6w7x8_add_nuvio_provider.py new file mode 100644 index 0000000..8df6a0d --- /dev/null +++ b/backend/migrations/versions/s3t4u5v6w7x8_add_nuvio_provider.py @@ -0,0 +1,30 @@ +"""add Nuvio provider + +Revision ID: s3t4u5v6w7x8 +Revises: r2s3t4u5v6w7 +Create Date: 2026-07-14 +""" + +from alembic import op + + +revision = "s3t4u5v6w7x8" +down_revision = "r2s3t4u5v6w7" +branch_labels = None +depends_on = None + + +def upgrade() -> None: + op.execute("ALTER TYPE collectionsource ADD VALUE IF NOT EXISTS 'nuvio'") + op.drop_constraint("ck_msc_type", "media_server_connections", type_="check") + op.create_check_constraint( + "ck_msc_type", + "media_server_connections", + "type IN ('plex', 'jellyfin', 'emby', 'nuvio')", + ) + + +def downgrade() -> None: + # PostgreSQL enum values cannot be removed safely in-place. Keep the value and + # the compatible constraint; older application versions simply do not create it. + pass diff --git a/backend/migrations/versions/t4u5v6w7x8y9_allow_subhour_sync_intervals.py b/backend/migrations/versions/t4u5v6w7x8y9_allow_subhour_sync_intervals.py new file mode 100644 index 0000000..5ab53e5 --- /dev/null +++ b/backend/migrations/versions/t4u5v6w7x8y9_allow_subhour_sync_intervals.py @@ -0,0 +1,37 @@ +"""allow sub-hour connection sync intervals + +Revision ID: t4u5v6w7x8y9 +Revises: s3t4u5v6w7x8 +Create Date: 2026-07-14 +""" +from typing import Sequence, Union + +from alembic import op +import sqlalchemy as sa + +revision: str = "t4u5v6w7x8y9" +down_revision: Union[str, Sequence[str], None] = "s3t4u5v6w7x8" +branch_labels: Union[str, Sequence[str], None] = None +depends_on: Union[str, Sequence[str], None] = None + + +def upgrade() -> None: + op.alter_column( + "media_server_connections", + "auto_sync_interval", + existing_type=sa.Integer(), + type_=sa.Float(), + existing_nullable=True, + postgresql_using="auto_sync_interval::double precision", + ) + + +def downgrade() -> None: + op.alter_column( + "media_server_connections", + "auto_sync_interval", + existing_type=sa.Float(), + type_=sa.Integer(), + existing_nullable=True, + postgresql_using="GREATEST(1, ROUND(auto_sync_interval))::integer", + ) diff --git a/backend/migrations/versions/u5v6w7x8y9z0_add_push_playback.py b/backend/migrations/versions/u5v6w7x8y9z0_add_push_playback.py new file mode 100644 index 0000000..98b5cfe --- /dev/null +++ b/backend/migrations/versions/u5v6w7x8y9z0_add_push_playback.py @@ -0,0 +1,26 @@ +"""add outbound playback progress flag + +Revision ID: u5v6w7x8y9z0 +Revises: t4u5v6w7x8y9 +Create Date: 2026-07-14 +""" + +from alembic import op +import sqlalchemy as sa + + +revision = "u5v6w7x8y9z0" +down_revision = "t4u5v6w7x8y9" +branch_labels = None +depends_on = None + + +def upgrade() -> None: + op.add_column( + "media_server_connections", + sa.Column("push_playback", sa.Boolean(), nullable=False, server_default="false"), + ) + + +def downgrade() -> None: + op.drop_column("media_server_connections", "push_playback") diff --git a/backend/migrations/versions/v6w7x8y9z0a1_add_mdblist.py b/backend/migrations/versions/v6w7x8y9z0a1_add_mdblist.py new file mode 100644 index 0000000..d9c8e76 --- /dev/null +++ b/backend/migrations/versions/v6w7x8y9z0a1_add_mdblist.py @@ -0,0 +1,56 @@ +"""add MDBList synchronization settings + +Revision ID: v6w7x8y9z0a1 +Revises: u5v6w7x8y9z0 +Create Date: 2026-07-17 +""" + +from alembic import op +import sqlalchemy as sa + + +revision = "v6w7x8y9z0a1" +down_revision = "u5v6w7x8y9z0" +branch_labels = None +depends_on = None + + +def upgrade() -> None: + op.add_column("user_settings", sa.Column("mdblist_api_key", sa.String(255), nullable=True)) + op.add_column( + "user_settings", + sa.Column("mdblist_sync_watched", sa.Boolean(), nullable=False, server_default="true"), + ) + op.add_column( + "user_settings", + sa.Column("mdblist_sync_ratings", sa.Boolean(), nullable=False, server_default="true"), + ) + op.add_column( + "user_settings", + sa.Column("mdblist_sync_watchlist", sa.Boolean(), nullable=False, server_default="true"), + ) + op.add_column( + "user_settings", + sa.Column("mdblist_push_watched", sa.Boolean(), nullable=False, server_default="false"), + ) + op.add_column( + "user_settings", + sa.Column("mdblist_push_ratings", sa.Boolean(), nullable=False, server_default="false"), + ) + op.add_column( + "user_settings", + sa.Column("mdblist_push_watchlist", sa.Boolean(), nullable=False, server_default="false"), + ) + op.add_column("lists", sa.Column("mdblist_slug", sa.String(255), nullable=True)) + op.execute("ALTER TYPE collectionsource ADD VALUE IF NOT EXISTS 'mdblist'") + + +def downgrade() -> None: + op.drop_column("lists", "mdblist_slug") + op.drop_column("user_settings", "mdblist_push_watchlist") + op.drop_column("user_settings", "mdblist_push_ratings") + op.drop_column("user_settings", "mdblist_push_watched") + op.drop_column("user_settings", "mdblist_sync_watchlist") + op.drop_column("user_settings", "mdblist_sync_ratings") + op.drop_column("user_settings", "mdblist_sync_watched") + op.drop_column("user_settings", "mdblist_api_key") diff --git a/backend/models/base.py b/backend/models/base.py index adf4363..28086d7 100644 --- a/backend/models/base.py +++ b/backend/models/base.py @@ -19,8 +19,10 @@ class CollectionSource(str, enum.Enum): jellyfin = "jellyfin" emby = "emby" plex = "plex" + nuvio = "nuvio" trakt = "trakt" simkl = "simkl" + mdblist = "mdblist" manual = "manual" tmdb = "tmdb" diff --git a/backend/models/connections.py b/backend/models/connections.py index d91c7a0..2bdf1aa 100644 --- a/backend/models/connections.py +++ b/backend/models/connections.py @@ -1,7 +1,7 @@ from datetime import datetime from typing import Optional -from sqlalchemy import Boolean, DateTime, ForeignKey, Integer, String, func +from sqlalchemy import Boolean, DateTime, Float, ForeignKey, Integer, String, func from sqlalchemy.dialects.postgresql import JSONB from sqlalchemy.orm import Mapped, mapped_column @@ -28,10 +28,11 @@ class MediaServerConnection(Base): # Outbound push flags (Scrob → source) push_watched : Mapped[bool] = mapped_column(Boolean, nullable=False, default=False, server_default="false") + push_playback : Mapped[bool] = mapped_column(Boolean, nullable=False, default=False, server_default="false") push_ratings : Mapped[bool] = mapped_column(Boolean, nullable=False, default=False, server_default="false") # Auto sync interval in hours (null = disabled) - auto_sync_interval : Mapped[Optional[int]] = mapped_column(Integer, nullable=True) + auto_sync_interval : Mapped[Optional[float]] = mapped_column(Float, nullable=True) # Plex watchlist → Radarr/Sonarr auto-request (Plex connections only) watchlist_to_radarr : Mapped[bool] = mapped_column(Boolean, nullable=False, default=False, server_default="false") diff --git a/backend/models/lists.py b/backend/models/lists.py index f1b8296..33e9d6a 100644 --- a/backend/models/lists.py +++ b/backend/models/lists.py @@ -16,6 +16,7 @@ class List(Base): description : Mapped[Optional[str]] = mapped_column(Text) privacy_level : Mapped[PrivacyLevel] = mapped_column(SQLEnum(PrivacyLevel), default=PrivacyLevel.private, nullable=False, server_default=PrivacyLevel.private.value) trakt_slug : Mapped[Optional[str]] = mapped_column(String(255), nullable=True) + mdblist_slug : Mapped[Optional[str]] = mapped_column(String(255), nullable=True) created_at : Mapped[datetime] = mapped_column(DateTime, server_default=func.now(), nullable=False) updated_at : Mapped[datetime] = mapped_column(DateTime, server_default=func.now(), onupdate=func.now(), nullable=False) diff --git a/backend/models/users.py b/backend/models/users.py index 1ca72a9..03a21fe 100644 --- a/backend/models/users.py +++ b/backend/models/users.py @@ -115,6 +115,15 @@ class UserSettings(Base): use_hls_player : Mapped[bool] = mapped_column(Boolean, nullable=False, default=False, server_default="false") next_up_hidden_shows : Mapped[Optional[list[int]]] = mapped_column(JSONB, server_default="'[]'") + # MDBList — API key authentication + mdblist_api_key: Mapped[Optional[str]] = mapped_column(String(255)) + mdblist_sync_watched: Mapped[bool] = mapped_column(Boolean, default=True, server_default="true") + mdblist_sync_ratings: Mapped[bool] = mapped_column(Boolean, default=True, server_default="true") + mdblist_sync_watchlist: Mapped[bool] = mapped_column(Boolean, default=True, server_default="true") + mdblist_push_watched: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false") + mdblist_push_ratings: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false") + mdblist_push_watchlist: Mapped[bool] = mapped_column(Boolean, default=False, server_default="false") + user : Mapped["User"] = relationship(back_populates="settings") diff --git a/backend/routers/auth.py b/backend/routers/auth.py index fd40588..7ae4a1f 100644 --- a/backend/routers/auth.py +++ b/backend/routers/auth.py @@ -25,6 +25,7 @@ from core.url_validator import validate_service_url from core.limiter import limiter from core.backup import restore_backup +from core.nuvio import NuvioAPIError, parse_profile_id import schemas from dependencies import get_current_user from sqlalchemy.orm import selectinload @@ -43,6 +44,25 @@ def _generate_api_key() -> str: return secrets.token_urlsafe(32) router = APIRouter() +def _parse_nuvio_profile_id(value: str | None) -> int: + try: + return parse_profile_id(value) + except NuvioAPIError: + raise HTTPException(status_code=400, detail="Nuvio profile must be an integer from 1 to 6") + + +def _nuvio_profile_name(profiles: list[dict], profile_id: int) -> str: + for profile in profiles: + try: + matches = int(profile.get("profile_index") or 0) == profile_id + except (TypeError, ValueError): + continue + if matches: + name = str(profile.get("name") or "").strip() + return name or f"Profile {profile_id}" + return f"Profile {profile_id}" + + async def _registration_allowed(db: AsyncSession) -> bool: @@ -325,6 +345,7 @@ async def _settings_response(settings: UserSettings, db: AsyncSession) -> schema data = schemas.UserSettings.model_validate(settings) data.trakt_connected = bool(settings.trakt_access_token) data.simkl_connected = bool(settings.simkl_access_token) + data.mdblist_connected = bool(settings.mdblist_api_key) gs_result = await db.execute(select(GlobalSettings).where(GlobalSettings.id == 1)) gs = gs_result.scalar_one_or_none() data.has_global_tmdb_key = bool(gs and gs.tmdb_api_key) @@ -369,7 +390,7 @@ async def update_user_settings( db.add(settings) # Computed read-only fields; never write them back - READ_ONLY_FIELDS = {"trakt_connected", "simkl_connected", "has_global_tmdb_key", "has_effective_tmdb_key", "has_global_tvdb_key", "has_effective_tvdb_key"} + READ_ONLY_FIELDS = {"trakt_connected", "simkl_connected", "mdblist_connected", "has_global_tmdb_key", "has_effective_tmdb_key", "has_global_tvdb_key", "has_effective_tvdb_key"} update_data = {k: v for k, v in settings_in.model_dump(exclude_unset=True).items() if k not in READ_ONLY_FIELDS} if "tmdb_api_key" in update_data and update_data["tmdb_api_key"]: @@ -383,6 +404,11 @@ async def update_user_settings( if not success: raise HTTPException(status_code=400, detail="Invalid TVDB API Key") + if "mdblist_api_key" in update_data and update_data["mdblist_api_key"]: + from core import mdblist + if not await mdblist.validate_api_key(update_data["mdblist_api_key"]): + raise HTTPException(status_code=400, detail="Invalid MDBList API key") + url_fields = {"radarr_url": "Radarr URL", "sonarr_url": "Sonarr URL"} for field, label in url_fields.items(): if field in update_data and update_data[field]: @@ -418,23 +444,39 @@ async def create_connection( db: AsyncSession = Depends(get_db), current_user: User = Depends(get_current_user), ): - if body.type not in ("plex", "jellyfin", "emby"): - raise HTTPException(status_code=400, detail="type must be plex, jellyfin, or emby") + if body.type not in ("plex", "jellyfin", "emby", "nuvio"): + raise HTTPException(status_code=400, detail="type must be plex, jellyfin, emby, or nuvio") validated_url = await validate_service_url(body.url, f"{body.type.capitalize()} URL") + connection_token = body.token + server_user_id = body.server_user_id + server_username = body.server_username + if body.type == "nuvio": + from core import nuvio + + profile_id = _parse_nuvio_profile_id(server_user_id) + try: + session, profiles = await nuvio.validate_connection(validated_url, connection_token, profile_id) + except nuvio.NuvioAPIError as exc: + raise HTTPException(status_code=400, detail=str(exc)) + connection_token = session.refresh_token + server_user_id = str(profile_id) + server_username = _nuvio_profile_name(profiles, profile_id) + conn = MediaServerConnection( user_id=current_user.id, type=body.type, name=body.name, url=validated_url, - token=body.token, - server_user_id=body.server_user_id, - server_username=body.server_username, + token=connection_token, + server_user_id=server_user_id, + server_username=server_username, sync_collection=body.sync_collection, sync_watched=body.sync_watched, - sync_ratings=body.sync_ratings, + sync_ratings=body.sync_ratings if body.type != "nuvio" else False, sync_playback=body.sync_playback, push_watched=body.push_watched, - push_ratings=body.push_ratings, + push_playback=body.push_playback if body.type == "nuvio" else False, + push_ratings=body.push_ratings if body.type != "nuvio" else False, auto_sync_interval=body.auto_sync_interval, ) db.add(conn) @@ -464,6 +506,25 @@ async def update_connection( if "url" in update_data and update_data["url"]: update_data["url"] = await validate_service_url(update_data["url"], f"{conn.type.capitalize()} URL") + if conn.type == "nuvio": + from core import nuvio + + candidate_url = update_data.get("url", conn.url) + candidate_token = update_data.get("token", conn.token) + profile_id = _parse_nuvio_profile_id(update_data.get("server_user_id", conn.server_user_id)) + try: + session, profiles = await nuvio.validate_connection(candidate_url, candidate_token, profile_id) + except nuvio.NuvioAPIError as exc: + raise HTTPException(status_code=400, detail=str(exc)) + update_data["token"] = session.refresh_token + update_data["server_user_id"] = str(profile_id) + update_data["server_username"] = _nuvio_profile_name(profiles, profile_id) + update_data["sync_ratings"] = False + update_data["push_ratings"] = False + update_data["push_playback"] = update_data.get("push_playback", conn.push_playback) + else: + update_data["push_playback"] = False + for field, value in update_data.items(): setattr(conn, field, value) @@ -668,6 +729,46 @@ async def test_plex( raise HTTPException(status_code=400, detail="Failed to connect to Plex") return {"status": "ok"} + +@router.post("/nuvio-login") +async def login_nuvio( + body: schemas.NuvioLoginRequest, + current_user: User = Depends(get_current_user), +): + from core import nuvio + + url = await validate_service_url(body.url, "Nuvio URL") + try: + session, profiles = await nuvio.authenticate(url, body.email, body.password) + except nuvio.NuvioAPIError as exc: + raise HTTPException(status_code=400, detail=str(exc)) + return { + "url": url, + "refresh_token": session.refresh_token, + "profiles": profiles, + } + + +@router.post("/test-nuvio") +async def test_nuvio( + body: schemas.NuvioConnectionTestRequest, + current_user: User = Depends(get_current_user), +): + from core import nuvio + + url = await validate_service_url(body.url, "Nuvio URL") + profile_id = _parse_nuvio_profile_id(str(body.profile_id)) + try: + session, profiles = await nuvio.validate_connection(url, body.token, profile_id) + except nuvio.NuvioAPIError as exc: + raise HTTPException(status_code=400, detail=str(exc)) + return { + "status": "ok", + "message": "Nuvio connection is valid.", + "refresh_token": session.refresh_token, + "profiles": profiles, + } + @router.post("/test-radarr") async def test_radarr( url: str = Query(...), @@ -776,15 +877,34 @@ async def check_trakt(): return {"configured": True, "connected": connected} async def check_media_server(conn): - from core import jellyfin, plex + from core import jellyfin, nuvio, plex + try: if conn.type == "plex": connected = await plex.validate_connection(conn.url, conn.token) + elif conn.type == "nuvio": + profile_id = _parse_nuvio_profile_id(conn.server_user_id) + session, profiles = await nuvio.validate_connection(conn.url, conn.token, profile_id) + conn.token = session.refresh_token + conn.server_username = _nuvio_profile_name(profiles, profile_id) + connected = True else: connected = await jellyfin.validate_connection(conn.url, conn.token, conn.server_user_id) except Exception: connected = False - return {"id": conn.id, "connected": connected} + result = {"id": conn.id, "connected": connected} + if conn.type == "nuvio" and connected: + result["token"] = conn.token + result["profile_name"] = conn.server_username + result["profiles"] = [ + { + "profile_index": profile.get("profile_index"), + "name": str(profile.get("name") or "").strip() + or f"Profile {profile.get('profile_index')}", + } + for profile in profiles + ] + return result async def check_simkl(): from core import simkl as simkl_client @@ -793,12 +913,21 @@ async def check_simkl(): connected = await simkl_client.validate_token(user_settings.simkl_client_id, user_settings.simkl_access_token) return {"configured": True, "connected": connected} + async def check_mdblist(): + from core import mdblist + if not user_settings or not user_settings.mdblist_api_key: + return {"configured": False, "connected": False} + connected = await mdblist.validate_api_key(user_settings.mdblist_api_key) + return {"configured": True, "connected": connected} + media_server_tasks = [check_media_server(c) for c in media_server_conns] - rdr_status, snr_status, trakt_status, simkl_status, *ms_statuses = await asyncio.gather( - check_radarr(), check_sonarr(), check_trakt(), check_simkl(), *media_server_tasks + rdr_status, snr_status, trakt_status, simkl_status, mdblist_status, *ms_statuses = await asyncio.gather( + check_radarr(), check_sonarr(), check_trakt(), check_simkl(), check_mdblist(), *media_server_tasks ) + if any(conn.type == "nuvio" for conn in media_server_conns): + await db.commit() - return {"radarr": rdr_status, "sonarr": snr_status, "trakt": trakt_status, "simkl": simkl_status, "connections": ms_statuses} + return {"radarr": rdr_status, "sonarr": snr_status, "trakt": trakt_status, "simkl": simkl_status, "mdblist": mdblist_status, "connections": ms_statuses} @router.get("/sonarr/profiles") diff --git a/backend/routers/history.py b/backend/routers/history.py index 3a143f4..f2e9fc8 100644 --- a/backend/routers/history.py +++ b/backend/routers/history.py @@ -1,5 +1,6 @@ import asyncio -from datetime import datetime +import logging +from datetime import datetime, timezone from fastapi import APIRouter, Depends, HTTPException, Query from sqlalchemy.ext.asyncio import AsyncSession from sqlalchemy import and_, or_, select, desc, func, delete @@ -23,8 +24,10 @@ import core.jellyfin as jellyfin_client import core.emby as emby_client import core.trakt as trakt_client +import core.nuvio as nuvio_client router = APIRouter() +logger = logging.getLogger(__name__) async def _push_watch_state( @@ -48,6 +51,7 @@ async def _push_watch_state( settings_result = await db.execute(select(UserSettings).where(UserSettings.user_id == user_id)) settings = settings_result.scalar_one_or_none() push_trakt = settings and settings.trakt_push_watched and settings.trakt_access_token + push_mdblist = settings and settings.mdblist_push_watched and settings.mdblist_api_key tasks = [] @@ -131,9 +135,84 @@ async def _push_watch_state( else: tasks.append(trakt_client.remove_episode_from_history(settings.trakt_client_id, settings.trakt_access_token, show.tmdb_id, media.season_number, media.episode_number)) + if push_mdblist: + from core import mdblist as mdblist_client + from routers.mdblist import _empty_payload, _payload_item + from routers.sync import _latest_watched_at + + mdblist_watched_at: dict[int, datetime] = {} + if watched: + mdblist_watched_at = await _latest_watched_at(db, user_id, media_ids) + + mdblist_payload = _empty_payload() + media_result = await db.execute(select(Media).where(Media.id.in_(media_ids))) + for media in media_result.scalars().all(): + item = _payload_item(media, watched_at=mdblist_watched_at.get(media.id, datetime.utcnow())) if watched else _payload_item(media) + if item: + mdblist_payload[item[0]].append(item[1]) + operation = mdblist_client.push_watched if watched else mdblist_client.remove_watched + tasks.append(operation(settings.mdblist_api_key, mdblist_payload)) + if tasks: await asyncio.gather(*tasks, return_exceptions=True) + nuvio_connections = [conn for conn in connections if conn.type == "nuvio"] + if nuvio_connections: + media_result = await db.execute(select(Media).where(Media.id.in_(media_ids))) + media_items = media_result.scalars().all() + show_ids = {media.show_id for media in media_items if media.show_id is not None} + shows_by_id: dict[int, Show] = {} + if show_ids: + show_result = await db.execute(select(Show).where(Show.id.in_(show_ids))) + shows_by_id = {show.id: show for show in show_result.scalars().all()} + from routers.sync import _ensure_nuvio_imdb_ids, _latest_watched_at, _nuvio_watched_item + + api_key = await get_user_tmdb_key(db, user_id) + await _ensure_nuvio_imdb_ids(media_items, shows_by_id, api_key) + + watched_at_by_media: dict[int, datetime] = {} + if watched: + watched_at_by_media = await _latest_watched_at(db, user_id, media_ids) + + nuvio_items: list[dict] = [] + nuvio_keys: list[dict] = [] + for media in media_items: + payload = _nuvio_watched_item( + media, + watched_at_by_media.get(media.id, datetime.utcnow()), + shows_by_id.get(media.show_id), + ) + if not payload: + continue + key = { + field: payload[field] + for field in ("content_id", "season", "episode") + if field in payload + } + if watched: + nuvio_items.append(payload) + else: + nuvio_keys.append(key) + + for conn in nuvio_connections: + try: + profile_id = nuvio_client.parse_profile_id(conn.server_user_id) + if watched and nuvio_items: + session = await nuvio_client.push_watched_items( + conn.url, conn.token, profile_id, nuvio_items + ) + elif not watched and nuvio_keys: + session = await nuvio_client.delete_watched_items( + conn.url, conn.token, profile_id, nuvio_keys + ) + else: + continue + conn.token = session.refresh_token + except Exception: + logger.exception("Failed to push watch state to Nuvio connection %s", conn.id) + continue + await db.commit() + def format_event(event: WatchEvent | PlaybackProgress, media: Media) -> dict: # Handle both WatchEvent (history) and PlaybackProgress (continue watching) diff --git a/backend/routers/lists.py b/backend/routers/lists.py index eb4f78f..80c1613 100644 --- a/backend/routers/lists.py +++ b/backend/routers/lists.py @@ -362,6 +362,42 @@ async def _push_list_item_to_trakt( logger.warning("Failed to push list item to Trakt (slug=%s, remove=%s): %s", list_trakt_slug, remove, exc) + +async def _push_list_item_to_mdblist( + db: AsyncSession, + user_id: int, + list_mdblist_slug: str, + media: Media, + remove: bool = False, +) -> None: + if ( + list_mdblist_slug != "__watchlist__" + or not media.tmdb_id + or media.media_type not in (MediaType.movie, MediaType.series) + ): + return + + settings_result = await db.execute(select(UserSettings).where(UserSettings.user_id == user_id)) + settings = settings_result.scalar_one_or_none() + if not settings or not settings.mdblist_push_watchlist or not settings.mdblist_api_key: + return + + from core import mdblist as mdblist_client + + kind = "movies" if media.media_type == MediaType.movie else "shows" + payload = {"movies": [], "shows": [], "seasons": [], "episodes": []} + payload[kind].append({"ids": {"tmdb": media.tmdb_id}}) + try: + operation = mdblist_client.remove_watchlist if remove else mdblist_client.push_watchlist + await operation(settings.mdblist_api_key, payload) + except Exception as exc: + logger.warning( + "Failed to push list item to MDBList watchlist (remove=%s): %s", + remove, + exc, + ) + + @router.post("/{list_id}/items", status_code=201) async def add_list_item( list_id: int, @@ -456,6 +492,11 @@ async def add_list_item( if lst.trakt_slug == "__plex_watchlist__": await _push_list_item_to_plex_watchlist(db, current_user.id, media, remove=False) + if lst.mdblist_slug: + await _push_list_item_to_mdblist( + db, current_user.id, lst.mdblist_slug, media, remove=False + ) + item_result = await db.execute( select(ListItem) .options(selectinload(ListItem.media).selectinload(Media.show)) @@ -501,4 +542,9 @@ async def remove_list_item( if lst.trakt_slug == "__plex_watchlist__": await _push_list_item_to_plex_watchlist(db, current_user.id, media, remove=True) + if lst and lst.mdblist_slug and media: + await _push_list_item_to_mdblist( + db, current_user.id, lst.mdblist_slug, media, remove=True + ) + return {"message": "Item removed"} diff --git a/backend/routers/mdblist.py b/backend/routers/mdblist.py new file mode 100644 index 0000000..f668fb9 --- /dev/null +++ b/backend/routers/mdblist.py @@ -0,0 +1,697 @@ +"""MDBList cloud synchronization endpoints.""" + +from __future__ import annotations + +import logging +from datetime import datetime, timezone +from typing import Any + +from dateutil import parser as dt_parser +from fastapi import APIRouter, BackgroundTasks, Depends, HTTPException +from sqlalchemy import delete, func, select, update +from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker + +from core import mdblist as mdblist_client +from core.enrichment import enrich_media +from db import engine, get_db +from dependencies import get_current_user +from models.base import CollectionSource, MediaType +from models.events import WatchEvent +from models.lists import List as ListModel, ListItem +from models.media import Media +from models.ratings import Rating +from models.show import Show +from models.sync import SyncJob, SyncStatus +from models.users import User, UserSettings +from routers.trakt import ( + _get_or_create_episode_media, + _get_or_create_movie_media, + _get_or_create_show, +) + +logger = logging.getLogger(__name__) +router = APIRouter() +WATCHLIST_SLUG = "__watchlist__" + + +def _utc_naive(value: Any) -> datetime: + if isinstance(value, datetime): + parsed = value + elif isinstance(value, str) and value: + try: + parsed = dt_parser.isoparse(value) + except (TypeError, ValueError): + return datetime.utcnow() + else: + return datetime.utcnow() + if parsed.tzinfo: + return parsed.astimezone(timezone.utc).replace(tzinfo=None) + return parsed + + +def _iso_utc(value: datetime | None) -> str: + value = value or datetime.utcnow() + if value.tzinfo is None: + value = value.replace(tzinfo=timezone.utc) + else: + value = value.astimezone(timezone.utc) + return value.isoformat().replace("+00:00", "Z") + + +def _integer(value: Any) -> int | None: + try: + return int(value) if value is not None else None + except (TypeError, ValueError): + return None + + +def _entry_data(kind: str, entry: dict[str, Any]) -> dict[str, Any]: + singular = {"movies": "movie", "shows": "show", "seasons": "season", "episodes": "episode"}[kind] + nested = entry.get(singular) + return nested if isinstance(nested, dict) else entry + + +def _tmdb_id(data: dict[str, Any]) -> int | None: + ids = data.get("ids") + ids = ids if isinstance(ids, dict) else {} + return _integer(ids.get("tmdb") or data.get("tmdb_id")) + +async def _resolve_external_tmdb_id( + data: dict[str, Any], + media_type: str, + api_key: str | None, + cache: dict[tuple[str, str], int | None], +) -> int | None: + direct_id = _tmdb_id(data) + if direct_id: + return direct_id + + ids = data.get("ids") + ids = ids if isinstance(ids, dict) else {} + from core import tmdb + + for provider, external_source in (("imdb", "imdb_id"), ("tvdb", "tvdb_id")): + external_id = ids.get(provider) or data.get(f"{provider}_id") + if external_id is None: + continue + cache_key = (external_source, str(external_id)) + if cache_key in cache: + return cache[cache_key] + try: + result = await tmdb.find_by_external_id( + str(external_id), + external_source, + api_key=api_key, + ) + result_key = "movie_results" if media_type == "movie" else "tv_results" + matches = result.get(result_key) or [] + resolved = _integer(matches[0].get("id")) if matches else None + except Exception as exc: + logger.warning( + "Could not resolve MDBList %s=%s through TMDB: %s", + provider, + external_id, + exc, + ) + resolved = None + cache[cache_key] = resolved + if resolved: + return resolved + return None + + + + +def _episode_identity(entry: dict[str, Any]) -> tuple[int | None, int | None, int | None, str]: + episode = _entry_data("episodes", entry) + show_data = entry.get("show") or episode.get("show") or {} + show_data = show_data if isinstance(show_data, dict) else {} + show_tmdb_id = _tmdb_id(show_data) + ids = episode.get("ids") if isinstance(episode.get("ids"), dict) else {} + show_tmdb_id = show_tmdb_id or _integer(ids.get("show_tmdb") or episode.get("show_tmdb_id")) + + season = episode.get("season", entry.get("season")) + if isinstance(season, dict): + season = season.get("number") + episode_number = episode.get("number", episode.get("episode", entry.get("episode"))) + title = str(episode.get("title") or episode.get("name") or "") + return show_tmdb_id, _integer(season), _integer(episode_number), title + + +async def _get_or_create_series_media( + db: AsyncSession, + tmdb_id: int, + title: str, + api_key: str | None, +) -> Media | None: + result = await db.execute( + select(Media).where(Media.tmdb_id == tmdb_id, Media.media_type == MediaType.series) + ) + media = result.scalars().first() + if media: + return media + + from core import tmdb + + try: + data = await tmdb.get_show(tmdb_id, api_key=api_key) + media = Media( + tmdb_id=tmdb_id, + media_type=MediaType.series, + title=data.get("name") or title, + ) + db.add(media) + await db.flush() + await enrich_media(media, api_key=api_key) + return media + except Exception as exc: + logger.warning("Could not fetch MDBList show tmdb=%s: %s", tmdb_id, exc) + return None + + +async def _resolve_media( + db: AsyncSession, + kind: str, + entry: dict[str, Any], + api_key: str | None, + external_cache: dict[tuple[str, str], int | None], +) -> Media | None: + data = _entry_data(kind, entry) + title = str(data.get("title") or data.get("name") or "") + if kind == "movies": + tmdb_id = await _resolve_external_tmdb_id(data, "movie", api_key, external_cache) + return await _get_or_create_movie_media(db, tmdb_id, title, api_key) if tmdb_id else None + if kind == "shows": + tmdb_id = await _resolve_external_tmdb_id(data, "tv", api_key, external_cache) + return await _get_or_create_series_media(db, tmdb_id, title, api_key) if tmdb_id else None + if kind == "episodes": + show_tmdb_id, season, episode, _ = _episode_identity(entry) + if show_tmdb_id is None: + episode_data = _entry_data("episodes", entry) + show_data = entry.get("show") or episode_data.get("show") or {} + if isinstance(show_data, dict): + show_tmdb_id = await _resolve_external_tmdb_id( + show_data, "tv", api_key, external_cache + ) + if show_tmdb_id is None or season is None or episode is None: + return None + show = await _get_or_create_show(db, show_tmdb_id, "", api_key) + if not show: + return None + return await _get_or_create_episode_media( + db, show.id, show_tmdb_id, season, episode, api_key + ) + return None + + +def _empty_payload() -> dict[str, list[dict[str, Any]]]: + return {"movies": [], "shows": [], "seasons": [], "episodes": []} + + +def _payload_item( + media: Media, + *, + watched_at: datetime | None = None, + rating: float | None = None, + rated_at: datetime | None = None, +) -> tuple[str, dict[str, Any]] | None: + if not media.tmdb_id: + return None + if media.media_type == MediaType.movie: + kind = "movies" + elif media.media_type == MediaType.series: + kind = "shows" + elif media.media_type == MediaType.episode: + kind = "episodes" + else: + return None + + item: dict[str, Any] = {"ids": {"tmdb": media.tmdb_id}} + if watched_at is not None: + item["watched_at"] = _iso_utc(watched_at) + if rating is not None: + item["rating"] = float(rating) + item["rated_at"] = _iso_utc(rated_at or datetime.now(timezone.utc)) + return kind, item + + +async def _effective_tmdb_key(db: AsyncSession, settings: UserSettings) -> str | None: + from models.global_settings import GlobalSettings + + if settings.tmdb_api_key: + return settings.tmdb_api_key + result = await db.execute(select(GlobalSettings).where(GlobalSettings.id == 1)) + global_settings = result.scalar_one_or_none() + return global_settings.tmdb_api_key if global_settings else None + + +async def _import_watched( + db: AsyncSession, + user_id: int, + payload: dict[str, Any], + api_key: str | None, + external_cache: dict[tuple[str, str], int | None], + stats: dict[str, int], +) -> set[int]: + existing_result = await db.execute( + select(WatchEvent.media_id).where( + WatchEvent.user_id == user_id, + WatchEvent.completed.is_(True), + ) + ) + existing = {row[0] for row in existing_result.all()} + changed: set[int] = set() + + for kind in ("movies", "shows", "episodes"): + for entry in payload.get(kind, []): + try: + async with db.begin_nested(): + media = await _resolve_media(db, kind, entry, api_key, external_cache) + if not media: + stats["skipped"] += 1 + continue + if media.id in existing: + stats["skipped"] += 1 + continue + watched_at = entry.get("watched_at") or entry.get("last_watched_at") + db.add( + WatchEvent( + user_id=user_id, + media_id=media.id, + watched_at=_utc_naive(watched_at), + completed=True, + play_count=max(_integer(entry.get("plays")) or 1, 1), + ) + ) + existing.add(media.id) + changed.add(media.id) + stats["watched"] += 1 + except Exception as exc: + logger.warning("Error importing MDBList %s watch item: %s", kind, exc) + stats["errors"] += 1 + + stats["skipped"] += len(payload.get("seasons", [])) + return changed + + +async def _import_ratings( + db: AsyncSession, + user_id: int, + payload: dict[str, Any], + api_key: str | None, + external_cache: dict[tuple[str, str], int | None], + stats: dict[str, int], +) -> dict[int, float]: + ratings_result = await db.execute( + select(Rating).where(Rating.user_id == user_id, Rating.season_number.is_(None)) + ) + existing = {rating.media_id: rating for rating in ratings_result.scalars().all()} + changed: dict[int, float] = {} + + for kind in ("movies", "shows", "episodes"): + for entry in payload.get(kind, []): + rating_value = entry.get("rating") + try: + rating = float(rating_value) + except (TypeError, ValueError): + stats["skipped"] += 1 + continue + try: + async with db.begin_nested(): + media = await _resolve_media(db, kind, entry, api_key, external_cache) + if not media: + stats["skipped"] += 1 + continue + current = existing.get(media.id) + if current and current.rating == rating: + stats["skipped"] += 1 + continue + if current: + current.rating = rating + current.rated_at = _utc_naive(entry.get("rated_at")) + else: + current = Rating( + user_id=user_id, + media_id=media.id, + rating=rating, + rated_at=_utc_naive(entry.get("rated_at")), + ) + db.add(current) + existing[media.id] = current + changed[media.id] = rating + stats["ratings"] += 1 + except Exception as exc: + logger.warning("Error importing MDBList %s rating: %s", kind, exc) + stats["errors"] += 1 + + stats["skipped"] += len(payload.get("seasons", [])) + return changed + + +async def _import_watchlist( + db: AsyncSession, + user_id: int, + payload: dict[str, Any], + api_key: str | None, + external_cache: dict[tuple[str, str], int | None], + stats: dict[str, int], +) -> None: + list_result = await db.execute( + select(ListModel).where( + ListModel.user_id == user_id, + ListModel.mdblist_slug == WATCHLIST_SLUG, + ) + ) + watchlist = list_result.scalar_one_or_none() + if not watchlist: + watchlist = ListModel( + user_id=user_id, + name="MDBList - Watchlist", + mdblist_slug=WATCHLIST_SLUG, + ) + db.add(watchlist) + await db.flush() + stats["lists"] += 1 + + existing_result = await db.execute( + select(ListItem.media_id).where(ListItem.list_id == watchlist.id) + ) + existing = {row[0] for row in existing_result.all()} + remote_ids: set[int] = set() + + for kind in ("movies", "shows"): + for entry in payload.get(kind, []): + try: + async with db.begin_nested(): + media = await _resolve_media(db, kind, entry, api_key, external_cache) + if not media: + stats["skipped"] += 1 + continue + remote_ids.add(media.id) + if media.id not in existing: + db.add(ListItem(list_id=watchlist.id, media_id=media.id)) + existing.add(media.id) + stats["watchlist_added"] += 1 + except Exception as exc: + logger.warning("Error importing MDBList watchlist %s: %s", kind, exc) + stats["errors"] += 1 + + stale = existing - remote_ids + if stale: + await db.execute( + delete(ListItem).where( + ListItem.list_id == watchlist.id, + ListItem.media_id.in_(stale), + ) + ) + stats["watchlist_removed"] += len(stale) + + +async def run_mdblist_sync(user_id: int, job_id: int) -> None: + session_factory = async_sessionmaker(engine, expire_on_commit=False, class_=AsyncSession) + async with session_factory() as db: + try: + await db.execute( + update(SyncJob).where(SyncJob.id == job_id).values(status=SyncStatus.running) + ) + await db.commit() + + settings_result = await db.execute( + select(UserSettings).where(UserSettings.user_id == user_id) + ) + settings = settings_result.scalar_one_or_none() + if not settings or not settings.mdblist_api_key: + raise RuntimeError("MDBList API key is not configured") + + requests = [] + labels = [] + if settings.mdblist_sync_watched: + labels.append("watched") + requests.append(mdblist_client.get_watched(settings.mdblist_api_key)) + if settings.mdblist_sync_ratings: + labels.append("ratings") + requests.append(mdblist_client.get_ratings(settings.mdblist_api_key)) + if settings.mdblist_sync_watchlist: + labels.append("watchlist") + requests.append(mdblist_client.get_watchlist(settings.mdblist_api_key)) + + import asyncio + + responses = await asyncio.gather(*requests) + snapshots = dict(zip(labels, responses, strict=True)) + total_items = sum( + len(values) + for snapshot in snapshots.values() + for values in snapshot.values() + if isinstance(values, list) + ) + await db.execute( + update(SyncJob).where(SyncJob.id == job_id).values(total_items=total_items) + ) + await db.commit() + + tmdb_key = await _effective_tmdb_key(db, settings) + stats = { + "watched": 0, + "ratings": 0, + "lists": 0, + "watchlist_added": 0, + "watchlist_removed": 0, + "skipped": 0, + "errors": 0, + } + new_watched: set[int] = set() + new_ratings: dict[int, float] = {} + external_cache: dict[tuple[str, str], int | None] = {} + + if "watched" in snapshots: + new_watched = await _import_watched( + db, user_id, snapshots["watched"], tmdb_key, external_cache, stats + ) + if "ratings" in snapshots: + new_ratings = await _import_ratings( + db, user_id, snapshots["ratings"], tmdb_key, external_cache, stats + ) + if "watchlist" in snapshots: + await _import_watchlist( + db, user_id, snapshots["watchlist"], tmdb_key, external_cache, stats + ) + await db.commit() + + from routers.sync import _fan_out_changes_to_other_connections + + await _fan_out_changes_to_other_connections( + db, + user_id, + None, + new_watched, + new_ratings, + settings=settings, + exclude_cloud_source=CollectionSource.mdblist, + ) + await db.execute( + update(SyncJob).where(SyncJob.id == job_id).values( + status=SyncStatus.completed, + processed_items=total_items, + errors=stats["errors"], + stats=stats, + ) + ) + await db.commit() + except Exception as exc: + logger.exception("MDBList pull job %s failed", job_id) + await db.rollback() + await db.execute( + update(SyncJob).where(SyncJob.id == job_id).values( + status=SyncStatus.failed, + error_message=str(exc), + ) + ) + await db.commit() + + +async def _load_payload_media(db: AsyncSession, media_ids: set[int]) -> dict[int, Media]: + if not media_ids: + return {} + from routers.sync import _select_in_chunks + + media = await _select_in_chunks( + db, + lambda chunk: select(Media).where(Media.id.in_(chunk)), + list(media_ids), + ) + return {item.id: item for item in media} + + +async def run_mdblist_push(user_id: int, job_id: int) -> None: + session_factory = async_sessionmaker(engine, expire_on_commit=False, class_=AsyncSession) + async with session_factory() as db: + try: + await db.execute( + update(SyncJob).where(SyncJob.id == job_id).values(status=SyncStatus.running) + ) + await db.commit() + + settings_result = await db.execute( + select(UserSettings).where(UserSettings.user_id == user_id) + ) + settings = settings_result.scalar_one_or_none() + if not settings or not settings.mdblist_api_key: + raise RuntimeError("MDBList API key is not configured") + + watched_rows: list[tuple[int, datetime]] = [] + rating_rows: list[tuple[int, float, datetime | None]] = [] + watchlist_ids: set[int] = set() + + if settings.mdblist_push_watched: + watched_result = await db.execute( + select(WatchEvent.media_id, func.max(WatchEvent.watched_at)) + .where(WatchEvent.user_id == user_id, WatchEvent.completed.is_(True)) + .group_by(WatchEvent.media_id) + ) + watched_rows = list(watched_result.all()) + if settings.mdblist_push_ratings: + ratings_result = await db.execute( + select(Rating.media_id, Rating.rating, Rating.rated_at).where( + Rating.user_id == user_id, + Rating.rating.isnot(None), + Rating.season_number.is_(None), + ) + ) + rating_rows = [ + (media_id, float(rating), rated_at) + for media_id, rating, rated_at in ratings_result.all() + ] + if settings.mdblist_push_watchlist: + watchlist_result = await db.execute( + select(ListModel).where( + ListModel.user_id == user_id, + ListModel.mdblist_slug == WATCHLIST_SLUG, + ) + ) + watchlist = watchlist_result.scalar_one_or_none() + if watchlist: + item_result = await db.execute( + select(ListItem.media_id).where(ListItem.list_id == watchlist.id) + ) + watchlist_ids = {row[0] for row in item_result.all()} + + all_ids = {row[0] for row in watched_rows} | {row[0] for row in rating_rows} | watchlist_ids + media_by_id = await _load_payload_media(db, all_ids) + watched_payload = _empty_payload() + ratings_payload = _empty_payload() + watchlist_payload = _empty_payload() + + for media_id, watched_at in watched_rows: + media = media_by_id.get(media_id) + item = _payload_item(media, watched_at=watched_at) if media else None + if item: + watched_payload[item[0]].append(item[1]) + for media_id, rating, rated_at in rating_rows: + media = media_by_id.get(media_id) + item = _payload_item(media, rating=rating, rated_at=rated_at) if media else None + if item: + ratings_payload[item[0]].append(item[1]) + for media_id in watchlist_ids: + media = media_by_id.get(media_id) + item = _payload_item(media) if media else None + if item and item[0] in ("movies", "shows"): + watchlist_payload[item[0]].append(item[1]) + + total_items = sum( + len(values) + for payload in (watched_payload, ratings_payload, watchlist_payload) + for values in payload.values() + ) + await db.execute( + update(SyncJob).where(SyncJob.id == job_id).values(total_items=total_items) + ) + await db.commit() + + results: dict[str, Any] = {} + if settings.mdblist_push_watched: + results["watched"] = await mdblist_client.push_watched( + settings.mdblist_api_key, watched_payload + ) + if settings.mdblist_push_ratings: + results["ratings"] = await mdblist_client.push_ratings( + settings.mdblist_api_key, ratings_payload + ) + if settings.mdblist_push_watchlist: + results["watchlist"] = await mdblist_client.push_watchlist( + settings.mdblist_api_key, watchlist_payload + ) + + submitted = sum(result["submitted"] for result in results.values()) + not_found = sum(result["not_found"] for result in results.values()) + await db.execute( + update(SyncJob).where(SyncJob.id == job_id).values( + status=SyncStatus.completed, + processed_items=submitted, + errors=not_found, + stats={"submitted": submitted, "not_found": not_found, "targets": results}, + ) + ) + await db.commit() + except Exception as exc: + logger.exception("MDBList push job %s failed", job_id) + await db.rollback() + await db.execute( + update(SyncJob).where(SyncJob.id == job_id).values( + status=SyncStatus.failed, + error_message=str(exc), + ) + ) + await db.commit() + + +def _require_key(settings: UserSettings | None) -> UserSettings: + if not settings or not settings.mdblist_api_key: + raise HTTPException( + status_code=400, + detail="Configure a valid MDBList API key in Settings first", + ) + return settings + + +@router.post("/sync") +async def sync_mdblist( + background_tasks: BackgroundTasks, + db: AsyncSession = Depends(get_db), + current_user: User = Depends(get_current_user), +): + result = await db.execute(select(UserSettings).where(UserSettings.user_id == current_user.id)) + settings = _require_key(result.scalar_one_or_none()) + if not any((settings.mdblist_sync_watched, settings.mdblist_sync_ratings, settings.mdblist_sync_watchlist)): + raise HTTPException(status_code=400, detail="Enable at least one MDBList pull option") + + job = SyncJob(user_id=current_user.id, source=CollectionSource.mdblist, status=SyncStatus.pending) + db.add(job) + await db.commit() + await db.refresh(job) + background_tasks.add_task(run_mdblist_sync, current_user.id, job.id) + return {"status": "started", "job_id": job.id, "message": "MDBList sync is running in the background"} + + +@router.post("/push") +async def push_mdblist( + background_tasks: BackgroundTasks, + db: AsyncSession = Depends(get_db), + current_user: User = Depends(get_current_user), +): + result = await db.execute(select(UserSettings).where(UserSettings.user_id == current_user.id)) + settings = _require_key(result.scalar_one_or_none()) + if not any((settings.mdblist_push_watched, settings.mdblist_push_ratings, settings.mdblist_push_watchlist)): + raise HTTPException(status_code=400, detail="Enable at least one MDBList push option") + + job = SyncJob( + user_id=current_user.id, + source=CollectionSource.mdblist, + status=SyncStatus.pending, + job_type="push", + ) + db.add(job) + await db.commit() + await db.refresh(job) + background_tasks.add_task(run_mdblist_push, current_user.id, job.id) + return {"status": "started", "job_id": job.id, "message": "MDBList push is running in the background"} diff --git a/backend/routers/media.py b/backend/routers/media.py index 094deb5..265fa77 100644 --- a/backend/routers/media.py +++ b/backend/routers/media.py @@ -12,7 +12,7 @@ from fastapi.responses import StreamingResponse, Response from starlette.background import BackgroundTask from sqlalchemy.ext.asyncio import AsyncSession -from sqlalchemy import select, or_, and_, func, cast as sa_cast, Text +from sqlalchemy import select, or_, and_, delete, func, cast as sa_cast, Text from sqlalchemy.orm import joinedload from db import get_db, AsyncSessionLocal diff --git a/backend/routers/ratings.py b/backend/routers/ratings.py index 3670e71..a95eb57 100644 --- a/backend/routers/ratings.py +++ b/backend/routers/ratings.py @@ -172,6 +172,15 @@ async def submit_rating( push_tasks.append(simkl_client.set_movie_rating(settings.simkl_client_id, settings.simkl_access_token, media.tmdb_id, body.rating)) elif media_type == MediaType.series: push_tasks.append(simkl_client.set_show_rating(settings.simkl_client_id, settings.simkl_access_token, media.tmdb_id, body.rating)) + if settings and settings.mdblist_push_ratings and settings.mdblist_api_key: + from core import mdblist as mdblist_client + from routers.mdblist import _empty_payload, _payload_item + + payload = _empty_payload() + item = _payload_item(media, rating=body.rating) + if item: + payload[item[0]].append(item[1]) + push_tasks.append(mdblist_client.push_ratings(settings.mdblist_api_key, payload)) if push_tasks: await asyncio.gather(*push_tasks, return_exceptions=True) @@ -288,6 +297,15 @@ async def delete_rating( push_tasks.append(simkl_client.remove_movie_rating(settings.simkl_client_id, settings.simkl_access_token, media.tmdb_id)) elif mt == MediaType.series: push_tasks.append(simkl_client.remove_show_rating(settings.simkl_client_id, settings.simkl_access_token, media.tmdb_id)) + if settings and settings.mdblist_push_ratings and settings.mdblist_api_key: + from core import mdblist as mdblist_client + from routers.mdblist import _empty_payload, _payload_item + + payload = _empty_payload() + item = _payload_item(media) + if item: + payload[item[0]].append(item[1]) + push_tasks.append(mdblist_client.remove_ratings(settings.mdblist_api_key, payload)) if push_tasks: await asyncio.gather(*push_tasks, return_exceptions=True) diff --git a/backend/routers/shows.py b/backend/routers/shows.py index 291bd3f..b323849 100644 --- a/backend/routers/shows.py +++ b/backend/routers/shows.py @@ -81,9 +81,10 @@ async def list_shows( ): offset = (page - 1) * page_size - # A show is "in the user's collection" if they have at least one countable episode - # collected — same criteria used by enrich_with_state for the percentage calculation. - user_show_ids = ( + # A show is in the user's collection when either the series itself is + # collected (cloud/watchlist providers) or at least one countable episode + # is collected (media-server libraries). + episode_show_ids = ( select(Media.show_id) .join(Collection, Collection.media_id == Media.id) .where( @@ -97,8 +98,24 @@ async def list_shows( .distinct() .subquery() ) + direct_series_tmdb_ids = ( + select(Media.tmdb_id) + .join(Collection, Collection.media_id == Media.id) + .where( + Collection.user_id == current_user.id, + Media.media_type == MediaType.series, + Media.tmdb_id.isnot(None), + ) + .distinct() + .subquery() + ) - base_query = select(ShowModel).where(ShowModel.id.in_(select(user_show_ids))) + base_query = select(ShowModel).where( + or_( + ShowModel.id.in_(select(episode_show_ids)), + ShowModel.tmdb_id.in_(select(direct_series_tmdb_ids)), + ) + ) if genre: base_query = base_query.where(sa_cast(ShowModel.tmdb_data["genres"], Text).contains(f'"{genre}"')) if year: diff --git a/backend/routers/sync.py b/backend/routers/sync.py index 1c64634..9944947 100644 --- a/backend/routers/sync.py +++ b/backend/routers/sync.py @@ -1,4 +1,5 @@ import asyncio +import logging import re from fastapi import APIRouter, Depends, Query, HTTPException, BackgroundTasks from pydantic import BaseModel @@ -17,18 +18,21 @@ from models.sync import SyncJob, SyncStatus from models.events import WatchEvent from models.ratings import Rating +from models.playback_progress import PlaybackProgress from models.library_selections import JellyfinLibrarySelection, EmbyLibrarySelection, PlexLibrarySelection from models.season_override import ShowSeasonOverride from datetime import datetime, timezone from dateutil import parser from models.base import MediaType, CollectionSource from models.global_settings import GlobalSettings -from core import jellyfin, emby, plex, tmdb +from core import jellyfin, emby, plex, nuvio, tmdb import core.trakt as trakt_client from core.enrichment import enrich_media from core.image_cache import pre_cache_all_collected_bg from dependencies import get_current_user +logger = logging.getLogger("uvicorn.error") + async def _get_effective_tmdb_key(db: AsyncSession, user_settings: UserSettings | None) -> str | None: @@ -47,6 +51,7 @@ async def _get_effective_tmdb_key(db: AsyncSession, user_settings: UserSettings TMDB_CONCURRENCY = 5 # Max concurrent TMDB requests # asyncpg hard limit is 32767 parameters per query; stay well under it _MAX_IN_PARAMS = 30_000 +_MEDIA_BROWSER_ITEM_SOURCES = (CollectionSource.jellyfin, CollectionSource.emby, CollectionSource.nuvio) async def _select_in_chunks(db: AsyncSession, stmt_builder, ids: list): @@ -61,10 +66,29 @@ async def _select_in_chunks(db: AsyncSession, stmt_builder, ids: list): return results +async def _latest_watched_at(db: AsyncSession, user_id: int, media_ids: list) -> dict: + """Latest completed WatchEvent.watched_at per media_id, chunked to avoid the 32767-parameter limit.""" + watched_at_by_media: dict[int, datetime] = {} + for i in range(0, len(media_ids), _MAX_IN_PARAMS): + chunk = media_ids[i : i + _MAX_IN_PARAMS] + result = await db.execute( + select(WatchEvent.media_id, WatchEvent.watched_at) + .where( + WatchEvent.user_id == user_id, + WatchEvent.media_id.in_(chunk), + WatchEvent.completed == True, + ) + .order_by(WatchEvent.watched_at.desc()) + ) + for media_id, watched_at in result.all(): + watched_at_by_media.setdefault(media_id, watched_at) + return watched_at_by_media + + def extract_watch_state(item: dict, source: CollectionSource) -> dict: state = {"completed": False, "last_played": None, "play_count": 0, "user_rating": None} - if source in (CollectionSource.jellyfin, CollectionSource.emby): + if source in _MEDIA_BROWSER_ITEM_SOURCES: user_data = item.get("UserData", {}) state["completed"] = user_data.get("Played", False) state["play_count"] = user_data.get("PlayCount", 1 if state["completed"] else 0) @@ -287,6 +311,245 @@ async def fetch_season(stid: int, sn: int): return warnings +def _nuvio_profile_id(conn: MediaServerConnection) -> int: + return nuvio.parse_profile_id(conn.server_user_id) + + +def _nuvio_imdb_id(entity: Media | Show | None) -> str | None: + if entity is None: + return None + data = entity.tmdb_data or {} + value = data.get("imdb_id") or (data.get("external_ids") or {}).get("imdb_id") + imdb_id = str(value or "").strip() + return imdb_id if imdb_id.startswith("tt") and imdb_id[2:].isdigit() else None + + +async def _ensure_nuvio_imdb_ids( + media_rows: list[Media], + shows_by_id: dict[int, Show], + api_key: str | None, +) -> None: + if not api_key: + return + + targets: dict[tuple[str, int], Media | Show] = {} + for media in media_rows: + if media.media_type == MediaType.episode: + show = shows_by_id.get(media.show_id) + if show and show.tmdb_id and not _nuvio_imdb_id(show): + targets[("tv", show.tmdb_id)] = show + elif media.tmdb_id and not _nuvio_imdb_id(media): + target_type = "movie" if media.media_type == MediaType.movie else "tv" + targets[(target_type, media.tmdb_id)] = media + if not targets: + return + + semaphore = asyncio.Semaphore(TMDB_CONCURRENCY) + + async def fetch_imdb_id(target_type: str, tmdb_id: int, entity: Media | Show) -> None: + async with semaphore: + try: + external_ids = await tmdb.get_external_ids(tmdb_id, target_type, api_key=api_key) + except Exception as exc: + logger.warning( + "Failed to resolve outbound Nuvio IMDb ID for TMDB %s (%s): %s", + tmdb_id, + target_type, + exc, + ) + return + imdb_id = str(external_ids.get("imdb_id") or "").strip() + if not (imdb_id.startswith("tt") and imdb_id[2:].isdigit()): + return + tmdb_data = dict(entity.tmdb_data or {}) + stored_external_ids = dict(tmdb_data.get("external_ids") or {}) + stored_external_ids["imdb_id"] = imdb_id + tmdb_data["external_ids"] = stored_external_ids + entity.tmdb_data = tmdb_data + + await asyncio.gather( + *[ + fetch_imdb_id(target_type, tmdb_id, entity) + for (target_type, tmdb_id), entity in targets.items() + ] + ) + logger.info("Resolved %s outbound Nuvio IMDb identifiers through TMDB", len(targets)) + + +def _nuvio_watched_item( + media: Media, + watched_at: datetime, + show: Show | None = None, +) -> dict | None: + if watched_at.tzinfo is None: + watched_at = watched_at.replace(tzinfo=timezone.utc) + watched_epoch_ms = int(watched_at.timestamp() * 1000) + + if media.media_type == MediaType.movie and (content_id := _nuvio_imdb_id(media)): + return { + "content_id": content_id, + "content_type": "movie", + "title": media.title, + "watched_at": watched_epoch_ms, + } + if ( + media.media_type == MediaType.episode + and (content_id := _nuvio_imdb_id(show)) + and media.season_number is not None + and media.episode_number is not None + ): + return { + "content_id": content_id, + "content_type": "series", + "title": media.title, + "season": media.season_number, + "episode": media.episode_number, + "watched_at": watched_epoch_ms, + } + if media.media_type == MediaType.series and (content_id := _nuvio_imdb_id(media)): + return { + "content_id": content_id, + "content_type": "series", + "title": media.title, + "watched_at": watched_epoch_ms, + } + return None + + +async def _build_nuvio_watched_items( + db: AsyncSession, + user_id: int, + media_ids: set[int] | None = None, + api_key: str | None = None, +) -> list[dict]: + event_query = ( + select(WatchEvent.media_id, WatchEvent.watched_at) + .where(WatchEvent.user_id == user_id, WatchEvent.completed == True) + .order_by(WatchEvent.watched_at.desc()) + ) + if media_ids is not None: + if not media_ids: + return [] + event_query = event_query.where(WatchEvent.media_id.in_(media_ids)) + event_result = await db.execute(event_query) + latest_watched_at: dict[int, datetime] = {} + for media_id, watched_at in event_result.all(): + latest_watched_at.setdefault(media_id, watched_at) + if not latest_watched_at: + return [] + + media_rows = await _select_in_chunks( + db, + lambda chunk: select(Media).where(Media.id.in_(chunk)), + list(latest_watched_at), + ) + show_ids = {media.show_id for media in media_rows if media.show_id is not None} + shows_by_id: dict[int, Show] = {} + if show_ids: + shows = await _select_in_chunks( + db, + lambda chunk: select(Show).where(Show.id.in_(chunk)), + list(show_ids), + ) + shows_by_id = {show.id: show for show in shows} + + await _ensure_nuvio_imdb_ids(media_rows, shows_by_id, api_key) + items: list[dict] = [] + for media in media_rows: + item = _nuvio_watched_item( + media, + latest_watched_at[media.id], + shows_by_id.get(media.show_id), + ) + if item: + items.append(item) + return items + + +def _nuvio_progress_item( + progress: PlaybackProgress, + media: Media, + show: Show | None = None, +) -> dict | None: + try: + progress_seconds = max(0, int(progress.progress_seconds)) + progress_percent = float(progress.progress_percent) + except (TypeError, ValueError): + return None + if progress_seconds <= 0 or progress_percent <= 0: + return None + + position_ms = progress_seconds * 1000 + if media.runtime and media.runtime > 0: + duration_ms = media.runtime * 60_000 + else: + duration_ms = round(position_ms / max(min(progress_percent, 1.0), 0.01)) + duration_ms = max(position_ms, duration_ms) + + updated_at = progress.updated_at + if updated_at.tzinfo is None: + updated_at = updated_at.replace(tzinfo=timezone.utc) + last_watched = int(updated_at.timestamp() * 1000) + + if media.media_type == MediaType.movie and (content_id := _nuvio_imdb_id(media)): + return { + "content_id": content_id, + "content_type": "movie", + "video_id": content_id, + "position": position_ms, + "duration": duration_ms, + "last_watched": last_watched, + } + if ( + media.media_type == MediaType.episode + and (content_id := _nuvio_imdb_id(show)) + and media.season_number is not None + and media.episode_number is not None + ): + return { + "content_id": content_id, + "content_type": "series", + "video_id": f"{content_id}:{media.season_number}:{media.episode_number}", + "season": media.season_number, + "episode": media.episode_number, + "position": position_ms, + "duration": duration_ms, + "last_watched": last_watched, + } + return None + + +async def _build_nuvio_progress_items( + db: AsyncSession, + user_id: int, + api_key: str | None = None, +) -> list[dict]: + result = await db.execute( + select(PlaybackProgress, Media) + .join(Media, Media.id == PlaybackProgress.media_id) + .where(PlaybackProgress.user_id == user_id) + .order_by(Media.id) + ) + rows = result.all() + show_ids = { + media.show_id + for _, media in rows + if media.media_type == MediaType.episode and media.show_id is not None + } + shows_by_id: dict[int, Show] = {} + if show_ids: + shows_result = await db.execute(select(Show).where(Show.id.in_(show_ids))) + shows_by_id = {show.id: show for show in shows_result.scalars().all()} + await _ensure_nuvio_imdb_ids([media for _, media in rows], shows_by_id, api_key) + + items: list[dict] = [] + for progress, media in rows: + item = _nuvio_progress_item(progress, media, shows_by_id.get(media.show_id)) + if item: + items.append(item) + return items + + async def _fan_out_changes_to_other_connections( db: AsyncSession, user_id: int, @@ -294,14 +557,12 @@ async def _fan_out_changes_to_other_connections( new_watched_ids: set[int], new_ratings: dict[int, float], settings: "UserSettings | None" = None, + exclude_cloud_source: CollectionSource | None = None, ) -> None: - """After an inbound sync, push the items that actually changed to every OTHER - connection (media servers + Trakt) that has push_watched / push_ratings enabled. - - Only the delta (what was added to Scrob during this sync) is pushed, so we - never blast unchanged history at the target server. + """Push an inbound sync delta to every enabled media server and cloud target. - pass exclude_connection_id=None when syncing from Trakt (no MediaServerConnection to skip). + ``exclude_connection_id`` prevents media-server echo. ``exclude_cloud_source`` + prevents a cloud pull from writing the same delta back to its source. """ if not new_watched_ids and not new_ratings: return @@ -346,7 +607,36 @@ async def _guarded(coro): async with sem: return await coro + nuvio_watched_items: list[dict] | None = None + nuvio_api_key = ( + await _get_effective_tmdb_key(db, settings) + if any(conn.type == "nuvio" and conn.push_watched for conn in push_candidates) + else None + ) + + async def _push_to_nuvio(conn: MediaServerConnection, items: list[dict]) -> bool: + session = await nuvio.push_watched_items( + conn.url, + conn.token, + _nuvio_profile_id(conn), + items, + ) + conn.token = session.refresh_token + return True + for conn in push_candidates: + if conn.type == "nuvio": + if conn.push_watched: + if nuvio_watched_items is None: + nuvio_watched_items = await _build_nuvio_watched_items( + db, + user_id, + new_watched_ids, + api_key=nuvio_api_key, + ) + if nuvio_watched_items: + push_tasks.append(_guarded(_push_to_nuvio(conn, nuvio_watched_items))) + continue conn_source = CollectionSource(conn.type) if conn.push_watched: for mid in new_watched_ids: @@ -368,8 +658,8 @@ async def _guarded(coro): push_tasks.append(_guarded(emby.set_rating(conn.url, conn.token, conn.server_user_id, sid, rating))) # ── Trakt fan-out ──────────────────────────────────────────────────────── - push_trakt_watched = settings and settings.trakt_push_watched and settings.trakt_access_token and settings.trakt_client_id - push_trakt_ratings = settings and settings.trakt_push_ratings and settings.trakt_access_token and settings.trakt_client_id + push_trakt_watched = settings and exclude_cloud_source != CollectionSource.trakt and settings.trakt_push_watched and settings.trakt_access_token and settings.trakt_client_id + push_trakt_ratings = settings and exclude_cloud_source != CollectionSource.trakt and settings.trakt_push_ratings and settings.trakt_access_token and settings.trakt_client_id if (push_trakt_watched or push_trakt_ratings) and all_changed_ids: media_items = await _select_in_chunks( @@ -428,13 +718,66 @@ async def _guarded(coro): trakt_movie_ratings, trakt_show_ratings, )) + # ── MDBList fan-out ────────────────────────────────────────────────────── + push_mdblist_watched = settings and exclude_cloud_source != CollectionSource.mdblist and settings.mdblist_push_watched and settings.mdblist_api_key + push_mdblist_ratings = settings and exclude_cloud_source != CollectionSource.mdblist and settings.mdblist_push_ratings and settings.mdblist_api_key + + if (push_mdblist_watched or push_mdblist_ratings) and all_changed_ids: + from core import mdblist as mdblist_client + from routers.mdblist import _empty_payload, _payload_item + + mdblist_media = await _select_in_chunks( + db, + lambda chunk: select(Media).where(Media.id.in_(chunk)), + list(all_changed_ids), + ) + mdblist_media_by_id = {media.id: media for media in mdblist_media} + + if push_mdblist_watched: + watched_at_by_media = await _latest_watched_at(db, user_id, list(new_watched_ids)) + + watched_payload = _empty_payload() + for media_id in new_watched_ids: + media = mdblist_media_by_id.get(media_id) + item = _payload_item(media, watched_at=watched_at_by_media.get(media_id, datetime.utcnow())) if media else None + if item: + watched_payload[item[0]].append(item[1]) + push_tasks.append(mdblist_client.push_watched(settings.mdblist_api_key, watched_payload)) + + if push_mdblist_ratings: + rated_media_ids = list(new_ratings.keys()) + rated_at_by_media: dict[int, datetime] = {} + for i in range(0, len(rated_media_ids), _MAX_IN_PARAMS): + chunk = rated_media_ids[i : i + _MAX_IN_PARAMS] + rated_at_result = await db.execute( + select(Rating.media_id, Rating.rated_at).where( + Rating.user_id == user_id, Rating.media_id.in_(chunk) + ) + ) + rated_at_by_media.update(dict(rated_at_result.all())) + ratings_payload = _empty_payload() + for media_id, rating in new_ratings.items(): + media = mdblist_media_by_id.get(media_id) + item = ( + _payload_item(media, rating=rating, rated_at=rated_at_by_media.get(media_id)) + if media + else None + ) + if item: + ratings_payload[item[0]].append(item[1]) + push_tasks.append(mdblist_client.push_ratings(settings.mdblist_api_key, ratings_payload)) + if push_tasks: - target_count = len(push_candidates) + (1 if (push_trakt_watched or push_trakt_ratings) else 0) - print(f" Fanning out {len(push_tasks)} changes to {target_count} other connection(s) (incl. Trakt)...") + target_count = len(push_candidates) + target_count += 1 if (push_trakt_watched or push_trakt_ratings) else 0 + target_count += 1 if (push_mdblist_watched or push_mdblist_ratings) else 0 + print(f" Fanning out {len(push_tasks)} changes to {target_count} other connection(s)...") results = await asyncio.gather(*push_tasks, return_exceptions=True) failed = sum(1 for r in results if isinstance(r, Exception)) if failed: print(f" {failed}/{len(push_tasks)} fan-out push tasks failed (non-fatal)") + if any(conn.type == "nuvio" for conn in push_candidates): + await db.commit() async def sync_items( @@ -504,7 +847,7 @@ async def sync_items( for item in items: tid = ( get_jellyfin_tmdb_id(item.get("ProviderIds", {})) - if source in (CollectionSource.jellyfin, CollectionSource.emby) + if source in _MEDIA_BROWSER_ITEM_SOURCES else plex.extract_tmdb_id(item.get("Guid", [])) ) if tid: @@ -526,7 +869,7 @@ async def sync_items( for item in items: tid = ( get_jellyfin_tmdb_id(item.get("ProviderIds", {})) - if source in (CollectionSource.jellyfin, CollectionSource.emby) + if source in _MEDIA_BROWSER_ITEM_SOURCES else plex.extract_tmdb_id(item.get("Guid", [])) ) if tid: @@ -561,7 +904,7 @@ async def sync_items( new_media: Media | None = None try: async with db.begin_nested(): - if source in (CollectionSource.jellyfin, CollectionSource.emby): + if source in _MEDIA_BROWSER_ITEM_SOURCES: source_id = str(item.get("Id")) quality = extract_jellyfin_quality(item) tmdb_id = get_jellyfin_tmdb_id(item.get("ProviderIds", {})) @@ -660,7 +1003,7 @@ async def sync_items( and not (existing_media_obj.tmdb_data or {}).get("show_title") ): _series_name = ( - item.get("SeriesName") if source in (CollectionSource.jellyfin, CollectionSource.emby) + item.get("SeriesName") if source in _MEDIA_BROWSER_ITEM_SOURCES else item.get("grandparentTitle") ) if _series_name: @@ -736,7 +1079,7 @@ async def sync_items( # Jellyfin hasn't finished fetching episode metadata yet). # Everything else (movies, episodes without show context) is skipped. series_name = ( - item.get("SeriesName") if source in (CollectionSource.jellyfin, CollectionSource.emby) + item.get("SeriesName") if source in _MEDIA_BROWSER_ITEM_SOURCES else item.get("grandparentTitle") ) if media_type == MediaType.episode else None @@ -823,7 +1166,8 @@ async def sync_items( ) coll_id = coll_result.scalar_one() existing_coll_by_media_id[media.id] = coll_id - stats["movies" if media_type == MediaType.movie else "episodes"] += 1 + stat_key = "movies" if media_type == MediaType.movie else "series" if media_type == MediaType.series else "episodes" + stats[stat_key] = stats.get(stat_key, 0) + 1 # else: collection already exists from another source — just add the file db.add(CollectionFile( collection_id=coll_id, @@ -914,7 +1258,7 @@ async def sync_items( series_title_map: dict[int, str] = {} if media_type == MediaType.episode: for item in items: - if source in (CollectionSource.jellyfin, CollectionSource.emby): + if source in _MEDIA_BROWSER_ITEM_SOURCES: parent_id = str(item.get("SeriesId", "")) title = item.get("SeriesName") else: @@ -1727,6 +2071,478 @@ async def resolve_show_tmdb_id(s: dict) -> None: await db.commit() +def _parse_nuvio_tmdb_id(content_id: object) -> int | None: + value = str(content_id or "") + if not value.startswith("tmdb:"): + return None + try: + return int(value[5:]) + except ValueError: + return None + + +async def _resolve_nuvio_tmdb_ids( + records: list[dict], + db: AsyncSession, + user_id: int, + api_key: str, +) -> dict[str, int]: + content_types: dict[str, str] = {} + resolved: dict[str, int] = {} + for record in records: + content_id = str(record.get("content_id") or "").strip() + if not content_id: + continue + if tmdb_id := _parse_nuvio_tmdb_id(content_id): + resolved[content_id] = tmdb_id + elif re.fullmatch(r"tt\d+", content_id, flags=re.IGNORECASE): + content_types.setdefault(content_id, str(record.get("content_type") or "").lower()) + + unresolved = set(content_types) - set(resolved) + if unresolved: + existing_result = await db.execute( + select(CollectionFile.source_id, Media.tmdb_id) + .join(Collection, Collection.id == CollectionFile.collection_id) + .join(Media, Media.id == Collection.media_id) + .where( + Collection.user_id == user_id, + CollectionFile.source == CollectionSource.nuvio, + Media.tmdb_id.isnot(None), + ) + ) + for source_id, tmdb_id in existing_result.all(): + parts = str(source_id).split(":") + if len(parts) >= 2 and parts[1] in unresolved: + resolved[parts[1]] = int(tmdb_id) + unresolved -= set(resolved) + + semaphore = asyncio.Semaphore(TMDB_CONCURRENCY) + + async def resolve_imdb_id(content_id: str) -> None: + async with semaphore: + try: + result = await tmdb.find_by_external_id(content_id, "imdb_id", api_key=api_key) + result_key = "movie_results" if content_types[content_id] == "movie" else "tv_results" + matches = result.get(result_key) or [] + if matches and matches[0].get("id") is not None: + resolved[content_id] = int(matches[0]["id"]) + except Exception as exc: + logger.warning( + "Failed to resolve Nuvio IMDb ID %s through TMDB: %s", + content_id, + exc, + ) + + if unresolved: + await asyncio.gather(*(resolve_imdb_id(content_id) for content_id in sorted(unresolved))) + logger.info( + "Resolved %s/%s new Nuvio IMDb IDs through TMDB", + len(unresolved & set(resolved)), + len(unresolved), + ) + return resolved + + +def _nuvio_datetime(epoch_ms: object) -> datetime | None: + try: + return datetime.fromtimestamp(int(epoch_ms) / 1000, tz=timezone.utc).replace(tzinfo=None) + except (TypeError, ValueError, OSError, OverflowError): + return None + + +def _normalize_nuvio_item( + record: dict, + profile_id: int, + watched: bool = False, + tmdb_id: int | None = None, +) -> tuple[MediaType, dict] | None: + tmdb_id = tmdb_id or _parse_nuvio_tmdb_id(record.get("content_id")) + if tmdb_id is None: + return None + + content_type = str(record.get("content_type") or "").lower() + season = record.get("season") + episode = record.get("episode") + is_episode = content_type == "series" and season is not None and episode is not None + if content_type == "movie": + media_type = MediaType.movie + elif is_episode: + media_type = MediaType.episode + elif content_type == "series": + media_type = MediaType.series + else: + return None + + content_id = str(record["content_id"]) + source_id = f"{profile_id}:{content_id}" + if is_episode: + source_id = f"{source_id}:s{season}e{episode}" + last_played = _nuvio_datetime(record.get("watched_at") or record.get("last_watched")) + title = record.get("title") or record.get("name") or content_id + + item = { + "Id": source_id, + "Name": title, + "ProviderIds": {} if is_episode else {"Tmdb": str(tmdb_id)}, + "MediaStreams": [], + "Path": None, + "SeriesId": content_id if is_episode else None, + "SeriesName": title if is_episode else None, + "ParentIndexNumber": int(season) if season is not None else None, + "IndexNumber": int(episode) if episode is not None else None, + "UserData": { + "Played": watched, + "PlayCount": 1 if watched else 0, + "LastPlayedDate": last_played.isoformat() if last_played else None, + }, + } + return media_type, item + + +async def _apply_nuvio_progress( + db: AsyncSession, + user_id: int, + rows: list[dict], + show_map: dict[str, int], + tmdb_ids: dict[str, int], +) -> None: + movie_tmdb_ids = { + tmdb_id + for row in rows + if str(row.get("content_type") or "").lower() == "movie" + if (tmdb_id := tmdb_ids.get(str(row.get("content_id") or ""))) is not None + } + movies_by_tmdb: dict[int, Media] = {} + if movie_tmdb_ids: + result = await db.execute( + select(Media).where(Media.media_type == MediaType.movie, Media.tmdb_id.in_(movie_tmdb_ids)) + ) + movies_by_tmdb = {media.tmdb_id: media for media in result.scalars().all() if media.tmdb_id is not None} + + show_ids = set(show_map.values()) + episodes_by_key: dict[tuple[int, int, int], Media] = {} + if show_ids: + result = await db.execute( + select(Media).where(Media.media_type == MediaType.episode, Media.show_id.in_(show_ids)) + ) + episodes_by_key = { + (media.show_id, media.season_number, media.episode_number): media + for media in result.scalars().all() + if media.show_id is not None and media.season_number is not None and media.episode_number is not None + } + + media_rows: list[tuple[dict, Media]] = [] + for row in rows: + content_id = str(row.get("content_id") or "") + tmdb_id = tmdb_ids.get(content_id) + if tmdb_id is None: + continue + if str(row.get("content_type") or "").lower() == "movie": + media = movies_by_tmdb.get(tmdb_id) + else: + season = row.get("season") + episode = row.get("episode") + show_id = show_map.get(content_id) + media = ( + episodes_by_key.get((show_id, int(season), int(episode))) + if show_id is not None and season is not None and episode is not None + else None + ) + if media is not None: + media_rows.append((row, media)) + + if not media_rows: + return + + media_ids = {media.id for _, media in media_rows} + existing_result = await db.execute( + select(PlaybackProgress).where( + PlaybackProgress.user_id == user_id, + PlaybackProgress.media_id.in_(media_ids), + ) + ) + existing = {progress.media_id: progress for progress in existing_result.scalars().all()} + + for row, media in media_rows: + try: + position_ms = max(0, int(row.get("position") or 0)) + duration_ms = max(0, int(row.get("duration") or 0)) + except (TypeError, ValueError): + continue + if duration_ms <= 0: + continue + progress_percent = min(1.0, position_ms / duration_ms) + progress = existing.get(media.id) + if 0.05 <= progress_percent < 0.90: + updated_at = _nuvio_datetime(row.get("last_watched")) or datetime.utcnow() + if progress: + progress.progress_percent = progress_percent + progress.progress_seconds = position_ms // 1000 + progress.updated_at = updated_at + else: + progress = PlaybackProgress( + user_id=user_id, + media_id=media.id, + progress_percent=progress_percent, + progress_seconds=position_ms // 1000, + updated_at=updated_at, + ) + db.add(progress) + existing[media.id] = progress + elif progress: + await db.delete(progress) + existing.pop(media.id, None) + await db.commit() + + +async def run_nuvio_sync( + user_id: int, + job_id: int, + movie_limit: int, + show_limit: int, + connection_id: int | None = None, +): + async with _sync_semaphore: + await _run_nuvio_sync(user_id, job_id, movie_limit, show_limit, connection_id) + + +async def _run_nuvio_sync( + user_id: int, + job_id: int, + movie_limit: int, + show_limit: int, + connection_id: int | None = None, +): + logger.info("Starting Nuvio sync for user %s, job %s", user_id, job_id) + async_session = async_sessionmaker(engine, expire_on_commit=False, class_=AsyncSession) + async with async_session() as db: + try: + await db.execute( + update(SyncJob) + .where(SyncJob.id == job_id) + .values(status=SyncStatus.running, processed_items=0, total_items=0) + ) + await db.commit() + + settings_result = await db.execute(select(UserSettings).where(UserSettings.user_id == user_id)) + settings = settings_result.scalar_one_or_none() + tmdb_api_key = await _get_effective_tmdb_key(db, settings) + + conn_query = select(MediaServerConnection).where( + MediaServerConnection.user_id == user_id, + MediaServerConnection.type == "nuvio", + ) + if connection_id: + conn_query = conn_query.where(MediaServerConnection.id == connection_id) + else: + conn_query = conn_query.order_by(MediaServerConnection.id.asc()).limit(1) + conn_result = await db.execute(conn_query) + conn = conn_result.scalar_one_or_none() + if not conn or not tmdb_api_key: + raise RuntimeError("Missing Nuvio connection or TMDB API key") + + try: + profile_id = int(conn.server_user_id or "") + except ValueError: + raise RuntimeError("Invalid Nuvio profile index") + if profile_id < 1 or profile_id > 6: + raise RuntimeError("Invalid Nuvio profile index") + + session, data = await nuvio.pull_sync_data(conn.url, conn.token, profile_id) + # Supabase refresh tokens rotate. Persist the replacement before doing + # any expensive metadata work so the connection remains recoverable. + conn.token = session.refresh_token + await db.commit() + + library_records = data["library"] if conn.sync_collection else [] + watched_records = data["watched"] if conn.sync_watched else [] + progress_records = data["progress"] if conn.sync_playback else [] + logger.info( + "Nuvio profile %s (index #%s): pulled %s library, %s watched, " + "and %s progress records; enabled for this sync: %s library, " + "%s watched, %s progress", + conn.server_username or f"#{profile_id}", + profile_id, + len(data["library"]), + len(data["watched"]), + len(data["progress"]), + len(library_records), + len(watched_records), + len(progress_records), + ) + + all_nuvio_records = [*library_records, *watched_records, *progress_records] + tmdb_ids = await _resolve_nuvio_tmdb_ids( + all_nuvio_records, + db, + user_id, + tmdb_api_key, + ) + normalized_library = [ + normalized + for record in library_records + if ( + normalized := _normalize_nuvio_item( + record, + profile_id, + tmdb_id=tmdb_ids.get(str(record.get("content_id") or "")), + ) + ) + is not None + ] + normalized_watched = [ + normalized + for record in watched_records + if ( + normalized := _normalize_nuvio_item( + record, + profile_id, + watched=True, + tmdb_id=tmdb_ids.get(str(record.get("content_id") or "")), + ) + ) + is not None + ] + normalized_progress = [ + normalized + for record in progress_records + if ( + normalized := _normalize_nuvio_item( + record, + profile_id, + tmdb_id=tmdb_ids.get(str(record.get("content_id") or "")), + ) + ) + is not None + ] + skipped_nuvio_records = len(all_nuvio_records) - ( + len(normalized_library) + len(normalized_watched) + len(normalized_progress) + ) + + if movie_limit: + normalized_library = [ + *[entry for entry in normalized_library if entry[0] == MediaType.movie][:movie_limit], + *[entry for entry in normalized_library if entry[0] != MediaType.movie], + ] + if show_limit: + normalized_library = [ + *[entry for entry in normalized_library if entry[0] != MediaType.series], + *[entry for entry in normalized_library if entry[0] == MediaType.series][:show_limit], + ] + + series_tmdb_map = { + str(record.get("content_id")): tmdb_id + for record in [*library_records, *watched_records, *progress_records] + if str(record.get("content_type") or "").lower() == "series" + if (tmdb_id := tmdb_ids.get(str(record.get("content_id") or ""))) is not None + } + if series_tmdb_map: + show_map, show_id_to_tmdb = await sync_shows_batch( + series_tmdb_map, + db, + api_key=tmdb_api_key, + ) + else: + show_map, show_id_to_tmdb = {}, {} + + all_entries = [*normalized_library, *normalized_watched, *normalized_progress] + await db.execute( + update(SyncJob).where(SyncJob.id == job_id).values(total_items=len(all_entries)) + ) + await db.commit() + + stats = {"movies": 0, "series": 0, "episodes": 0, "skipped": skipped_nuvio_records, "errors": 0} + warnings: list[dict] = [] + new_watched_ids: set[int] = set() + + async def sync_group( + entries: list[tuple[MediaType, dict]], + media_type: MediaType, + *, + sync_collection: bool, + sync_watched: bool, + ) -> None: + items = [item for item_type, item in entries if item_type == media_type] + if not items: + return + group_warnings = await sync_items( + items, + media_type, + CollectionSource.nuvio, + db, + stats, + user_id, + job_id, + show_map if media_type == MediaType.episode else {}, + api_key=tmdb_api_key, + show_id_to_tmdb=show_id_to_tmdb if media_type == MediaType.episode else {}, + sync_collection=sync_collection, + sync_watched=sync_watched, + sync_ratings=False, + new_watched_ids=new_watched_ids, + connection_id=conn.id, + ) + warnings.extend(group_warnings) + + for media_type in (MediaType.movie, MediaType.series, MediaType.episode): + await sync_group( + normalized_library, + media_type, + sync_collection=True, + sync_watched=False, + ) + for media_type in (MediaType.movie, MediaType.series, MediaType.episode): + await sync_group( + normalized_watched, + media_type, + sync_collection=False, + sync_watched=True, + ) + for media_type in (MediaType.movie, MediaType.series, MediaType.episode): + await sync_group( + normalized_progress, + media_type, + sync_collection=False, + sync_watched=False, + ) + + if progress_records: + await _apply_nuvio_progress(db, user_id, progress_records, show_map, tmdb_ids) + + await _fan_out_changes_to_other_connections( + db, + user_id, + conn.id, + new_watched_ids, + {}, + settings=settings, + exclude_cloud_source=CollectionSource.nuvio, + ) + warnings = await _stamp_matched_show_warnings(db, user_id, warnings) + await db.execute( + update(SyncJob) + .where(SyncJob.id == job_id) + .values( + status=SyncStatus.completed, + stats=stats, + warnings=warnings or None, + updated_at=func.now(), + ) + ) + await db.commit() + asyncio.create_task(pre_cache_all_collected_bg()) + logger.info("Nuvio sync job %s completed. Stats: %s", job_id, stats) + except Exception as exc: + logger.exception("Nuvio sync job %s failed", job_id) + await db.rollback() + await db.execute( + update(SyncJob) + .where(SyncJob.id == job_id) + .values(status=SyncStatus.failed, error_message=str(exc)[:900]) + ) + await db.commit() + + class LibrarySelectionBody(BaseModel): library_ids: list[str] @@ -1807,6 +2623,9 @@ async def get_connection_libraries( ] return {"libraries": libraries, "all_selected": len(selected_keys) == 0} + elif conn.type == "nuvio": + return {"libraries": [], "all_selected": True} + else: raise HTTPException(status_code=400, detail=f"Unknown connection type: {conn.type}") except HTTPException: @@ -1818,12 +2637,59 @@ async def get_connection_libraries( @router.post("/connection/{connection_id}/scan") async def trigger_library_scan( connection_id: int, + background_tasks: BackgroundTasks, db: AsyncSession = Depends(get_db), current_user: User = Depends(get_current_user), ): conn = await _get_connection_or_404(db, connection_id, current_user.id) try: + if conn.type == "nuvio": + settings_result = await db.execute( + select(UserSettings).where(UserSettings.user_id == current_user.id) + ) + settings = settings_result.scalar_one_or_none() + if not await _get_effective_tmdb_key(db, settings): + raise HTTPException(status_code=400, detail="TMDB API key required") + active_result = await db.execute( + select(SyncJob) + .where( + SyncJob.user_id == current_user.id, + SyncJob.connection_id == conn.id, + SyncJob.status.in_([SyncStatus.pending, SyncStatus.running]), + ) + .limit(1) + ) + active_job = active_result.scalar_one_or_none() + if active_job: + return { + "status": "started", + "job_id": active_job.id, + "message": "Nuvio sync is already running", + } + job = SyncJob( + user_id=current_user.id, + source=CollectionSource.nuvio, + status=SyncStatus.pending, + connection_id=conn.id, + job_type="pull", + ) + db.add(job) + await db.commit() + await db.refresh(job) + background_tasks.add_task( + run_nuvio_sync, + current_user.id, + job.id, + 0, + 0, + conn.id, + ) + return { + "status": "started", + "job_id": job.id, + "message": "Nuvio library, watch history, and playback progress sync started", + } if conn.type in ("jellyfin", "emby"): client = jellyfin if conn.type == "jellyfin" else emby ok = await client.scan_libraries(conn.url, conn.token) @@ -1888,6 +2754,9 @@ async def save_connection_libraries( await db.commit() return {"saved": len(library_keys)} + elif conn.type == "nuvio": + return {"saved": 0} + else: raise HTTPException(status_code=400, detail=f"Unknown connection type: {conn.type}") except HTTPException: @@ -1912,7 +2781,7 @@ async def sync_connection( if not await _get_effective_tmdb_key(db, settings): raise HTTPException(status_code=400, detail="TMDB API key required") - source_map = {"jellyfin": CollectionSource.jellyfin, "emby": CollectionSource.emby, "plex": CollectionSource.plex} + source_map = {"jellyfin": CollectionSource.jellyfin, "emby": CollectionSource.emby, "plex": CollectionSource.plex, "nuvio": CollectionSource.nuvio} source = source_map.get(conn.type) if not source: raise HTTPException(status_code=400, detail=f"Unknown connection type: {conn.type}") @@ -1922,7 +2791,7 @@ async def sync_connection( await db.commit() await db.refresh(job) - runner_map = {"jellyfin": run_jellyfin_sync, "emby": run_emby_sync, "plex": run_plex_sync} + runner_map = {"jellyfin": run_jellyfin_sync, "emby": run_emby_sync, "plex": run_plex_sync, "nuvio": run_nuvio_sync} background_tasks.add_task(runner_map[conn.type], current_user.id, job.id, movie_limit, show_limit, connection_id) return {"status": "started", "job_id": job.id, "message": f"{conn.type.capitalize()} sync is running in the background"} @@ -1948,6 +2817,61 @@ async def _run_full_push(user_id: int, connection_id: int, job_id: int) -> None: await db.commit() return + if conn.type == "nuvio": + settings_result = await db.execute( + select(UserSettings).where(UserSettings.user_id == user_id) + ) + user_settings = settings_result.scalar_one_or_none() + api_key = await _get_effective_tmdb_key(db, user_settings) + watched_items = ( + await _build_nuvio_watched_items(db, user_id, api_key=api_key) + if conn.push_watched + else [] + ) + progress_items = ( + await _build_nuvio_progress_items(db, user_id, api_key=api_key) + if conn.push_playback + else [] + ) + total = len(watched_items) + len(progress_items) + await db.execute( + update(SyncJob) + .where(SyncJob.id == job_id) + .values(total_items=total, processed_items=0) + ) + await db.commit() + if watched_items or progress_items: + session = await nuvio.push_sync_items( + conn.url, + conn.token, + _nuvio_profile_id(conn), + watched_items, + progress_items, + ) + conn.token = session.refresh_token + await db.execute( + update(SyncJob) + .where(SyncJob.id == job_id) + .values( + status=SyncStatus.completed, + processed_items=total, + stats={ + "succeeded": total, + "failed": 0, + "watched": len(watched_items), + "progress": len(progress_items), + }, + ) + ) + await db.commit() + logger.info( + "Full Nuvio push for connection %s: %s watched and %s progress items", + connection_id, + len(watched_items), + len(progress_items), + ) + return + conn_source = CollectionSource(conn.type) watched_ids: set[int] = set() ratings_map: dict[int, float] = {} @@ -2178,10 +3102,10 @@ async def push_upstream( current_user: User = Depends(get_current_user), ): conn = await _get_connection_or_404(db, connection_id, current_user.id) - if not conn.push_watched and not conn.push_ratings: + if not conn.push_watched and not conn.push_ratings and not conn.push_playback: raise HTTPException(status_code=400, detail="Enable 'Scrob → Server' push flags for this connection first") - source_map = {"jellyfin": CollectionSource.jellyfin, "emby": CollectionSource.emby, "plex": CollectionSource.plex} + source_map = {"jellyfin": CollectionSource.jellyfin, "emby": CollectionSource.emby, "plex": CollectionSource.plex, "nuvio": CollectionSource.nuvio} source = source_map.get(conn.type, CollectionSource.jellyfin) job = SyncJob(user_id=current_user.id, source=source, status=SyncStatus.pending, connection_id=connection_id, job_type="push") db.add(job) diff --git a/backend/routers/trakt.py b/backend/routers/trakt.py index 1efd51c..30a9478 100644 --- a/backend/routers/trakt.py +++ b/backend/routers/trakt.py @@ -307,6 +307,7 @@ async def run_trakt_sync(user_id: int, job_id: int): stats = {"movies": 0, "episodes": 0, "ratings": 0, "lists": 0, "list_items": 0, "skipped": 0, "errors": 0} _new_watched: set[int] = set() _new_ratings: dict[int, float] = {} + watched_processed = 0 # ── Watched Movies ──────────────────────────────────────────────── if sync_watched: @@ -322,42 +323,52 @@ async def run_trakt_sync(user_id: int, job_id: int): ) existing_watched: set[int] = {row[0] for row in we_res} - for item in watched_movies: + for movie_index, item in enumerate(watched_movies, start=1): movie_data = item.get("movie", {}) tmdb_id = movie_data.get("ids", {}).get("tmdb") - if not tmdb_id: - stats["skipped"] += 1 - continue try: - async with db.begin_nested(): - media = await _get_or_create_movie_media(db, tmdb_id, movie_data.get("title", ""), api_key) - if not media: - stats["errors"] += 1 - continue - if media.id not in existing_watched: - last_watched = item.get("last_watched_at") - watched_at = None - if last_watched: - from dateutil import parser as dt_parser - dt = dt_parser.isoparse(last_watched) - if dt.tzinfo: - dt = dt.astimezone(timezone.utc).replace(tzinfo=None) - watched_at = dt - db.add(WatchEvent( - user_id=user_id, - media_id=media.id, - watched_at=watched_at or datetime.utcnow(), - completed=True, - play_count=item.get("plays", 1), - )) - existing_watched.add(media.id) - _new_watched.add(media.id) - stats["movies"] += 1 - else: - stats["skipped"] += 1 - except Exception as exc: - logger.warning("Error processing Trakt movie tmdb=%s: %s", tmdb_id, exc) - stats["errors"] += 1 + if not tmdb_id: + stats["skipped"] += 1 + continue + try: + async with db.begin_nested(): + media = await _get_or_create_movie_media(db, tmdb_id, movie_data.get("title", ""), api_key) + if not media: + stats["errors"] += 1 + continue + if media.id not in existing_watched: + last_watched = item.get("last_watched_at") + watched_at = None + if last_watched: + from dateutil import parser as dt_parser + dt = dt_parser.isoparse(last_watched) + if dt.tzinfo: + dt = dt.astimezone(timezone.utc).replace(tzinfo=None) + watched_at = dt + db.add(WatchEvent( + user_id=user_id, + media_id=media.id, + watched_at=watched_at or datetime.utcnow(), + completed=True, + play_count=item.get("plays", 1), + )) + existing_watched.add(media.id) + _new_watched.add(media.id) + stats["movies"] += 1 + else: + stats["skipped"] += 1 + except Exception as exc: + logger.warning("Error processing Trakt movie tmdb=%s: %s", tmdb_id, exc) + stats["errors"] += 1 + finally: + watched_processed = movie_index + if movie_index % 25 == 0 or movie_index == len(watched_movies): + await db.execute( + update(SyncJob) + .where(SyncJob.id == job_id) + .values(processed_items=watched_processed) + ) + await db.commit() await db.commit() @@ -372,7 +383,8 @@ async def run_trakt_sync(user_id: int, job_id: int): for s in watched_shows ) await db.execute(update(SyncJob).where(SyncJob.id == job_id).values( - total_items=len(watched_movies) + total_episodes + total_items=len(watched_movies) + total_episodes, + processed_items=watched_processed, )) await db.commit() @@ -443,9 +455,14 @@ async def process_show(show_entry: dict): for i, s in enumerate(watched_shows): await process_show(s) - if (i + 1) % 10 == 0: + watched_processed += sum( + len(season.get("episodes", [])) + for season in s.get("seasons", []) + if season.get("number") not in (None, 0) + ) + if (i + 1) % 10 == 0 or i + 1 == len(watched_shows): await db.execute(update(SyncJob).where(SyncJob.id == job_id).values( - processed_items=stats["movies"] + stats["episodes"] + processed_items=watched_processed )) await db.commit() await db.commit() @@ -819,12 +836,12 @@ async def _get_or_create_split_list(slug: str, name: str) -> ListModel: f"Errors: {stats['errors']}." ) from routers.sync import _fan_out_changes_to_other_connections - await _fan_out_changes_to_other_connections(db, user_id, None, _new_watched, _new_ratings, settings=settings) + await _fan_out_changes_to_other_connections(db, user_id, None, _new_watched, _new_ratings, settings=settings, exclude_cloud_source=CollectionSource.trakt) await db.execute( update(SyncJob).where(SyncJob.id == job_id).values( status=SyncStatus.completed, stats=stats, - processed_items=stats["movies"] + stats["episodes"] + stats["ratings"], + processed_items=watched_processed, ) ) await db.commit() diff --git a/backend/schemas.py b/backend/schemas.py index 960d84e..9642615 100644 --- a/backend/schemas.py +++ b/backend/schemas.py @@ -117,6 +117,16 @@ class UserSettings(BaseModel): simkl_push_ratings: Optional[bool] = None simkl_scrobble: Optional[bool] = None + # MDBList — API key authentication + mdblist_api_key: Optional[str] = None + mdblist_connected: Optional[bool] = None # read-only, validated by /auth/connection-status + mdblist_sync_watched: Optional[bool] = None + mdblist_sync_ratings: Optional[bool] = None + mdblist_sync_watchlist: Optional[bool] = None + mdblist_push_watched: Optional[bool] = None + mdblist_push_ratings: Optional[bool] = None + mdblist_push_watchlist: Optional[bool] = None + preferences: Optional[dict] = None blur_explicit: Optional[bool] = None time_format_24h: Optional[bool] = None @@ -126,6 +136,18 @@ class Config: from_attributes = True +class NuvioLoginRequest(BaseModel): + email: EmailStr + password: str + url: str = "https://api.nuvio.tv" + + +class NuvioConnectionTestRequest(BaseModel): + url: str + token: str + profile_id: int + + class MediaServerConnectionBase(BaseModel): type: str name: str @@ -138,8 +160,9 @@ class MediaServerConnectionBase(BaseModel): sync_ratings: bool = True sync_playback: bool = True push_watched: bool = False + push_playback: bool = False push_ratings: bool = False - auto_sync_interval: Optional[int] = None + auto_sync_interval: Optional[float] = None watchlist_to_radarr: bool = False watchlist_to_sonarr: bool = False watchlist_all_users: bool = False @@ -163,8 +186,9 @@ class MediaServerConnectionUpdate(BaseModel): sync_ratings: Optional[bool] = None sync_playback: Optional[bool] = None push_watched: Optional[bool] = None + push_playback: Optional[bool] = None push_ratings: Optional[bool] = None - auto_sync_interval: Optional[int] = None + auto_sync_interval: Optional[float] = None watchlist_to_radarr: Optional[bool] = None watchlist_to_sonarr: Optional[bool] = None watchlist_all_users: Optional[bool] = None diff --git a/backend/tests/test_mdblist.py b/backend/tests/test_mdblist.py new file mode 100644 index 0000000..1fcf0dd --- /dev/null +++ b/backend/tests/test_mdblist.py @@ -0,0 +1,218 @@ +import json +import os +import unittest +from datetime import datetime +from types import SimpleNamespace +from unittest.mock import AsyncMock, MagicMock, patch + +import httpx + +os.environ.setdefault("SECRET_KEY", "test-secret") +os.environ.setdefault("DATABASE_URL", "postgresql+asyncpg://test:test@localhost/test") + +from core import mdblist +from models.base import MediaType +from models.media import Media +from routers.mdblist import _episode_identity, _payload_item, _resolve_external_tmdb_id +from routers.lists import _push_list_item_to_mdblist + + +_REAL_ASYNC_CLIENT = httpx.AsyncClient + + +class MDBListClientTests(unittest.IsolatedAsyncioTestCase): + async def test_get_watched_follows_cursor_pagination(self) -> None: + cursors: list[str | None] = [] + + def handler(request: httpx.Request) -> httpx.Response: + self.assertEqual(request.url.path, "/sync/watched") + self.assertEqual(request.url.params["apikey"], "secret-key") + self.assertEqual(request.url.params["limit"], "1000") + cursor = request.url.params.get("cursor") + cursors.append(cursor) + if cursor is None: + return httpx.Response( + 200, + json={ + "movies": [{"movie": {"ids": {"tmdb": 550}}}], + "pagination": {"next_cursor": "next-page"}, + }, + ) + self.assertEqual(cursor, "next-page") + return httpx.Response( + 200, + json={ + "shows": [{"show": {"ids": {"tmdb": 1396}}}], + "pagination": {"next_cursor": None}, + }, + ) + + transport = httpx.MockTransport(handler) + with patch.object( + mdblist.httpx, + "AsyncClient", + side_effect=lambda **kwargs: _REAL_ASYNC_CLIENT(transport=transport, **kwargs), + ): + result = await mdblist.get_watched("secret-key") + + self.assertEqual(cursors, [None, "next-page"]) + self.assertEqual(len(result["movies"]), 1) + self.assertEqual(len(result["shows"]), 1) + + async def test_get_watchlist_falls_back_to_offset_pagination(self) -> None: + offsets: list[int] = [] + + def handler(request: httpx.Request) -> httpx.Response: + offset = int(request.url.params.get("offset", 0)) + offsets.append(offset) + if offset == 0: + return httpx.Response( + 200, + json={ + "movies": [ + {"movie": {"ids": {"tmdb": 1}}}, + {"movie": {"ids": {"tmdb": 2}}}, + ], + "pagination": {"has_more": True}, + }, + ) + return httpx.Response( + 200, + json={ + "movies": [{"movie": {"ids": {"tmdb": 3}}}], + "pagination": {"has_more": False}, + }, + ) + + transport = httpx.MockTransport(handler) + with patch.object( + mdblist.httpx, + "AsyncClient", + side_effect=lambda **kwargs: _REAL_ASYNC_CLIENT(transport=transport, **kwargs), + ): + result = await mdblist.get_watchlist("secret-key") + + self.assertEqual(offsets, [0, 2]) + self.assertEqual(len(result["movies"]), 3) + + async def test_push_watched_batches_each_media_type(self) -> None: + calls: list[tuple[str, int]] = [] + + def handler(request: httpx.Request) -> httpx.Response: + self.assertEqual(request.url.path, "/sync/watched") + self.assertEqual(request.url.params["apikey"], "secret-key") + payload = json.loads(request.content) + self.assertEqual(len(payload), 1) + key, values = next(iter(payload.items())) + calls.append((key, len(values))) + return httpx.Response(200, json={"added": {}, "not_found": {}}) + + payload = { + "movies": [{"ids": {"tmdb": value}} for value in (1, 2, 3)], + "shows": [], + "seasons": [], + "episodes": [{"ids": {"tmdb": 4}}], + } + transport = httpx.MockTransport(handler) + with ( + patch.object(mdblist, "PUSH_BATCH_SIZE", 2), + patch.object( + mdblist.httpx, + "AsyncClient", + side_effect=lambda **kwargs: _REAL_ASYNC_CLIENT(transport=transport, **kwargs), + ), + ): + result = await mdblist.push_watched("secret-key", payload) + + self.assertEqual(calls, [("movies", 2), ("movies", 1), ("episodes", 1)]) + self.assertEqual(result, {"submitted": 4, "batches": 3, "not_found": 0}) + + + +class MDBListListFanoutTests(unittest.IsolatedAsyncioTestCase): + async def test_managed_watchlist_edit_pushes_to_mdblist(self) -> None: + result = MagicMock() + result.scalar_one_or_none.return_value = SimpleNamespace( + mdblist_push_watchlist=True, + mdblist_api_key="secret-key", + ) + db = AsyncMock() + db.execute.return_value = result + media = Media(id=1, tmdb_id=550, media_type=MediaType.movie, title="Fight Club") + push_watchlist = AsyncMock() + + with patch.object(mdblist, "push_watchlist", push_watchlist): + await _push_list_item_to_mdblist( + db, + user_id=1, + list_mdblist_slug="__watchlist__", + media=media, + ) + + push_watchlist.assert_awaited_once_with( + "secret-key", + { + "movies": [{"ids": {"tmdb": 550}}], + "shows": [], + "seasons": [], + "episodes": [], + }, + ) + + +class MDBListNormalizationTests(unittest.IsolatedAsyncioTestCase): + def test_episode_identity_accepts_nested_show_shape(self) -> None: + entry = { + "episode": {"season": 3, "number": 2, "title": "Caballo sin Nombre"}, + "show": {"ids": {"tmdb": 1396}}, + } + self.assertEqual(_episode_identity(entry), (1396, 3, 2, "Caballo sin Nombre")) + + async def test_show_imdb_id_resolves_to_tmdb_once(self) -> None: + find = AsyncMock(return_value={"tv_results": [{"id": 1396}]}) + cache: dict[tuple[str, str], int | None] = {} + with patch("core.tmdb.find_by_external_id", find): + first = await _resolve_external_tmdb_id( + {"ids": {"imdb": "tt0903747"}}, + "tv", + "tmdb-token", + cache, + ) + second = await _resolve_external_tmdb_id( + {"ids": {"imdb": "tt0903747"}}, + "tv", + "tmdb-token", + cache, + ) + + self.assertEqual((first, second), (1396, 1396)) + find.assert_awaited_once_with("tt0903747", "imdb_id", api_key="tmdb-token") + + def test_payload_item_uses_episode_tmdb_identifier(self) -> None: + media = Media( + id=1, + tmdb_id=62085, + media_type=MediaType.episode, + title="Caballo sin Nombre", + season_number=3, + episode_number=2, + ) + kind, item = _payload_item(media, watched_at=datetime(2026, 7, 17, 12, 0, 0)) + self.assertEqual(kind, "episodes") + self.assertEqual(item["ids"], {"tmdb": 62085}) + self.assertEqual(item["watched_at"], "2026-07-17T12:00:00Z") + + def test_payload_item_preserves_rating_timestamp(self) -> None: + media = Media(id=1, tmdb_id=550, media_type=MediaType.movie, title="Fight Club") + kind, item = _payload_item( + media, + rating=8.0, + rated_at=datetime(2026, 7, 17, 12, 0, 0), + ) + self.assertEqual(kind, "movies") + self.assertEqual(item["rating"], 8.0) + self.assertEqual(item["rated_at"], "2026-07-17T12:00:00Z") + + +if __name__ == "__main__": + unittest.main() diff --git a/backend/tests/test_nuvio.py b/backend/tests/test_nuvio.py new file mode 100644 index 0000000..432c1cc --- /dev/null +++ b/backend/tests/test_nuvio.py @@ -0,0 +1,465 @@ +import json +from datetime import datetime, timezone +import os +import unittest +from unittest.mock import patch + +import httpx + +os.environ.setdefault("SECRET_KEY", "test-secret") +os.environ.setdefault("DATABASE_URL", "postgresql+asyncpg://test:test@localhost/test") + +from core import nuvio +from models.base import MediaType +from models.media import Media +from models.playback_progress import PlaybackProgress +from models.show import Show +from routers.sync import ( + _ensure_nuvio_imdb_ids, + _normalize_nuvio_item, + _nuvio_progress_item, + _nuvio_watched_item, +) + + +_REAL_ASYNC_CLIENT = httpx.AsyncClient + + +class NuvioClientTests(unittest.IsolatedAsyncioTestCase): + async def test_pull_sync_data_refreshes_session_and_paginates_library(self) -> None: + library_offsets: list[int] = [] + + def handler(request: httpx.Request) -> httpx.Response: + if request.url.path == "/auth/v1/token": + self.assertEqual(request.url.params["grant_type"], "refresh_token") + self.assertEqual(json.loads(request.content), {"refresh_token": "old-refresh"}) + return httpx.Response( + 200, + json={ + "access_token": "access-token", + "refresh_token": "new-refresh", + "expires_in": 3600, + }, + ) + self.assertEqual(request.headers["authorization"], "Bearer access-token") + payload = json.loads(request.content or b"{}") + if request.url.path.endswith("/sync_pull_profiles"): + return httpx.Response(200, json=[{"profile_index": 2, "name": "Main"}]) + if request.url.path.endswith("/sync_pull_library"): + library_offsets.append(payload["p_offset"]) + item_count = 500 if payload["p_offset"] == 0 else 1 + return httpx.Response( + 200, + json=[ + {"content_id": f"tmdb:{index + payload['p_offset']}", "content_type": "movie"} + for index in range(item_count) + ], + ) + if request.url.path.endswith("/sync_pull_watched_items"): + return httpx.Response( + 200, + json=[{"content_id": "tmdb:550", "content_type": "movie", "watched_at": 1711600000000}], + ) + if request.url.path.endswith("/sync_pull_watch_progress"): + return httpx.Response( + 200, + json=[{"content_id": "tmdb:550", "content_type": "movie", "position": 1000, "duration": 2000}], + ) + return httpx.Response(404, json={"message": "unexpected request"}) + + transport = httpx.MockTransport(handler) + with patch.object( + nuvio.httpx, + "AsyncClient", + side_effect=lambda **kwargs: _REAL_ASYNC_CLIENT(transport=transport, **kwargs), + ): + session, data = await nuvio.pull_sync_data( + "https://api.nuvio.tv/", + "old-refresh", + 2, + ) + + self.assertEqual(session.refresh_token, "new-refresh") + self.assertEqual(library_offsets, [0, 500]) + self.assertEqual(len(data["library"]), 501) + self.assertEqual(len(data["watched"]), 1) + self.assertEqual(len(data["progress"]), 1) + + async def test_pull_watch_progress_omits_unsupported_offset_param(self) -> None: + """Regression test: sync_pull_watch_progress has no p_offset parameter + on the real API — sending one 404s with "could not find the function" + because PostgREST can't match the signature. Only p_profile_id/p_limit + are valid.""" + + def handler(request: httpx.Request) -> httpx.Response: + if request.url.path == "/auth/v1/token": + return httpx.Response( + 200, + json={"access_token": "access-token", "refresh_token": "new-refresh", "expires_in": 3600}, + ) + payload = json.loads(request.content or b"{}") + if request.url.path.endswith("/sync_pull_profiles"): + return httpx.Response(200, json=[{"profile_index": 2, "name": "Main"}]) + if request.url.path.endswith("/sync_pull_library"): + return httpx.Response(200, json=[]) + if request.url.path.endswith("/sync_pull_watched_items"): + return httpx.Response(200, json=[]) + if request.url.path.endswith("/sync_pull_watch_progress"): + if "p_offset" in payload: + return httpx.Response( + 404, + json={ + "message": "Could not find the function public.sync_pull_watch_progress" + "(p_limit, p_offset, p_profile_id) in the schema cache" + }, + ) + self.assertEqual(payload, {"p_profile_id": 2, "p_limit": 200}) + return httpx.Response( + 200, + json=[{"content_id": "tmdb:550", "content_type": "movie", "position": 1, "duration": 2}], + ) + return httpx.Response(404, json={"message": "unexpected request"}) + + transport = httpx.MockTransport(handler) + with patch.object( + nuvio.httpx, + "AsyncClient", + side_effect=lambda **kwargs: _REAL_ASYNC_CLIENT(transport=transport, **kwargs), + ): + _, data = await nuvio.pull_sync_data("https://api.nuvio.tv/", "old-refresh", 2) + + self.assertEqual(len(data["progress"]), 1) + + async def test_pull_sync_data_tolerates_null_profile_index(self) -> None: + """Regression test: a profile with profile_index: null must not crash + pull_sync_data the way it previously crashed on int(None).""" + + def handler(request: httpx.Request) -> httpx.Response: + if request.url.path == "/auth/v1/token": + return httpx.Response( + 200, + json={"access_token": "access-token", "refresh_token": "new-refresh", "expires_in": 3600}, + ) + if request.url.path.endswith("/sync_pull_profiles"): + return httpx.Response(200, json=[{"profile_index": None, "name": "Kids"}, {"profile_index": 2, "name": "Main"}]) + if request.url.path.endswith(("/sync_pull_library", "/sync_pull_watched_items", "/sync_pull_watch_progress")): + return httpx.Response(200, json=[]) + return httpx.Response(404, json={"message": "unexpected request"}) + + transport = httpx.MockTransport(handler) + with patch.object( + nuvio.httpx, + "AsyncClient", + side_effect=lambda **kwargs: _REAL_ASYNC_CLIENT(transport=transport, **kwargs), + ): + session, data = await nuvio.pull_sync_data("https://api.nuvio.tv/", "old-refresh", 2) + + self.assertEqual(session.refresh_token, "new-refresh") + + async def test_push_watched_items_batches_without_full_replace(self) -> None: + batch_sizes: list[int] = [] + + def handler(request: httpx.Request) -> httpx.Response: + if request.url.path == "/auth/v1/token": + return httpx.Response( + 200, + json={ + "access_token": "access-token", + "refresh_token": "rotated-refresh", + "expires_in": 3600, + }, + ) + if request.url.path.endswith("/sync_push_watched_items"): + payload = json.loads(request.content) + self.assertEqual(payload["p_profile_id"], 1) + batch_sizes.append(len(payload["p_items"])) + return httpx.Response(204) + return httpx.Response(404, json={"message": "unexpected request"}) + + transport = httpx.MockTransport(handler) + items = [ + { + "content_id": f"tmdb:{index}", + "content_type": "movie", + "watched_at": 1711600000000, + } + for index in range(501) + ] + with patch.object( + nuvio.httpx, + "AsyncClient", + side_effect=lambda **kwargs: _REAL_ASYNC_CLIENT(transport=transport, **kwargs), + ): + session = await nuvio.push_watched_items( + "https://api.nuvio.tv", + "old-refresh", + 1, + items, + ) + + self.assertEqual(session.refresh_token, "rotated-refresh") + self.assertEqual(batch_sizes, [500, 1]) + + async def test_push_sync_items_uses_watched_and_progress_endpoints(self) -> None: + calls: list[tuple[str, int]] = [] + refresh_count = 0 + + def handler(request: httpx.Request) -> httpx.Response: + nonlocal refresh_count + if request.url.path == "/auth/v1/token": + refresh_count += 1 + return httpx.Response( + 200, + json={ + "access_token": "access-token", + "refresh_token": "rotated-refresh", + "expires_in": 3600, + }, + ) + payload = json.loads(request.content) + self.assertEqual(payload["p_profile_id"], 3) + function_name = request.url.path.rsplit("/", 1)[-1] + items_key = "p_entries" if function_name == "sync_push_watch_progress" else "p_items" + self.assertEqual(set(payload), {"p_profile_id", items_key}) + calls.append((function_name, len(payload[items_key]))) + return httpx.Response(204) + + watched_items = [{"content_id": "tmdb:550", "content_type": "movie", "watched_at": 1}] + progress_items = [ + { + "content_id": f"tmdb:{index}", + "content_type": "movie", + "video_id": f"tmdb:{index}", + "position": 1000, + "duration": 2000, + "last_watched": 1, + } + for index in range(501) + ] + transport = httpx.MockTransport(handler) + with patch.object( + nuvio.httpx, + "AsyncClient", + side_effect=lambda **kwargs: _REAL_ASYNC_CLIENT(transport=transport, **kwargs), + ): + session = await nuvio.push_sync_items( + "https://api.nuvio.tv", + "old-refresh", + 3, + watched_items, + progress_items, + ) + + self.assertEqual(session.refresh_token, "rotated-refresh") + self.assertEqual(refresh_count, 1) + self.assertEqual( + calls, + [ + ("sync_push_watched_items", 1), + ("sync_push_watch_progress", 500), + ("sync_push_watch_progress", 1), + ], + ) + + async def test_missing_imdb_ids_are_resolved_and_cached(self) -> None: + movie = Media( + id=10, + tmdb_id=550, + media_type=MediaType.movie, + title="Fight Club", + tmdb_data={}, + ) + episode = Media( + id=11, + media_type=MediaType.episode, + title="It's All Good", + show_id=5, + season_number=3, + episode_number=2, + ) + show = Show(id=5, tmdb_id=125988, title="Silo", tmdb_data={}) + + async def external_ids(tmdb_id: int, media_type: str, api_key: str | None = None) -> dict: + self.assertEqual(api_key, "tmdb-token") + return { + "imdb_id": { + ("movie", 550): "tt0137523", + ("tv", 125988): "tt14688458", + }[(media_type, tmdb_id)] + } + + with patch("routers.sync.tmdb.get_external_ids", side_effect=external_ids) as get_external_ids: + await _ensure_nuvio_imdb_ids( + [movie, episode], + {show.id: show}, + "tmdb-token", + ) + + self.assertEqual(get_external_ids.await_count, 2) + self.assertEqual(movie.tmdb_data["external_ids"]["imdb_id"], "tt0137523") + self.assertEqual(show.tmdb_data["external_ids"]["imdb_id"], "tt14688458") + + +class NuvioNormalizationTests(unittest.TestCase): + def test_episode_history_maps_to_tmdb_series_and_watch_state(self) -> None: + normalized = _normalize_nuvio_item( + { + "content_id": "tmdb:1396", + "content_type": "series", + "title": "Pilot", + "season": 1, + "episode": 1, + "watched_at": 1711600000000, + }, + profile_id=3, + watched=True, + ) + + self.assertIsNotNone(normalized) + media_type, item = normalized + self.assertEqual(media_type, MediaType.episode) + self.assertEqual(item["Id"], "3:tmdb:1396:s1e1") + self.assertEqual(item["SeriesId"], "tmdb:1396") + self.assertEqual(item["ProviderIds"], {}) + self.assertEqual(item["UserData"]["Played"], True) + self.assertEqual(item["UserData"]["PlayCount"], 1) + self.assertIsNotNone(item["UserData"]["LastPlayedDate"]) + + def test_imdb_content_uses_resolved_tmdb_id(self) -> None: + normalized = _normalize_nuvio_item( + { + "content_id": "tt0411008", + "content_type": "series", + "name": "Lost", + }, + profile_id=1, + tmdb_id=4607, + ) + + self.assertIsNotNone(normalized) + media_type, item = normalized + self.assertEqual(media_type, MediaType.series) + self.assertEqual(item["Id"], "1:tt0411008") + self.assertEqual(item["ProviderIds"], {"Tmdb": "4607"}) + + def test_unsupported_content_identifier_is_skipped(self) -> None: + self.assertIsNone( + _normalize_nuvio_item( + {"content_id": "imdb:tt0137523", "content_type": "movie"}, + profile_id=1, + ) + ) + + def test_progress_payload_maps_movies_and_episodes(self) -> None: + updated_at = datetime(2026, 7, 14, 12, 0, tzinfo=timezone.utc) + progress = PlaybackProgress( + user_id=1, + media_id=10, + progress_seconds=1800, + progress_percent=0.25, + updated_at=updated_at, + ) + movie = Media( + id=10, + tmdb_id=550, + media_type=MediaType.movie, + title="Fight Club", + runtime=120, + tmdb_data={"external_ids": {"imdb_id": "tt0137523"}}, + ) + self.assertEqual( + _nuvio_progress_item(progress, movie), + { + "content_id": "tt0137523", + "content_type": "movie", + "video_id": "tt0137523", + "position": 1800000, + "duration": 7200000, + "last_watched": int(updated_at.timestamp() * 1000), + }, + ) + + episode = Media( + id=11, + media_type=MediaType.episode, + title="Pilot", + show_id=5, + season_number=1, + episode_number=1, + ) + show = Show( + id=5, + tmdb_id=1396, + title="Breaking Bad", + tmdb_data={"external_ids": {"imdb_id": "tt0903747"}}, + ) + self.assertEqual( + _nuvio_progress_item(progress, episode, show), + { + "content_id": "tt0903747", + "content_type": "series", + "video_id": "tt0903747:1:1", + "season": 1, + "episode": 1, + "position": 1800000, + "duration": 7200000, + "last_watched": int(updated_at.timestamp() * 1000), + }, + ) + + def test_watched_payload_uses_bare_imdb_ids(self) -> None: + watched_at = datetime(2026, 7, 14, 12, 0, tzinfo=timezone.utc) + movie = Media( + id=10, + tmdb_id=550, + media_type=MediaType.movie, + title="Fight Club", + tmdb_data={"external_ids": {"imdb_id": "tt0137523"}}, + ) + self.assertEqual( + _nuvio_watched_item(movie, watched_at), + { + "content_id": "tt0137523", + "content_type": "movie", + "title": "Fight Club", + "watched_at": int(watched_at.timestamp() * 1000), + }, + ) + + episode = Media( + id=11, + media_type=MediaType.episode, + title="It's All Good", + show_id=5, + season_number=3, + episode_number=2, + ) + show = Show( + id=5, + tmdb_id=125988, + title="Silo", + tmdb_data={"external_ids": {"imdb_id": "tt14688458"}}, + ) + self.assertEqual( + _nuvio_watched_item(episode, watched_at, show), + { + "content_id": "tt14688458", + "content_type": "series", + "title": "It's All Good", + "season": 3, + "episode": 2, + "watched_at": int(watched_at.timestamp() * 1000), + }, + ) + + tmdb_only_movie = Media( + id=12, + tmdb_id=550, + media_type=MediaType.movie, + title="Fight Club", + ) + self.assertIsNone(_nuvio_watched_item(tmdb_only_movie, watched_at)) + + +if __name__ == "__main__": + unittest.main() diff --git a/backend/tests/test_trakt.py b/backend/tests/test_trakt.py new file mode 100644 index 0000000..659b0a5 --- /dev/null +++ b/backend/tests/test_trakt.py @@ -0,0 +1,122 @@ +import unittest +from unittest.mock import patch + +import httpx + +from core import trakt + + +_REAL_ASYNC_CLIENT = httpx.AsyncClient + + +class TraktClientTests(unittest.IsolatedAsyncioTestCase): + async def test_get_watched_movies_fetches_every_page(self) -> None: + requested_pages: list[int] = [] + + def handler(request: httpx.Request) -> httpx.Response: + self.assertEqual(request.url.path, "/sync/watched/movies") + self.assertEqual(request.url.params["limit"], "250") + self.assertEqual(request.headers["authorization"], "Bearer access-token") + + page = int(request.url.params["page"]) + requested_pages.append(page) + page_items = { + 1: [{"movie": {"ids": {"tmdb": index}}} for index in range(1, 251)], + 2: [{"movie": {"ids": {"tmdb": index}}} for index in range(251, 501)], + 3: [{"movie": {"ids": {"tmdb": index}}} for index in range(501, 518)], + }[page] + return httpx.Response( + 200, + json=page_items, + headers={"X-Pagination-Page-Count": "3"}, + ) + + transport = httpx.MockTransport(handler) + with patch.object( + trakt.httpx, + "AsyncClient", + side_effect=lambda **kwargs: _REAL_ASYNC_CLIENT(transport=transport, **kwargs), + ): + movies = await trakt.get_watched_movies("client-id", "access-token") + + self.assertEqual(requested_pages, [1, 2, 3]) + self.assertEqual(len(movies), 517) + self.assertEqual(movies[-1]["movie"]["ids"]["tmdb"], 517) + + + async def test_get_watched_shows_requests_progress(self) -> None: + requested_pages: list[int] = [] + + def handler(request: httpx.Request) -> httpx.Response: + self.assertEqual(request.url.path, "/sync/watched/shows") + self.assertEqual(request.url.params["limit"], "250") + self.assertEqual(request.url.params["extended"], "progress") + page = int(request.url.params["page"]) + requested_pages.append(page) + # Trakt's /sync/watched/* endpoints are not paginated: they always + # return the complete list and never send X-Pagination-Page-Count. + return httpx.Response( + 200, + json=[ + { + "show": {"ids": {"tmdb": 1396}}, + "seasons": [ + { + "number": 1, + "episodes": [ + { + "number": 1, + "plays": 1, + "last_watched_at": "2026-07-15T20:00:00.000Z", + } + ], + } + ], + } + ], + ) + + transport = httpx.MockTransport(handler) + with patch.object( + trakt.httpx, + "AsyncClient", + side_effect=lambda **kwargs: _REAL_ASYNC_CLIENT(transport=transport, **kwargs), + ): + shows = await trakt.get_watched_shows("client-id", "access-token") + + # No pagination header means this was the only (complete) page — the + # client must not keep re-requesting a non-paginated endpoint forever. + self.assertEqual(requested_pages, [1]) + self.assertEqual(shows[0]["seasons"][0]["episodes"][0]["number"], 1) + + + async def test_get_watched_movies_stops_without_page_count_header(self) -> None: + """Regression test: a non-paginating endpoint that omits + X-Pagination-Page-Count must not be re-requested forever.""" + request_count = 0 + + def handler(request: httpx.Request) -> httpx.Response: + nonlocal request_count + request_count += 1 + # Simulate Trakt's real behavior for /sync/watched/*: the full, + # non-empty list is returned on every request, with no pagination + # header, regardless of the requested page. + return httpx.Response( + 200, + json=[{"movie": {"ids": {"tmdb": 1}}}], + ) + + transport = httpx.MockTransport(handler) + with patch.object( + trakt.httpx, + "AsyncClient", + side_effect=lambda **kwargs: _REAL_ASYNC_CLIENT(transport=transport, **kwargs), + ): + movies = await trakt.get_watched_movies("client-id", "access-token") + + self.assertEqual(request_count, 1) + self.assertEqual(len(movies), 1) + + +if __name__ == "__main__": + unittest.main() diff --git a/frontend/src/lib/api.ts b/frontend/src/lib/api.ts index 3c240e1..4590d2e 100644 --- a/frontend/src/lib/api.ts +++ b/frontend/src/lib/api.ts @@ -415,6 +415,16 @@ export interface UserSettings { simkl_push_ratings: boolean; simkl_scrobble: boolean; + // MDBList + mdblist_api_key: string | null; + mdblist_connected: boolean; + mdblist_sync_watched: boolean; + mdblist_sync_ratings: boolean; + mdblist_sync_watchlist: boolean; + mdblist_push_watched: boolean; + mdblist_push_ratings: boolean; + mdblist_push_watchlist: boolean; + preferences: UserPreferences | null; blur_explicit: boolean; time_format_24h: boolean; @@ -424,7 +434,7 @@ export interface UserSettings { export interface MediaServerConnection { id: number; user_id: number; - type: "jellyfin" | "emby" | "plex"; + type: "jellyfin" | "emby" | "plex" | "nuvio"; name: string; url: string; token: string; @@ -435,13 +445,14 @@ export interface MediaServerConnection { sync_ratings: boolean; sync_playback: boolean; push_watched: boolean; + push_playback: boolean; push_ratings: boolean; auto_sync_interval: number | null; created_at: string; } export interface MediaServerConnectionCreate { - type: "jellyfin" | "emby" | "plex"; + type: "jellyfin" | "emby" | "plex" | "nuvio"; name: string; url: string; token: string; @@ -452,6 +463,7 @@ export interface MediaServerConnectionCreate { sync_ratings?: boolean; sync_playback?: boolean; push_watched?: boolean; + push_playback?: boolean; push_ratings?: boolean; auto_sync_interval?: number | null; } @@ -496,6 +508,7 @@ export interface ConnectionStatus { sonarr: ServiceStatus; trakt: ServiceStatus; simkl: ServiceStatus; + mdblist: ServiceStatus; } export interface MediaItem { @@ -1003,6 +1016,13 @@ export const api = { post<{ status: string; message: string }>("/simkl/push", undefined, token), }, + mdblist: { + sync: (token: string) => + post<{ status: string; job_id: number; message: string }>("/mdblist/sync", undefined, token), + push: (token: string) => + post<{ status: string; job_id: number; message: string }>("/mdblist/push", undefined, token), + }, + media: { list: (params?: { type?: string; sort?: string; page?: number; genre?: string; year?: number }, token?: string) => get<{ page: number; page_size: number; total_pages: number; total_results: number; results: MediaItem[] }>("/media", params, token), diff --git a/frontend/src/pages/admin.astro b/frontend/src/pages/admin.astro index 0c82bf3..7096642 100644 --- a/frontend/src/pages/admin.astro +++ b/frontend/src/pages/admin.astro @@ -53,7 +53,7 @@ const adminCount = users.filter((u: AdminUser) => u.is_admin).length; @@ -433,7 +433,7 @@ const adminCount = users.filter((u: AdminUser) => u.is_admin).length;

Heal missing metadata

-

Re-enrich all items across every user's collection that are missing poster or date metadata. Requires a global TMDB API key to be set.

+

Re-enrich all items across every user's collection that are missing poster or date metadata. Requires a global TMDB Read Access Token to be set.

{settings.has_global_tmdb_key && !settings.tmdb_api_key ? ( -

A server-wide TMDB key is configured by your admin. You can leave this blank to use it, or enter your own key to override it.

+

A server-wide TMDB token is configured by your admin. You can leave this blank to use it, or enter your own token to override it.

) : settings.has_global_tmdb_key && settings.tmdb_api_key ? ( -

A server-wide TMDB key is configured, but you are using your own key. Clear this field to fall back to the server key.

+

A server-wide TMDB token is configured, but you are using your own token. Clear this field to fall back to the server token.

) : ( -

Required for fetching metadata and posters. Will be validated before saving.

+

Paste the API Read Access Token (v4) from TMDB Settings → API. Required for metadata and posters; validated before saving.

)} @@ -137,13 +137,13 @@ if (me.totp_enabled) {
-

Media Server Connections

+

Media & Cloud Connections

{connections.map((conn) => { - const typeColor = conn.type === "jellyfin" ? "bg-purple-500" : conn.type === "emby" ? "bg-green-500" : "bg-yellow-500"; - const typeLabel = conn.type === "jellyfin" ? "Jellyfin" : conn.type === "emby" ? "Emby" : "Plex"; + const typeColor = conn.type === "jellyfin" ? "bg-purple-500" : conn.type === "emby" ? "bg-green-500" : conn.type === "plex" ? "bg-yellow-500" : "bg-cyan-500"; + const typeLabel = conn.type === "jellyfin" ? "Jellyfin" : conn.type === "emby" ? "Emby" : conn.type === "plex" ? "Plex" : "Nuvio"; return (
@@ -188,11 +188,11 @@ if (me.totp_enabled) {
- - + +
- +
- {conn.type !== "plex" && ( + {conn.type === "nuvio" ? ( +
+ + +
+ ) : conn.type !== "plex" && (
@@ -214,35 +221,39 @@ if (me.totp_enabled) {
)} -
- -
-
Checking connection…
-

Leave all unchecked to sync every library.

-
- + {conn.type !== "nuvio" && ( + <> +
+ +
+
Checking connection…
+

Leave all unchecked to sync every library.

+
+ +
+
-
-
-
- -
- - -
-
+
+ +
+ + +
+
+ + )}

{typeLabel} → Scrob

{[ - { key: "sync_collection", label: "Collection status" }, - { key: "sync_watched", label: "Watched status" }, - { key: "sync_ratings", label: "Ratings" }, - { key: "sync_playback", label: "Playback events" }, + { key: "sync_collection", label: "Collection status" }, + { key: "sync_watched", label: "Watched status" }, + ...(conn.type === "nuvio" ? [] : [{ key: "sync_ratings", label: "Ratings" }]), + { key: "sync_playback", label: conn.type === "nuvio" ? "Playback progress" : "Playback events" }, ...(conn.type === "plex" ? [{ key: "plex_sync_watchlist", label: "Watchlist" }] : []), ].map(({ key, label }) => (
@@ -263,7 +274,8 @@ if (me.totp_enabled) {

Scrob → {typeLabel}

{[ { key: "push_watched", label: "Watched status" }, - { key: "push_ratings", label: "Ratings" }, + ...(conn.type === "nuvio" ? [{ key: "push_playback", label: "Playback progress" }] : []), + ...(conn.type === "nuvio" ? [] : [{ key: "push_ratings", label: "Ratings" }]), ...(conn.type === "plex" ? [{ key: "plex_push_watchlist", label: "Watchlist" }] : []), ].map(({ key, label }) => ( ))}
@@ -282,8 +294,17 @@ if (me.totp_enabled) {
@@ -350,19 +371,24 @@ if (me.totp_enabled) {
- +
-
- - +
+ + +
+
- +
+
-

Server → Scrob

+

Server → Scrob

{[ { id: "new-sync-collection", label: "Collection status", defaultOn: true }, { id: "new-sync-watched", label: "Watched status", defaultOn: true }, { id: "new-sync-ratings", label: "Ratings", defaultOn: true }, { id: "new-sync-playback", label: "Playback events", defaultOn: true }, ].map(({ id, label, defaultOn }) => ( -
-

Scrob → Server

+

Scrob → Server

{[ { id: "new-push-watched", label: "Watched status" }, { id: "new-push-ratings", label: "Ratings" }, ].map(({ id, label }) => ( -

Auto Sync

@@ -418,7 +454,16 @@ if (me.totp_enabled) {
@@ -606,7 +651,7 @@ if (me.totp_enabled) {
{settings.trakt_connected ? ( - + ) : ( )} @@ -703,7 +748,7 @@ if (me.totp_enabled) { type="button" data-action="sync_trakt" disabled={!hasTmdbKey} - title={!hasTmdbKey ? "TMDB API Key is required for sync" : "Pull watched history and ratings from Trakt"} + title={!hasTmdbKey ? "TMDB Read Access Token is required for sync" : "Pull watched history and ratings from Trakt"} class={`js-action-btn text-sm px-3 py-1.5 rounded-lg transition-colors cursor-pointer disabled:opacity-50 ${hasTmdbKey ? "bg-red-500/10 hover:bg-red-500/20 text-red-400 border border-red-500/30" : "bg-zinc-800 text-zinc-400 border border-zinc-700 opacity-50"}`} >Pull
@@ -751,7 +796,7 @@ if (me.totp_enabled) {
{settings.simkl_connected ? ( - + ) : ( )} @@ -820,7 +865,7 @@ if (me.totp_enabled) { type="button" data-action="sync_simkl" disabled={!hasTmdbKey} - title={!hasTmdbKey ? "TMDB API Key is required for sync" : "Pull watched history and ratings from Simkl"} + title={!hasTmdbKey ? "TMDB Read Access Token is required for sync" : "Pull watched history and ratings from Simkl"} class={`js-action-btn text-sm px-3 py-1.5 rounded-lg transition-colors cursor-pointer disabled:opacity-50 ${hasTmdbKey ? "bg-teal-500/10 hover:bg-teal-500/20 text-teal-400 border border-teal-500/30" : "bg-zinc-800 text-zinc-400 border border-zinc-700 opacity-50"}`} >Pull
@@ -854,6 +899,101 @@ if (me.totp_enabled) {
+ +
+
+
+ +
+ MDBList + +
+
+ + + +
+ + +
+
@@ -1103,7 +1243,7 @@ if (me.totp_enabled) { id="heal-btn" type="button" disabled={!hasTmdbKey} - title={!hasTmdbKey ? "TMDB API Key is required" : ""} + title={!hasTmdbKey ? "TMDB Read Access Token is required" : ""} class="ml-4 shrink-0 text-sm px-4 py-2 rounded-lg transition-colors cursor-pointer disabled:opacity-50 bg-blue-600/20 hover:bg-blue-600/30 text-blue-400 border border-blue-500/30" > Heal @@ -1889,6 +2029,77 @@ if (me.totp_enabled) { pollTraktJob(); } + // ── MDBList progress tracking ───────────────────────────────────────────── + let mdblistPoller: ReturnType | null = null; + + function renderMdblistProgress(job: any) { + const strip = document.getElementById('mdblist-progress'); + const bar = document.getElementById('mdblist-job-bar'); + const pctEl = document.getElementById('mdblist-job-pct'); + const label = document.getElementById('mdblist-job-label'); + const items = document.getElementById('mdblist-job-items'); + if (!strip) return; + if (!job) { strip.classList.add('hidden'); return; } + strip.classList.remove('hidden'); + + const isPush = job.job_type === 'push'; + const isFailed = job.status === 'failed'; + const isCompleted = job.status === 'completed'; + const isPending = job.status === 'pending'; + + if (isFailed) { + if (bar) { bar.className = 'h-1.5 rounded-full transition-all duration-500 bg-red-500'; bar.style.width = '100%'; } + if (label) label.textContent = isPush ? 'Push failed' : 'Pull failed'; + if (pctEl) pctEl.textContent = ''; + if (items) items.textContent = esc(job.error_message || 'Unknown error'); + return; + } + if (isCompleted) { + if (bar) { bar.className = 'h-1.5 rounded-full transition-all duration-500 bg-violet-500'; bar.style.width = '100%'; } + if (label) label.textContent = isPush ? 'Push complete' : 'Pull complete'; + if (pctEl) pctEl.textContent = '100%'; + if (items) items.textContent = `${job.processed_items ?? 0} items`; + return; + } + + const pct = job.total_items > 0 ? Math.min(100, Math.round((job.processed_items / job.total_items) * 100)) : 0; + const indeterminate = isPending || pct === 0; + if (bar) { + bar.className = `h-1.5 rounded-full transition-all duration-500 bg-violet-500${indeterminate ? ' animate-pulse' : ''}`; + bar.style.width = indeterminate ? '100%' : `${pct}%`; + } + if (label) label.textContent = isPush ? (isPending ? 'Push queued…' : 'Pushing…') : (isPending ? 'Pull queued…' : 'Pulling…'); + if (pctEl) pctEl.textContent = pct > 0 ? `${pct}%` : ''; + if (items) items.textContent = job.total_items > 0 ? `${job.processed_items} / ${job.total_items} items` : 'Loading…'; + } + + async function pollMdblistJob() { + try { + const res = await fetch('/api/proxy/sync/status', { headers: { 'Authorization': `Bearer ${token}` } }); + if (!res.ok) return; + const jobs: any[] = await res.json(); + const job = jobs.find(j => j.source === 'mdblist'); + const isActive = job && (job.status === 'pending' || job.status === 'running'); + const isRecentDone = job && (job.status === 'completed' || job.status === 'failed') + && new Date(job.updated_at).getTime() > Date.now() - 8000; + renderMdblistProgress((isActive || isRecentDone) ? job : null); + if (isActive) { + mdblistPoller = setTimeout(pollMdblistJob, 2000); + } else if (isRecentDone) { + mdblistPoller = setTimeout(() => { renderMdblistProgress(null); mdblistPoller = null; }, 4000); + } else { + mdblistPoller = null; + } + } catch { + mdblistPoller = null; + } + } + + function startMdblistPoller() { + if (mdblistPoller != null) clearTimeout(mdblistPoller); + pollMdblistJob(); + } + // ── Heal progress tracking ──────────────────────────────────────────────── let healPoller: ReturnType | null = null; @@ -1979,6 +2190,9 @@ if (me.totp_enabled) { if (job.source === 'trakt' && (job.status === 'pending' || job.status === 'running')) { startTraktPoller(); } + if (job.source === 'mdblist' && (job.status === 'pending' || job.status === 'running')) { + startMdblistPoller(); + } if (job.source === 'tmdb' && job.job_type === 'heal' && (job.status === 'pending' || job.status === 'running')) { startHealPoller(); } @@ -2054,17 +2268,19 @@ if (me.totp_enabled) { if (!token) return; const traktConnected = !!document.getElementById('trakt-disconnect-btn'); const simklConnected = !!document.getElementById('simkl-disconnect-btn'); + const mdblistConfigured = !!document.getElementById('mdblist_api_key')?.value; const radarrConfigured = !!document.getElementById('radarr_url')?.value; const sonarrConfigured = !!document.getElementById('sonarr_url')?.value; const mediaServerIds = Array.from(document.querySelectorAll('[data-connection-id]')) .map(el => el.getAttribute('data-connection-id')) .filter(Boolean) as string[]; - if (!radarrConfigured && !sonarrConfigured && !traktConnected && !simklConnected && mediaServerIds.length === 0) return; + if (!radarrConfigured && !sonarrConfigured && !traktConnected && !simklConnected && !mdblistConfigured && mediaServerIds.length === 0) return; if (radarrConfigured) setConnStatus('radarr-conn-status', 'checking'); if (sonarrConfigured) setConnStatus('sonarr-conn-status', 'checking'); if (traktConnected) setConnStatus('trakt-conn-status', 'checking'); if (simklConnected) setConnStatus('simkl-conn-status', 'checking'); + if (mdblistConfigured) setConnStatus('mdblist-conn-status', 'checking'); for (const id of mediaServerIds) setConnStatus(`conn-status-${id}`, 'checking'); try { @@ -2110,8 +2326,29 @@ if (me.totp_enabled) { if (data.simkl?.configured) { setConnStatus('simkl-conn-status', data.simkl.connected ? 'connected' : 'disconnected'); } + if (data.mdblist?.configured) { + setConnStatus('mdblist-conn-status', data.mdblist.connected ? 'connected' : 'disconnected'); + } for (const cs of (data.connections ?? [])) { setConnStatus(`conn-status-${cs.id}`, cs.connected ? 'connected' : 'disconnected'); + if (cs.token) { + const card = document.querySelector(`[data-connection-id="${cs.id}"]`); + const tokenInput = card?.querySelector('.conn-token-input') as HTMLInputElement | null; + if (tokenInput) tokenInput.value = cs.token; + } + if (cs.profiles?.length) { + const card = document.querySelector(`[data-connection-id="${cs.id}"]`); + const profileSelect = card?.querySelector('.conn-user-id-input') as HTMLSelectElement | null; + if (profileSelect) { + const selectedProfile = profileSelect.value; + profileSelect.innerHTML = cs.profiles.map((profile: any) => + `` + ).join(''); + if ([...profileSelect.options].some(option => option.value === selectedProfile)) { + profileSelect.value = selectedProfile; + } + } + } if (cs.connected) loadLibraries(String(cs.id)); } } catch { @@ -2201,6 +2438,26 @@ if (me.totp_enabled) { }); } + async function persistMdblistPreferences(keys: string[]) { + const preferences = Object.fromEntries(keys.map(key => { + const input = document.querySelector(`input[name="${key}"]`); + return [key, input?.checked ?? false]; + })); + const res = await fetch('/api/proxy/auth/settings', { + method: 'PATCH', + headers: { + 'Authorization': `Bearer ${token}`, + 'Content-Type': 'application/json', + }, + body: JSON.stringify(preferences), + }); + if (!res.ok) { + const data = await res.json().catch(() => ({})); + throw new Error(data.detail || 'Failed to save MDBList options'); + } + } + + function setupActions() { const btns = document.querySelectorAll('.js-action-btn'); btns.forEach(btn => { @@ -2234,19 +2491,39 @@ if (me.totp_enabled) { const u = (card?.querySelector('.conn-url-input') as HTMLInputElement)?.value || ''; const t = (card?.querySelector('.conn-token-input') as HTMLInputElement)?.value || ''; const ui = (card?.querySelector('.conn-user-id-input') as HTMLInputElement)?.value || ''; - let testEndpoint: string; - if (connType === 'plex') { - testEndpoint = `/api/proxy/auth/test-plex?url=${encodeURIComponent(u)}&token=${encodeURIComponent(t)}`; - } else if (connType === 'emby') { - testEndpoint = `/api/proxy/auth/test-emby?url=${encodeURIComponent(u)}&token=${encodeURIComponent(t)}&user_id=${encodeURIComponent(ui)}`; + let testRes: Response; + if (connType === 'nuvio') { + testRes = await fetch('/api/proxy/auth/test-nuvio', { + method: 'POST', + headers: { 'Authorization': `Bearer ${token}`, 'Content-Type': 'application/json' }, + body: JSON.stringify({ url: u, token: t, profile_id: parseInt(ui) }), + }); } else { - testEndpoint = `/api/proxy/auth/test-jellyfin?url=${encodeURIComponent(u)}&token=${encodeURIComponent(t)}&user_id=${encodeURIComponent(ui)}`; + let testEndpoint: string; + if (connType === 'plex') { + testEndpoint = `/api/proxy/auth/test-plex?url=${encodeURIComponent(u)}&token=${encodeURIComponent(t)}`; + } else if (connType === 'emby') { + testEndpoint = `/api/proxy/auth/test-emby?url=${encodeURIComponent(u)}&token=${encodeURIComponent(t)}&user_id=${encodeURIComponent(ui)}`; + } else { + testEndpoint = `/api/proxy/auth/test-jellyfin?url=${encodeURIComponent(u)}&token=${encodeURIComponent(t)}&user_id=${encodeURIComponent(ui)}`; + } + testRes = await fetch(testEndpoint, { method: 'POST', headers: { 'Authorization': `Bearer ${token}` } }); } - const testRes = await fetch(testEndpoint, { method: 'POST', headers: { 'Authorization': `Bearer ${token}` } }); const testData = await testRes.json(); if (testRes.ok) { showMessage(testData.message || 'Connection successful!'); if (connId) setConnStatus(`conn-status-${connId}`, 'connected'); + if (connType === 'nuvio' && connId) { + const saveTokenRes = await fetch(`/api/proxy/auth/connections/${connId}`, { + method: 'PATCH', + headers: { 'Authorization': `Bearer ${token}`, 'Content-Type': 'application/json' }, + body: JSON.stringify({ token: testData.refresh_token }), + }); + if (!saveTokenRes.ok) throw new Error('Nuvio session was validated but could not be saved'); + const savedConnection = await saveTokenRes.json(); + const tokenInput = card?.querySelector('.conn-token-input') as HTMLInputElement | null; + if (tokenInput) tokenInput.value = savedConnection.token; + } } else { if (connId) setConnStatus(`conn-status-${connId}`, 'disconnected'); throw new Error(testData.detail || 'Connection test failed'); @@ -2262,7 +2539,7 @@ if (me.totp_enabled) { server_username: (card?.querySelector('.conn-username-input') as HTMLInputElement)?.value || null, auto_sync_interval: (() => { const v = (card?.querySelector('.conn-auto-sync') as HTMLSelectElement)?.value; - return v ? parseInt(v) : null; + return v ? parseFloat(v) : null; })(), }; card?.querySelectorAll('.conn-pref-checkbox').forEach((cb: any) => { @@ -2283,6 +2560,9 @@ if (me.totp_enabled) { const err = await saveRes.json(); throw new Error(err.detail || 'Failed to save connection'); } + const savedConnection = await saveRes.json(); + const savedTokenInput = card?.querySelector('.conn-token-input') as HTMLInputElement | null; + if (savedTokenInput && savedConnection.token) savedTokenInput.value = savedConnection.token; // Save libraries if they've been loaded const libList = card?.querySelector('.conn-libraries-list'); if (libList && libList.querySelector('.library-checkbox')) { @@ -2387,6 +2667,7 @@ if (me.totp_enabled) { const pushPrefLabels: Record = { push_watched: 'watched status', push_ratings: 'ratings', + push_playback: 'playback progress', }; const pushing: string[] = []; card?.querySelectorAll('.conn-pref-checkbox').forEach((el: any) => { @@ -2576,6 +2857,48 @@ if (me.totp_enabled) { ); if (!ok) return; url = `/api/proxy/trakt/push`; + } else if (action === 'sync_mdblist') { + const labels: Record = { + mdblist_sync_watched: 'watched history', + mdblist_sync_ratings: 'ratings', + mdblist_sync_watchlist: 'watchlist', + }; + const selected = Array.from(document.querySelectorAll('.sync-pref-checkbox')) + .filter(el => el.checked && labels[el.name]) + .map(el => labels[el.name]); + const empty = selected.length === 0; + const ok = await showConfirmModal( + empty ? 'Nothing to pull' : 'Pull from MDBList?', + empty + ? 'Enable at least one option in the "MDBList → Scrob" section.' + : `This will import your ${formatList(selected)} from MDBList into Scrob.`, + empty ? 'Got it' : 'Pull', + empty + ); + if (!ok) return; + await persistMdblistPreferences(Object.keys(labels)); + url = '/api/proxy/mdblist/sync'; + } else if (action === 'push_mdblist') { + const labels: Record = { + mdblist_push_watched: 'watched history', + mdblist_push_ratings: 'ratings', + mdblist_push_watchlist: 'watchlist', + }; + const selected = Array.from(document.querySelectorAll('.sync-pref-checkbox')) + .filter(el => el.checked && labels[el.name]) + .map(el => labels[el.name]); + const empty = selected.length === 0; + const ok = await showConfirmModal( + empty ? 'Nothing to push' : 'Push to MDBList?', + empty + ? 'Enable at least one option in the "Scrob → MDBList" section.' + : `This will add or update your ${formatList(selected)} on MDBList. Existing remote items are retained.`, + empty ? 'Got it' : 'Push', + empty + ); + if (!ok) return; + await persistMdblistPreferences(Object.keys(labels)); + url = '/api/proxy/mdblist/push'; } const res = await fetch(url, { @@ -2589,6 +2912,7 @@ if (me.totp_enabled) { if (syncConnId) startConnPoller(syncConnId); if (action === 'sync_simkl' || action === 'push_simkl') startSimklPoller(); if (action === 'sync_trakt' || action === 'push_trakt') startTraktPoller(); + if (action === 'sync_mdblist' || action === 'push_mdblist') startMdblistPoller(); } else { if (syncConnId) renderConnProgress(syncConnId, null); throw new Error(data.detail || 'Action failed'); @@ -3070,18 +3394,87 @@ if (me.totp_enabled) { cancelBtn.addEventListener('click', () => addForm.classList.add('hidden')); } - // Show/hide user_id vs username based on new connection type + // Provider-specific fields for a new connection const typeSelect = document.getElementById('new-conn-type') as HTMLSelectElement | null; const userIdRow = document.getElementById('new-conn-user-id-row'); const usernameRow = document.getElementById('new-conn-username-row'); + const nuvioEmailRow = document.getElementById('new-nuvio-email-row'); + const nuvioProfileRow = document.getElementById('new-nuvio-profile-row'); + const urlInput = document.getElementById('new-conn-url') as HTMLInputElement | null; + const urlLabel = document.getElementById('new-conn-url-label'); + const tokenInput = document.getElementById('new-conn-token') as HTMLInputElement | null; + const tokenLabel = document.getElementById('new-conn-token-label'); + const nuvioProfileSelect = document.getElementById('new-nuvio-profile') as HTMLSelectElement | null; + let pendingNuvioRefreshToken = ''; + let previousType = typeSelect?.value ?? 'jellyfin'; + const updateTypeFields = () => { - const isPlex = typeSelect?.value === 'plex'; - userIdRow?.classList.toggle('hidden', isPlex); + const type = typeSelect?.value ?? 'jellyfin'; + const isPlex = type === 'plex'; + const isNuvio = type === 'nuvio'; + userIdRow?.classList.toggle('hidden', isPlex || isNuvio); usernameRow?.classList.toggle('hidden', !isPlex); + nuvioEmailRow?.classList.toggle('hidden', !isNuvio); + nuvioProfileRow?.classList.toggle('hidden', !isNuvio); + if (urlInput) { + if (isNuvio) { + urlInput.value = 'https://api.nuvio.tv'; + urlInput.disabled = true; + } else { + if (previousType === 'nuvio' && urlInput.value === 'https://api.nuvio.tv') urlInput.value = ''; + urlInput.disabled = false; + } + } + if (urlLabel) urlLabel.textContent = isNuvio ? 'Cloud API URL' : 'Server URL'; + if (tokenLabel) tokenLabel.textContent = isNuvio ? 'Nuvio Password' : 'Token'; + if (tokenInput) tokenInput.autocomplete = isNuvio ? 'current-password' : 'off'; + const syncRatings = document.getElementById('new-sync-ratings') as HTMLInputElement | null; + const pushRatings = document.getElementById('new-push-ratings') as HTMLInputElement | null; + const pushPlayback = document.getElementById('new-push-playback') as HTMLInputElement | null; + pushPlayback?.closest('label')?.classList.toggle('hidden', !isNuvio); + if (pushPlayback) pushPlayback.disabled = !isNuvio; + for (const checkbox of [syncRatings, pushRatings]) { + checkbox?.closest('label')?.classList.toggle('hidden', isNuvio); + if (checkbox) checkbox.disabled = isNuvio; + } + const playbackLabel = document.getElementById('new-sync-playback-label'); + if (playbackLabel) playbackLabel.textContent = isNuvio ? 'Playback progress' : 'Playback events'; + const pullHeading = document.getElementById('new-pull-heading'); + const pushHeading = document.getElementById('new-push-heading'); + if (pullHeading) pullHeading.textContent = isNuvio ? 'Nuvio → Scrob' : 'Server → Scrob'; + if (pushHeading) pushHeading.textContent = isNuvio ? 'Scrob → Nuvio' : 'Scrob → Server'; + if (type !== previousType) pendingNuvioRefreshToken = ''; + previousType = type; }; typeSelect?.addEventListener('change', updateTypeFields); updateTypeFields(); + const authenticateNuvio = async () => { + const email = (document.getElementById('new-nuvio-email') as HTMLInputElement | null)?.value.trim() || ''; + const password = tokenInput?.value || ''; + const url = urlInput?.value || 'https://api.nuvio.tv'; + if (!email || !password) throw new Error('Nuvio email and password are required.'); + const selectedProfile = nuvioProfileSelect?.value || ''; + const response = await fetch('/api/proxy/auth/nuvio-login', { + method: 'POST', + headers: { 'Authorization': `Bearer ${token}`, 'Content-Type': 'application/json' }, + body: JSON.stringify({ email, password, url }), + }); + const data = await response.json(); + if (!response.ok) throw new Error(data.detail || 'Nuvio sign-in failed'); + pendingNuvioRefreshToken = data.refresh_token; + if (nuvioProfileSelect) { + nuvioProfileSelect.innerHTML = (data.profiles || []).map((profile: any) => + `` + ).join(''); + if (selectedProfile && [...nuvioProfileSelect.options].some(option => option.value === selectedProfile)) { + nuvioProfileSelect.value = selectedProfile; + } + } + if (!nuvioProfileSelect?.value) throw new Error('No Nuvio profile is available for this account.'); + return data; + }; + // Test new connection button const testNewBtn = document.getElementById('test-new-connection-btn'); if (testNewBtn) { @@ -3094,18 +3487,23 @@ if (me.totp_enabled) { const orig = testNewBtn.textContent; testNewBtn.textContent = '...'; try { - let ep: string; - if (type === 'plex') { - ep = `/api/proxy/auth/test-plex?url=${encodeURIComponent(u)}&token=${encodeURIComponent(t)}`; - } else if (type === 'emby') { - ep = `/api/proxy/auth/test-emby?url=${encodeURIComponent(u)}&token=${encodeURIComponent(t)}&user_id=${encodeURIComponent(ui)}`; + if (type === 'nuvio') { + await authenticateNuvio(); + showMessage('Nuvio connection successful. Select the profile to sync.'); } else { - ep = `/api/proxy/auth/test-jellyfin?url=${encodeURIComponent(u)}&token=${encodeURIComponent(t)}&user_id=${encodeURIComponent(ui)}`; + let ep: string; + if (type === 'plex') { + ep = `/api/proxy/auth/test-plex?url=${encodeURIComponent(u)}&token=${encodeURIComponent(t)}`; + } else if (type === 'emby') { + ep = `/api/proxy/auth/test-emby?url=${encodeURIComponent(u)}&token=${encodeURIComponent(t)}&user_id=${encodeURIComponent(ui)}`; + } else { + ep = `/api/proxy/auth/test-jellyfin?url=${encodeURIComponent(u)}&token=${encodeURIComponent(t)}&user_id=${encodeURIComponent(ui)}`; + } + const res = await fetch(ep, { method: 'POST', headers: { 'Authorization': `Bearer ${token}` } }); + const data = await res.json(); + if (res.ok) showMessage(data.message || 'Connection successful!'); + else throw new Error(data.detail || 'Connection test failed'); } - const res = await fetch(ep, { method: 'POST', headers: { 'Authorization': `Bearer ${token}` } }); - const data = await res.json(); - if (res.ok) showMessage(data.message || 'Connection successful!'); - else throw new Error(data.detail || 'Connection test failed'); } catch (e: any) { showMessage(e.message, 'error'); } finally { @@ -3122,32 +3520,41 @@ if (me.totp_enabled) { const name = (document.getElementById('new-conn-name') as HTMLInputElement)?.value.trim(); const type = typeSelect?.value ?? 'jellyfin'; const u = (document.getElementById('new-conn-url') as HTMLInputElement)?.value.trim(); - const t = (document.getElementById('new-conn-token') as HTMLInputElement)?.value.trim(); - const ui = (document.getElementById('new-conn-user-id') as HTMLInputElement)?.value.trim() || null; - const un = (document.getElementById('new-conn-username') as HTMLInputElement)?.value.trim() || null; - if (!name || !u || !t) { showMessage('Name, URL and Token are required.', 'error'); return; } + let connectionToken = (document.getElementById('new-conn-token') as HTMLInputElement)?.value.trim(); + let userId = (document.getElementById('new-conn-user-id') as HTMLInputElement)?.value.trim() || null; + const username = (document.getElementById('new-conn-username') as HTMLInputElement)?.value.trim() || null; + if (!name || !u || !connectionToken) { + showMessage(type === 'nuvio' ? 'Name, Nuvio email and password are required.' : 'Name, URL and Token are required.', 'error'); + return; + } createBtn.disabled = true; const orig = createBtn.textContent; createBtn.textContent = 'Adding…'; - const body: Record = { - name, type, url: u, token: t, - sync_collection: (document.getElementById('new-sync-collection') as HTMLInputElement)?.checked ?? true, - sync_watched: (document.getElementById('new-sync-watched') as HTMLInputElement)?.checked ?? true, - sync_ratings: (document.getElementById('new-sync-ratings') as HTMLInputElement)?.checked ?? true, - sync_playback: (document.getElementById('new-sync-playback') as HTMLInputElement)?.checked ?? true, - push_watched: (document.getElementById('new-push-watched') as HTMLInputElement)?.checked ?? false, - push_ratings: (document.getElementById('new-push-ratings') as HTMLInputElement)?.checked ?? false, - auto_sync_interval: (() => { - const v = (document.getElementById('new-auto-sync') as HTMLSelectElement)?.value; - return v ? parseInt(v) : null; - })(), - }; - if (type === 'plex') body.server_username = un; - else body.server_user_id = ui; - try { + if (type === 'nuvio') { + await authenticateNuvio(); + connectionToken = pendingNuvioRefreshToken; + userId = nuvioProfileSelect?.value || null; + } + const body: Record = { + name, type, url: u, token: connectionToken, + sync_collection: (document.getElementById('new-sync-collection') as HTMLInputElement)?.checked ?? true, + sync_watched: (document.getElementById('new-sync-watched') as HTMLInputElement)?.checked ?? true, + sync_ratings: type === 'nuvio' ? false : ((document.getElementById('new-sync-ratings') as HTMLInputElement)?.checked ?? true), + sync_playback: (document.getElementById('new-sync-playback') as HTMLInputElement)?.checked ?? true, + push_watched: (document.getElementById('new-push-watched') as HTMLInputElement)?.checked ?? false, + push_playback: type === 'nuvio' && ((document.getElementById('new-push-playback') as HTMLInputElement)?.checked ?? false), + push_ratings: type === 'nuvio' ? false : ((document.getElementById('new-push-ratings') as HTMLInputElement)?.checked ?? false), + auto_sync_interval: (() => { + const value = (document.getElementById('new-auto-sync') as HTMLSelectElement)?.value; + return value ? parseFloat(value) : null; + })(), + }; + if (type === 'plex') body.server_username = username; + else body.server_user_id = userId; + const res = await fetch('/api/proxy/auth/connections', { method: 'POST', headers: { 'Authorization': `Bearer ${token}`, 'Content-Type': 'application/json' }, @@ -3337,14 +3744,17 @@ if (me.totp_enabled) { trakt_client_secret: formData.get('trakt_client_secret') || null, simkl_client_id: formData.get('simkl_client_id') || null, + mdblist_api_key: formData.get('mdblist_api_key') || null, }; - // Collect boolean checkboxes (Trakt + Simkl sync prefs + app preferences) + // Collect boolean checkboxes (cloud sync preferences + app preferences) const syncKeys = [ 'trakt_sync_watched', 'trakt_sync_ratings', 'trakt_sync_lists', 'trakt_watchlist_split', 'trakt_push_watched', 'trakt_push_ratings', 'trakt_push_lists', 'trakt_scrobble', 'simkl_sync_watched', 'simkl_sync_ratings', 'simkl_sync_lists', 'simkl_push_watched', 'simkl_push_ratings', 'simkl_scrobble', + 'mdblist_sync_watched', 'mdblist_sync_ratings', 'mdblist_sync_watchlist', + 'mdblist_push_watched', 'mdblist_push_ratings', 'mdblist_push_watchlist', 'blur_explicit', 'time_format_24h', 'use_hls_player', @@ -3730,7 +4140,7 @@ if (me.totp_enabled) { return `${Math.floor(diff / 86400)}d ago`; } - for (const source of ['jellyfin', 'emby', 'plex'] as const) { + for (const source of ['jellyfin', 'emby', 'plex', 'nuvio'] as const) { const el = document.getElementById(`${source}-last-sync`); if (!el) continue; const job = jobs.find(j => j.source === source && (j.status === 'completed' || j.status === 'failed')); diff --git a/frontend/src/pages/shows.astro b/frontend/src/pages/shows.astro index 9d3cf94..2f78723 100644 --- a/frontend/src/pages/shows.astro +++ b/frontend/src/pages/shows.astro @@ -89,8 +89,8 @@ const baseUrl = `/shows?${baseParams.toString()}`; {!hasTmdbKey ? (
📺
-

TMDB API Key Required

-

To explore shows from The Movie Database, you must add your API key in the settings.

+

TMDB Read Access Token Required

+

To explore shows from The Movie Database, add your TMDB Read Access Token in settings.

Go to Settings