feat(cloud-sync): add Nuvio and MDBList providers

README.md

@@ -13,7 +13,7 @@
 
 ---
 
-Scrob syncs your libraries from **Jellyfin**, **Plex**, and **Emby**, tracks your watch history, ratings, and personal lists, and lets you push your activity back to your media server - all from a clean, app-like web interface that installs as a PWA on any device.
+Scrob syncs your libraries from **Jellyfin**, **Plex**, **Emby**, and **Nuvio**, tracks your watch history, ratings, and personal lists, and can push watched activity back to connected providers - all from a clean, app-like web interface that installs as a PWA on any device.
 
 ## Table of Contents
 
@@ -26,7 +26,11 @@ Scrob syncs your libraries from **Jellyfin**, **Plex**, and **Emby**, tracks you
   - [First Setup](#first-setup)
   - [Updating](#updating)
 - [Configuration](#configuration)
-- [Development](#development)
+- [Nuvio Cloud Synchronization](#nuvio-cloud-synchronization)
+  - [Connect Nuvio](#connect-nuvio)
+  - [Synchronization Directions](#synchronization-directions)
+  - [Scheduling and Limitations](#scheduling-and-limitations)
+- [MDBList Synchronization](#mdblist-synchronization)
 - [Webhooks](#webhooks-real-time-scrobbling)
   - [Jellyfin](#jellyfin)
   - [Plex](#plex)
@@ -36,16 +40,18 @@ Scrob syncs your libraries from **Jellyfin**, **Plex**, and **Emby**, tracks you
 - [Email Validation & SMTP](#email-validation--smtp)
 - [Contributing](#contributing)
 - [Contributors](#contributors)
+- [Development](#development)
 - [License](#license)
 
 ## Features
 
-- **Multi-source sync**: Import your full library, watch history, and ratings from Jellyfin, Plex, and Emby. Incremental syncs keep everything up to date.
-- **Keep all servers in sync**: Keep your watched status in sync between all your servers. Supports multiple instances.
+- **Multi-source sync**: Import libraries and watch history from Jellyfin, Plex, Emby, and Nuvio. Nuvio also imports playback progress for Continue Watching.
+- **Keep providers in sync**: Keep watched status synchronized between your media servers and Nuvio. Supports multiple instances and Nuvio profiles.
 - **Real-time scrobbling**: Webhooks from Jellyfin, Plex, Emby, and Kodi update your watch state as you play - no manual sync needed.
 - **Manual scrobble**: Start a watching session directly from any movie or episode page. Pause, resume, stop, or mark as watched - session progress shows live on the home screen.
 - **Trakt integration**: Sync your watched history and ratings from Trakt, and push Scrob activity back to Trakt automatically.
 - **Simkl integration**: Sync your watched history and ratings from Simkl, and push Scrob activity back to Simkl automatically.
+- **MDBList integration**: Pull watched history, ratings, and watchlist items from MDBList, and optionally push Scrob changes back using an MDBList API key.
 - **Watch history & ratings**: Track every movie and episode you've watched, including multiple plays with individual timestamps. Log plays manually with a custom date, or remove individual entries — all from the watched button on any movie or episode page. Rate them on a 10-point scale with optional reviews.
 - **Season ratings**: Rate individual seasons separately from the overall show.
 - **Personal lists**: Create and curate lists of movies and shows. Mark them public to share with other users on the same instance.
@@ -111,7 +117,7 @@ Scrob syncs your libraries from **Jellyfin**, **Plex**, and **Emby**, tracks you
 ### Prerequisites
 
 - [Docker](https://docs.docker.com/get-docker/) and [Docker Compose](https://docs.docker.com/compose/install/)
-- A [TMDB API key](https://www.themoviedb.org/settings/api) (free) - used for metadata, search, and images
+- A [TMDB Read Access Token](https://www.themoviedb.org/settings/api) (free) - used for metadata, search, and images
 
 ### Docker Compose
 
@@ -249,8 +255,10 @@ docker run -d \
 ### First Setup
 
 1. Open `http://localhost:7330` and create your account.
-2. Go to **Settings → Integrations** to add your TMDB API key and connect Jellyfin, Plex, or Emby.
-3. Select which libraries to sync, then trigger your first sync from **Settings → Sync**.
+2. Go to **Settings → Integrations** to add your TMDB Read Access Token and connect Jellyfin, Plex, Emby, or Nuvio.
+3. Select which media-server libraries to sync, or select a Nuvio profile, then trigger your first sync.
+
+For Nuvio, choose **Nuvio** as the provider, sign in, and select one of the returned profiles. See [Nuvio Cloud Synchronization](#nuvio-cloud-synchronization) for credential handling, sync directions, scheduling, and current limitations.
 
 ### Updating
 
@@ -296,6 +304,50 @@ Remove the `scrob-db` service and set `DATABASE_URL` to your existing instance:
 DATABASE_URL: postgresql+asyncpg://user:password@your-db-host:5432/scrob
 ```
 
+## Nuvio Cloud Synchronization
+
+Scrob connects to the [Nuvio public Cloud API](https://nuvio.tv/docs) at `https://api.nuvio.tv`. A TMDB Read Access Token must be configured in Scrob so Nuvio content identifiers can be matched to movies and shows.
+
+### Connect Nuvio
+
+1. Open **Settings → Media & Cloud Connections** and select **Add Connection**.
+2. Choose **Nuvio**, then enter a connection name, your Nuvio email, and your Nuvio password.
+3. Select **Test** to authenticate and load the profiles attached to the account.
+4. Select the Nuvio profile to synchronize, choose the pull and push options, then select **Add**.
+
+Scrob exchanges the email and password for a refresh token. The password is never persisted. Refresh-token rotation is handled automatically during connection checks and synchronization.
+
+Each connection targets one Nuvio profile. Add another connection if you need to synchronize another profile from the same account.
+
+### Synchronization Directions
+
+| Direction | Setting | Behavior |
+|---|---|---|
+| Nuvio → Scrob | **Collection status** | Imports the profile's library movies and series. |
+| Nuvio → Scrob | **Watched status** | Imports watched movies and episodes with their latest watch timestamps. |
+| Nuvio → Scrob | **Playback progress** | Imports position and duration into Continue Watching. |
+| Scrob → Nuvio | **Watched status** | Pushes watched and unwatched changes made in Scrob or imported from another connected provider. |
+| Scrob → Nuvio | **Playback progress** | Pushes current playback positions into Nuvio's Continue Watching state as non-destructive upserts. |
+
+**Sync now** runs an inbound synchronization using the enabled Nuvio → Scrob settings. **Push** sends the enabled watched-history and playback-progress data from Scrob to Nuvio. Both operations use non-destructive upserts; items absent from Scrob are not removed from Nuvio.
+
+Library membership is currently pull-only. Ratings are not synchronized with Nuvio.
+
+### Scheduling and Limitations
+
+**Auto Pull** repeats the enabled inbound synchronization every 15 minutes, 30 minutes, 1 hour, 3 hours, 6 hours, 12 hours, 24 hours, or 48 hours. Nuvio synchronization is polling-based; Nuvio does not use the media-server webhook URLs documented below.
+
+Inbound Nuvio identifiers are normalized to TMDB for Scrob's internal matching. Before an outbound push, Scrob resolves those TMDB identifiers to Nuvio-compatible bare IMDb identifiers (`tt...`) and caches the mapping. Unsupported identifiers are skipped rather than attached to the wrong title.
+
+## MDBList Synchronization
+
+1. Open **Settings → Connections → MDBList**.
+2. Copy the API key from [MDBList Preferences](https://mdblist.com/preferences), paste it into Scrob, and select **Save Changes**.
+3. Choose the data to import under **MDBList → Scrob**, then select **Pull**. MDBList pulls run only when this button is selected.
+4. To send changes back, enable the required **Scrob → MDBList** options. Watched-state and rating edits are pushed as they happen; edits to the managed **MDBList - Watchlist** are pushed to the MDBList watchlist.
+
+The manual **Push** action sends the complete enabled watched, ratings, or managed-watchlist snapshot. MDBList pagination follows `next_cursor` and requests the documented maximum of 1,000 items per page.
+
 ## Webhooks (Real-time Scrobbling)
 
 Webhooks update your watch history and Continue Watching in real time. Each user's webhook URL is shown in **Settings** next to the relevant integration.

backend/core/enrichment.py

@@ -36,6 +36,7 @@ async def enrich_media(media: Media, api_key: str = None, series_tmdb_id: int =
             media.tmdb_data = {
                 "runtime": data.get("runtime"),
                 "genres": [g["name"] for g in data.get("genres", [])],
+                "external_ids": data.get("external_ids", {}),
                 "cast": [
                     {"name": c["name"], "character": c["character"], "profile_path": tmdb.poster_url(c.get("profile_path"), size="w185")}
                     for c in data.get("credits", {}).get("cast", [])[:10]

backend/core/image_cache.py

@@ -24,6 +24,9 @@
 _last_prune_at: datetime = datetime.min.replace(tzinfo=timezone.utc)
 _PRUNE_INTERVAL = timedelta(minutes=5)
 
+# asyncpg hard limit is 32767 parameters per query; stay well under it
+_MAX_IN_PARAMS = 30_000
+
 
 def parse_tmdb_url(url: str) -> tuple[Optional[str], Optional[str]]:
     """Parse TMDB URL or path into (size, path) tuple."""
@@ -120,8 +123,9 @@ async def _prune_type(db, image_type: str, total_size: int, limit_bytes: int) ->
         total_size -= row.file_size
         ids_to_delete.append(row.id)
 
-    if ids_to_delete:
-        await db.execute(sa_delete(ImageCache).where(ImageCache.id.in_(ids_to_delete)))
+    for i in range(0, len(ids_to_delete), _MAX_IN_PARAMS):
+        chunk = ids_to_delete[i : i + _MAX_IN_PARAMS]
+        await db.execute(sa_delete(ImageCache).where(ImageCache.id.in_(chunk)))
 
     return total_size
 

backend/core/mdblist.py

@@ -0,0 +1,167 @@
+"""Async client for MDBList's user synchronization API."""
+
+from __future__ import annotations
+
+from collections.abc import Iterable
+from typing import Any
+
+import httpx
+
+MDBLIST_BASE = "https://api.mdblist.com"
+PAGE_SIZE = 1000
+PUSH_BATCH_SIZE = 500
+
+
+class MDBListAPIError(RuntimeError):
+    """Raised when MDBList rejects or cannot complete a request."""
+
+
+async def _request(
+    method: str,
+    path: str,
+    api_key: str,
+    *,
+    params: dict[str, Any] | None = None,
+    payload: dict[str, Any] | None = None,
+) -> dict[str, Any]:
+    query = dict(params or {})
+    query["apikey"] = api_key
+    try:
+        async with httpx.AsyncClient(timeout=60.0) as client:
+            response = await client.request(
+                method,
+                f"{MDBLIST_BASE}{path}",
+                params=query,
+                json=payload,
+            )
+        response.raise_for_status()
+    except httpx.HTTPStatusError as exc:
+        detail = exc.response.text.strip()[:500]
+        suffix = f": {detail}" if detail else ""
+        raise MDBListAPIError(
+            f"MDBList {method} {path} failed ({exc.response.status_code}){suffix}"
+        ) from exc
+    except httpx.HTTPError as exc:
+        raise MDBListAPIError(f"MDBList {method} {path} failed: {exc}") from exc
+
+    if response.status_code == 204 or not response.content:
+        return {}
+    data = response.json()
+    if not isinstance(data, dict):
+        raise MDBListAPIError(f"MDBList {method} {path} returned an invalid response")
+    return data
+
+
+async def validate_api_key(api_key: str) -> bool:
+    if not api_key:
+        return False
+    try:
+        await _request("GET", "/sync/last_activities", api_key)
+        return True
+    except MDBListAPIError:
+        return False
+
+
+async def _get_all(api_key: str, path: str) -> dict[str, Any]:
+    merged: dict[str, Any] = {
+        "movies": [],
+        "shows": [],
+        "seasons": [],
+        "episodes": [],
+    }
+    cursor: str | None = None
+    total_seen = 0
+    seen_cursors: set[str] = set()
+
+    while True:
+        params: dict[str, Any] = {"limit": PAGE_SIZE}
+        if cursor:
+            params["cursor"] = cursor
+        elif total_seen:
+            params["offset"] = total_seen
+
+        page = await _request("GET", path, api_key, params=params)
+        page_count = 0
+        for key in ("movies", "shows", "seasons", "episodes"):
+            values = page.get(key)
+            if isinstance(values, list):
+                merged[key].extend(values)
+                page_count += len(values)
+        total_seen += page_count
+
+        pagination = page.get("pagination")
+        pagination = pagination if isinstance(pagination, dict) else {}
+        next_cursor = pagination.get("next_cursor")
+        if next_cursor:
+            next_cursor = str(next_cursor)
+            if next_cursor in seen_cursors:
+                raise MDBListAPIError(f"MDBList {path} returned a repeated pagination cursor")
+            seen_cursors.add(next_cursor)
+            cursor = next_cursor
+            continue
+
+        if pagination.get("has_more"):
+            if page_count == 0:
+                raise MDBListAPIError(f"MDBList {path} reported more pages without returning items")
+            cursor = None
+            continue
+        break
+
+    return merged
+
+
+async def get_watched(api_key: str) -> dict[str, Any]:
+    return await _get_all(api_key, "/sync/watched")
+
+
+async def get_ratings(api_key: str) -> dict[str, Any]:
+    return await _get_all(api_key, "/sync/ratings")
+
+
+async def get_watchlist(api_key: str) -> dict[str, Any]:
+    return await _get_all(api_key, "/watchlist/items")
+
+
+def _batched_payloads(payload: dict[str, list[dict[str, Any]]]) -> Iterable[dict[str, list[dict[str, Any]]]]:
+    for key in ("movies", "shows", "seasons", "episodes"):
+        values = payload.get(key, [])
+        for offset in range(0, len(values), PUSH_BATCH_SIZE):
+            yield {key: values[offset : offset + PUSH_BATCH_SIZE]}
+
+
+async def _push(path: str, api_key: str, payload: dict[str, list[dict[str, Any]]]) -> dict[str, int]:
+    stats = {"submitted": 0, "batches": 0, "not_found": 0}
+    for batch in _batched_payloads(payload):
+        stats["batches"] += 1
+        result = await _request("POST", path, api_key, payload=batch)
+        stats["submitted"] += sum(len(values) for values in batch.values())
+        not_found = result.get("not_found")
+        if isinstance(not_found, dict):
+            stats["not_found"] += sum(
+                len(values) for values in not_found.values() if isinstance(values, list)
+            )
+    return stats
+
+
+async def push_watched(api_key: str, payload: dict[str, list[dict[str, Any]]]) -> dict[str, int]:
+    return await _push("/sync/watched", api_key, payload)
+
+
+async def remove_watched(api_key: str, payload: dict[str, list[dict[str, Any]]]) -> dict[str, int]:
+    return await _push("/sync/watched/remove", api_key, payload)
+
+
+async def push_ratings(api_key: str, payload: dict[str, list[dict[str, Any]]]) -> dict[str, int]:
+    return await _push("/sync/ratings", api_key, payload)
+
+
+async def remove_ratings(api_key: str, payload: dict[str, list[dict[str, Any]]]) -> dict[str, int]:
+    return await _push("/sync/ratings/remove", api_key, payload)
+
+
+async def push_watchlist(api_key: str, payload: dict[str, list[dict[str, Any]]]) -> dict[str, int]:
+    return await _push("/watchlist/items/add", api_key, payload)
+
+
+async def remove_watchlist(api_key: str, payload: dict[str, list[dict[str, Any]]]) -> dict[str, int]:
+    return await _push("/watchlist/items/remove", api_key, payload)