feat: refresh buttons, bubble controls, inner boundary + per-bubble studio actions

docs/feature-plan-next.md

@@ -0,0 +1,173 @@
+# Next Feature Plan
+
+Four features to implement after PR #6 merges. Work on `master`.
+
+---
+
+## 1. Refresh button — Jobs and Library pages
+
+**Files:** `server/ClientApp/src/pages/JobsListPage.tsx`, `server/ClientApp/src/pages/LibraryPage.tsx`
+
+Both pages already expose `refetch` / `refetchVolumes` / `refetchChapters` from `createResource`.
+
+**Jobs page:** Add a `RefreshCw` (lucide-solid) button in the header row, right-aligned next to the existing icon. Call `refetch()` on click. Spin the icon while `jobs.loading` is true.
+
+**Library page:** Add a Refresh button next to "New Volume" / "New Chapter" in the header. Call both `refetchVolumes()` and `refetchChapters()`. Disable while either resource is loading.
+
+No server changes needed.
+
+---
+
+## 2. Delete button per item — Jobs list page ✅ DONE (merged in PR #6)
+
+**Already implemented.** `JobCard` uses `<div class="group relative ...">` outer wrapper with two sibling native `<button>` elements — delete (absolute top-left) and nav (full-width). No nested interactive elements. `ConfirmDialog` and `deleteError` signal fully wired.
+
+---
+
+## 3. Bubble inner-boundary detection — server-side (SkiaSharp) + frontend padding
+
+### Context
+
+The server already uses SkiaSharp (not OpenCV). The model is RT-DETR outputting bounding boxes only (no segmentation masks). Gemini's Scenario B (contour extraction from the cropped bbox) applies, but translated to SkiaSharp.
+
+### Server-side: inner boundary refinement in `BubbleDetectionService.cs`
+
+After detecting a `BubbleBox`, call a new helper `RefineToInnerBoundary` that crops the bbox from the original bitmap, finds the actual white inner area using SkiaSharp pixel access, and returns a tighter `BubbleBox`.
+
+**Algorithm (pure SkiaSharp, no new dependencies):**
+
+```csharp
+private static BubbleBox RefineToInnerBoundary(SKBitmap src, BubbleBox box, byte brightnessThreshold = 220)
+{
+    int x0 = (int)box.X, y0 = (int)box.Y;
+    int w  = (int)box.Width, h = (int)box.Height;
+
+    // Clamp to image bounds
+    x0 = Math.Max(0, x0); y0 = Math.Max(0, y0);
+    w  = Math.Min(w, src.Width  - x0);
+    h  = Math.Min(h, src.Height - y0);
+    if (w <= 0 || h <= 0) return box;
+
+    // BFS flood-fill from center to find connected bright region
+    int cx = x0 + w / 2, cy = y0 + h / 2;
+    var visited = new bool[w, h];
+    var queue   = new Queue<(int, int)>();
+
+    bool IsBright(int px, int py) {
+        var c = src.GetPixel(px, py);
+        return c.Red > brightnessThreshold && c.Green > brightnessThreshold && c.Blue > brightnessThreshold;
+    }
+
+    if (!IsBright(cx, cy)) return box; // center not white → no refinement
+
+    queue.Enqueue((cx - x0, cy - y0));
+    visited[cx - x0, cy - y0] = true;
+    int minX = cx, maxX = cx, minY = cy, maxY = cy;
+
+    while (queue.Count > 0) {
+        var (lx, ly) = queue.Dequeue();
+        int gx = lx + x0, gy = ly + y0;
+        if (gx < minX) minX = gx; if (gx > maxX) maxX = gx;
+        if (gy < minY) minY = gy; if (gy > maxY) maxY = gy;
+        foreach (var (dx, dy) in new[]{(1,0),(-1,0),(0,1),(0,-1)}) {
+            int nx = lx+dx, ny = ly+dy;
+            if (nx < 0 || ny < 0 || nx >= w || ny >= h) continue;
+            if (visited[nx,ny]) continue;
+            visited[nx,ny] = true;
+            if (IsBright(nx+x0, ny+y0)) queue.Enqueue((nx,ny));
+        }
+    }
+
+    // Apply a small fixed inset (3 px) to avoid edge contamination
+    const int EdgeInset = 3;
+    minX += EdgeInset; minY += EdgeInset;
+    maxX -= EdgeInset; maxY -= EdgeInset;
+    if (maxX <= minX || maxY <= minY) return box;
+
+    return new BubbleBox(minX, minY, maxX - minX, maxY - minY, box.Confidence);
+}
+```
+
+Call it in `DetectRtDetr` and `DetectYolo` just before adding to `result`:
+```csharp
+var refined = RefineToInnerBoundary(bitmap, new BubbleBox(x1, y1, bw, bh, score));
+result.Add(refined);
+```
+
+**Trade-off:** BFS on the cropped region is O(w×h) per bubble — for a typical 200×200 crop this is ~40K pixel checks, negligible compared to ONNX inference time. `GetPixelSpan` can replace `GetPixel` for better cache performance if needed.
+
+### Frontend: configurable display padding prop (fine-tuning only)
+
+Even with server-side refinement, a small visual inset prop on `BubbleCanvas` is useful for tweaking:
+
+**`BubbleCanvas.tsx`** — add prop:
+```ts
+/** Additional pixel inset on all sides for display only. Does not affect stored coords. */
+bubblePadding?: number;
+```
+Apply in the rect/handle rendering helper. Default `0`.
+
+**StudioPage** — expose as a number input (0–20 px), persisted to `localStorage`.
+
+---
+
+## 4. Push corrected job result back to the extension
+
+This requires three layers: server SSE → `background.ts` → `content.ts`.
+
+### Server — new SSE endpoint
+
+Add to `PortalRoutes.cs`:
+```http
+GET /api/portal/jobs/{id}/events
+Content-Type: text/event-stream
+```
+
+Keeps the connection open. Implementation: poll `AppDbContext` every ~2 s in a loop with `HttpContext.RequestAborted` as cancellation token. When `ResultImagePath` is populated or `Status` changes to `"done"`, emit:
+```text
+event: job-updated
+data: {"id":"...","status":"done","result_available":true}
+```
+
+Close stream after emitting the done event (one-shot).
+
+### Server — `/ocr` opt-in job tracking
+
+Add an optional `track_job` boolean to `OcrRequest`. When `true`, save the submitted image as a `PageTranslationJob` and return `job_id` in the `OcrResponse`. The existing fast path (no `track_job`) is unchanged.
+
+### Extension — `types.ts` additions
+
+```ts
+// New message type
+export interface JobResultReadyMsg {
+  type: "job-result-ready";
+  jobId: string;
+  resultImageDataUrl: string;
+}
+export type ToContentMsg = ... | JobResultReadyMsg;
+```
+
+### Extension — `background.ts`
+
+After `runServerFlow` completes and response includes `job_id`:
+1. Open `EventSource` to `{serverUrl}/api/portal/jobs/{jobId}/events`
+2. On `job-updated` event with `result_available: true`:
+   - Fetch `{serverUrl}/api/portal/jobs/{jobId}/result` as blob
+   - Convert blob to base64 data URL
+   - `sendToTab(tabId, { type: "job-result-ready", jobId, resultImageDataUrl })`
+   - Close the `EventSource`
+
+Store active `EventSource` instances in a `Map<string, EventSource>` keyed by `tabId` so they can be cleaned up on tab close.
+
+### Extension — `content.ts`
+
+Handle `"job-result-ready"` message:
+- If the result panel is still open, replace or add the corrected image alongside the original crop
+- OR show a small "Corrected image ready" banner with a click-to-view action
+
+### Implementation order
+
+1. `#1` Refresh buttons — ✅ DONE (committed to master)
+2. `#2` Delete button — ✅ DONE (merged in PR #6)
+3. `#3` Bubble inner-boundary refinement — ~2 h (server-side SkiaSharp BFS + optional frontend padding prop)
+4. `#4` Push corrected jobs — ~3 h (server SSE + `/ocr` opt-in tracking + extension `background.ts`/`content.ts` wiring)

extension/package.json

@@ -1,6 +1,6 @@
 {
   "name": "selfhost-ocr-extension",
-  "version": "1.3.1",
+  "version": "1.3.4",
   "private": true,
   "type": "module",
   "scripts": {

extension/src/background.ts

@@ -7,6 +7,8 @@ import type {
   JishoEntry,
   PopupModeMsg,
   FetchImageMsg,
+  JobResultReadyMsg,
+  JobResultErrorMsg,
 } from "./types";
 import { DEFAULT_SETTINGS } from "./types";
 
@@ -168,6 +170,7 @@ async function runServerFlow(
       body: JSON.stringify({
         image: `data:image/jpeg;base64,${imageB64}`,
         translate_engine: settings.serverTranslation,
+        track_job: true,
       }),
     });
 
@@ -177,19 +180,141 @@ async function runServerFlow(
       return;
     }
 
-    const data = await res.json() as { text: string; translation?: string; elapsed_ms: number };
+    const data = await res.json() as { text: string; translation?: string; elapsed_ms: number; job_id?: string };
 
     sendToTab(tabId, {
       type: "ocr-result",
       text: data.text,
       translation: data.translation ?? null,
       elapsed_ms: Date.now() - start,
     } satisfies ToContentMsg);
+
+    // If the server started a background page-translation job, schedule alarm-based polling
+    if (data.job_id) {
+      void schedulePollJob(data.job_id, settings.serverUrl, tabId);
+    }
   } catch (e) {
     sendToTab(tabId, { type: "ocr-error", message: errMsg(e) });
   }
 }
 
+// ── Job result polling via chrome.alarms (MV3-safe) ──────────────────────────
+
+interface PendingJob {
+  jobId: string;
+  serverUrl: string;
+  tabId: number;
+  deadline: number; // epoch ms — abandon after 3 min
+  errorCount: number;
+}
+
+const ALARM_PREFIX = "poll:";
+const POLL_INTERVAL_SECONDS = 3;
+const POLL_DEADLINE_MS = 3 * 60 * 1000;
+const MAX_CONSECUTIVE_ERRORS = 3;
+
+function alarmName(jobId: string): string {
+  return `${ALARM_PREFIX}${jobId}`;
+}
+
+async function schedulePollJob(jobId: string, serverUrl: string, tabId: number): Promise<void> {
+  const job: PendingJob = {
+    jobId,
+    serverUrl,
+    tabId,
+    deadline: Date.now() + POLL_DEADLINE_MS,
+    errorCount: 0,
+  };
+  await chrome.storage.session.set({ [alarmName(jobId)]: job });
+  chrome.alarms.create(alarmName(jobId), { delayInMinutes: POLL_INTERVAL_SECONDS / 60 });
+}
+
+async function handlePollAlarm(name: string): Promise<void> {
+  const key = name;
+  const stored = await chrome.storage.session.get(key);
+  const job = stored[key] as PendingJob | undefined;
+  if (!job) return; // already cleared
+
+  if (Date.now() > job.deadline) {
+    sendToTab(job.tabId, {
+      type: "job-result-error",
+      jobId: job.jobId,
+      reason: "timeout",
+    } satisfies JobResultErrorMsg);
+    await chrome.storage.session.remove(key);
+    return;
+  }
+
+  const base = job.serverUrl.replace(/\/$/, "");
+  try {
+    const res = await fetch(`${base}/jobs/${job.jobId}/status`);
+    if (!res.ok) throw new Error(`HTTP ${res.status}`);
+
+    const data = await res.json() as { status: string };
+
+    if (data.status === "done") {
+      const imgRes = await fetch(`${base}/jobs/${job.jobId}/result-image`);
+      if (imgRes.ok) {
+        const blob    = await imgRes.blob();
+        const dataUrl = await blobToDataUrl(blob);
+        sendToTab(job.tabId, {
+          type: "job-result-ready",
+          jobId: job.jobId,
+          resultImageDataUrl: dataUrl,
+        } satisfies JobResultReadyMsg);
+      }
+      await chrome.storage.session.remove(key);
+      return;
+    }
+
+    if (data.status === "error") {
+      sendToTab(job.tabId, {
+        type: "job-result-error",
+        jobId: job.jobId,
+        reason: "server-error",
+      } satisfies JobResultErrorMsg);
+      await chrome.storage.session.remove(key);
+      return;
+    }
+
+    // Still pending — reschedule and reset error count
+    const updated: PendingJob = { ...job, errorCount: 0 };
+    await chrome.storage.session.set({ [key]: updated });
+    chrome.alarms.create(name, { delayInMinutes: POLL_INTERVAL_SECONDS / 60 });
+  } catch {
+    const newCount = job.errorCount + 1;
+    if (newCount >= MAX_CONSECUTIVE_ERRORS) {
+      sendToTab(job.tabId, {
+        type: "job-result-error",
+        jobId: job.jobId,
+        reason: "network-error",
+      } satisfies JobResultErrorMsg);
+      await chrome.storage.session.remove(key);
+      return;
+    }
+    const updated: PendingJob = { ...job, errorCount: newCount };
+    await chrome.storage.session.set({ [key]: updated });
+    chrome.alarms.create(name, { delayInMinutes: POLL_INTERVAL_SECONDS / 60 });
+  }
+}
+
+chrome.alarms.onAlarm.addListener((alarm) => {
+  if (alarm.name.startsWith(ALARM_PREFIX)) {
+    void handlePollAlarm(alarm.name);
+  }
+});
+
+async function blobToDataUrl(blob: Blob): Promise<string> {
+  const buf   = await blob.arrayBuffer();
+  const bytes = new Uint8Array(buf);
+  let binary  = "";
+  const chunk = 8192;
+  for (let i = 0; i < bytes.byteLength; i += chunk) {
+    binary += String.fromCharCode(...bytes.subarray(i, i + chunk));
+  }
+  return `data:${blob.type || "image/png"};base64,${btoa(binary)}`;
+}
+
 // ── Explain flow (server-only) ────────────────────────────────────────────────
 
 async function handleExplain(text: string, tabId: number): Promise<void> {

extension/src/content.ts

@@ -7,6 +7,7 @@ import type {
   TokenInfo,
   JishoEntry,
   OcrResultMsg,
+  JobResultReadyMsg,
   ToEngineMsg,
   FromEngineMsg,
   FetchImageMsg,
@@ -53,6 +54,7 @@ function init(): void {
     else if (msg.type === "ocr-error")        showError(msg.message);
     else if (msg.type === "explain-result")   showExplain(msg.tokens, msg.definitions, msg.mode);
     else if (msg.type === "explain-error")    showExplainError(msg.message);
+    else if (msg.type === "job-result-ready") appendJobImage(msg.resultImageDataUrl);
   });
 
   // Engine iframe messages (postMessage from engine.html)
@@ -306,6 +308,20 @@ function showResult(msg: OcrResultMsg): void {
   wirePanelButtons(resultPanelEl);
 }
 
+function appendJobImage(resultImageDataUrl: string): void {
+  if (!resultPanelEl) return;
+  const inner = resultPanelEl.querySelector<HTMLElement>(".socr-panel-inner");
+  if (!inner) return;
+  inner.querySelector(".socr-job-image")?.remove();
+  const section = document.createElement("div");
+  section.className = "socr-job-image";
+  section.innerHTML = `
+    <div class="socr-text-label">Translated Page</div>
+    <img class="socr-result-image" src="${escAttr(resultImageDataUrl)}" alt="Translated page" />
+  `;
+  inner.appendChild(section);
+}
+
 function showError(message: string): void {
   const rect = selectionRect;
   removeResultPanel();

extension/src/types.ts

@@ -78,6 +78,8 @@ export interface ExplainResultMsg   { type: "explain-result"; tokens: TokenInfo[
 export interface ExplainErrorMsg    { type: "explain-error"; message: string }
 
 export interface StartImageModeMsg  { type: "start-image-mode" }
+export interface JobResultReadyMsg  { type: "job-result-ready"; jobId: string; resultImageDataUrl: string }
+export interface JobResultErrorMsg  { type: "job-result-error"; jobId: string; reason: "timeout" | "server-error" | "network-error" }
 
 export type ToContentMsg =
   | StartSelectionMsg
@@ -86,7 +88,9 @@ export type ToContentMsg =
   | OcrErrorMsg
   | ExplainResultMsg
   | ExplainErrorMsg
-  | StartImageModeMsg;
+  | StartImageModeMsg
+  | JobResultReadyMsg
+  | JobResultErrorMsg;
 
 // ── Messages: content → background ───────────────────────────────────────────