Chore/dockerize app - #21
Conversation
Ollama unloaded gemma4:e4b-it-qat after 5m idle, and reloading it took ~140-150s — longer than field_mapping's 120s request timeout, so every call after a gap timed out on all retries. Set OLLAMA_KEEP_ALIVE=-1 to keep the model loaded, and raise field_mapping's timeout to 300s to cover the first cold load. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…se profiles Adds a "gpu" compose profile (alongside the default "cpu" one, toggled via COMPOSE_PROFILES in .env) that requests GPU device access for surya-inference, ocr, and ollama. Each pair of variants shares a network alias so downstream services never need to know which is active. - surya-inference: optional CUDA build of llama.cpp (build args), with entrypoint.sh probing nvidia-smi at startup to pick -ngl and falling back to CPU even if GPU access was requested but isn't actually there. - ocr: optional CUDA torch wheel (build arg) instead of the pinned CPU one. - ollama: GPU device reservation only; the official image already auto-detects CUDA and falls back to CPU on its own. - Fixes entrypoint.sh CRLF line endings (broke its shebang when checked out on Windows) and adds .gitattributes to keep shell scripts LF going forward.
|
|
||
| class Settings(BaseSettings): | ||
| model_config = SettingsConfigDict(env_file=".env", env_file_encoding="utf-8") | ||
| model_config = SettingsConfigDict(env_file=".env", env_file_encoding="utf-8", extra="ignore") |
There was a problem hiding this comment.
extra="ignore" lets pydantic-settings skip env vars not declared on Settings (e.g. POSTGRES_USER, POSTGRES_PASSWORD, POSTGRES_DB, POSTGRES_HOST_PORT, COMPOSE_PROFILES — used by docker-compose, not read by the app). Without it, pydantic's default extra="forbid" would raise ValidationError on startup since those keys aren't fields on the model.
Tradeoff: a typo in a key we do care about (e.g. DATABASE_URL) would also be silently ignored rather than erroring.
There was a problem hiding this comment.
Fail fast on Surya backend startup instead of hanging the first upload
Previously, Surya's inference backend (llama-server on CPU / vLLM on CUDA, both running in WSL) was spawned lazily on the first /extract call. If the backend was missing or misconfigured, that first request would just hang until Surya's internal timeout — with no indication anything was wrong.
This PR moves backend startup to service boot:
SuryaEngine._ensure_ready() now calls manager.start() explicitly instead of relying on lazy spawn, and exposes a warm_up() method to trigger this without processing a document.
api.py's FastAPI lifespan kicks off warm_up() as a background task on startup, without blocking the service from coming up.
/health now reports initializing / healthy / unhealthy based on warm-up state, surfacing the underlying error if startup failed.
/extract returns a 503 while OCR isn't ready, instead of accepting the upload and blocking on Surya's timeout.
Testing: verified /health transitions initializing → healthy on a working backend, and returns unhealthy with the underlying error message when the WSL backend is unavailable; /extract returns 503 during warm-up instead of hanging.
| finally: | ||
| # Clean up temporary file | ||
| if temp_file and os.path.exists(temp_file_path): | ||
| if temp_file_path and os.path.exists(temp_file_path): |
There was a problem hiding this comment.
Fix UnboundLocalError masking exceptions during temp file cleanup
temp_file_path was only assigned inside the with **tempfile.NamedTemporaryFile(...)** block, after .write(content). If an exception occurred before that line (e.g. during file creation or write), the finally block's if temp_file and os.path.exists(temp_file_path) would short-circuit past the truthy temp_file object and try to evaluate os.path.exists(temp_file_path) on an unset variable — raising UnboundLocalError inside finally itself, which masked the original error and skipped cleanup (leaking the temp file).
Fix: initialize temp_file_path = "" before the try, and guard on temp_file_path directly instead of temp_file (which was never the right variable to check — it doesn't tell you whether a valid path exists to clean up).
torch was already pinned to the CPU-only index, but torchvision was left to resolve from default PyPI via surya-ocr's dependency, pulling in a CUDA-linked build. Mismatched torch/torchvision builds can break torchvision's compiled ops at runtime. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Consolidates setup instructions (env config, migrations, Ollama model pull, service ports, and GPU-profile usage) that were previously scattered across Database_setup.md and the OCR README, into one guide for a fresh clone. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Surfaces the Ollama model config used by translation and field_mapping, with a note on pulling the model into the container before first use. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
There was a problem hiding this comment.
🟡 Changes recommended
There are a few concrete correctness/operational issues in the new warm-up lifecycle handling and example env defaults that can break shutdown behavior and out-of-the-box configuration.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review details
Files not reviewed (1)
- lending-poc/frontend/package-lock.json: Generated file
- Files reviewed: 31/33 changed files
- Comments generated: 5
- Review effort level: Lite
| async def warm_up() -> None: | ||
| try: | ||
| await run_in_threadpool(extractor.engine.warm_up) | ||
| except Exception as exc: |
There was a problem hiding this comment.
Didn't feel necessary. Ignored for now.
| @@ -0,0 +1,3 @@ | |||
| DATABASE_URL=postgresql+asyncpg://postgres:postgres@localhost:55432/lending_poc | |||
There was a problem hiding this comment.
This file is dead. Needs to be deleted. Done it in the latest PR
| VITE_API_BASE_URL=http://localhost:8000 | ||
| VITE_TRANSLATION_API_BASE_URL=http://localhost:8000 |
| set -e | ||
|
|
||
| MODEL_PATH="${MODEL_DIR}/${SURYA_GGUF_MODEL_FILE}" | ||
| MMPROJ_PATH="${MODEL_DIR}/${SURYA_GGUF_MMPROJ_FILE}" |
There was a problem hiding this comment.
Copilot is right in the narrow sense, though the default path is already covered: Dockerfile:49 creates /models, and MODEL_DIR defaults to /models.
So the failure would only occur if someone overrides MODEL_DIR to a path that hasn't been created or mounted. Bind mounts and named volumes are auto-created by Docker.
Still worth taking — it's a single idempotent line, and it makes the script self-contained rather than relying on the Dockerfile to prepare the directory.
Done in the latest PR.
cancel() + await keeps the asyncio side tidy by ensuring there is no unfinished task left when the event loop closes. It also makes shutdown deterministic from the async side and keeps the implementation consistent with stop_monitoring(). Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Dockerize the lending-poc application stack
Summary
Adds a complete docker-compose setup so the whole lending-poc stack — API, gateway, OCR, Surya inference, translation, field mapping, frontend, and Ollama — can be brought up with a single
docker compose up, instead of each service needing its own manually-managed venv/process. GPU acceleration (Surya OCR, Ollama) is opt-in via a compose profile rather than required, so the default path works on any machine. Bundled in are a few small fixes discovered while getting the containerized stack to actually run end-to-end.What's new
New Dockerfiles for every service that didn't have one: gateway, document_processing/ocr, document_processing/translation, field_mapping_poc, frontend, and surya-inference (new standalone service, see below). Each gets a matching
.dockerignoreso build contexts don't drag in.venv,node_modules,.git,.env, or per-service scratch/output directories.surya-inference service — a new standalone container that builds llama-server (llama.cpp) from source and serves Surya's OCR model over HTTP, so the ocr service can point
SURYA_INFERENCE_URLat it instead of spawning its own in-process copy. CPU by default; a CUDA build is available via build args for the GPU profile.CPU/GPU opt-in via compose profiles — ollama, surya-inference, and ocr each get a -gpu sibling service (ollama-gpu, surya-inference-gpu, ocr-gpu) gated behind
COMPOSE_PROFILES=gpuin.env(default is cpu). Both variants of a pair share a network alias so downstream services (OLLAMA_HOST,SURYA_INFERENCE_URL,OCR_BASE_URL) never need to know which one is active. surya-inference's entrypoint.sh additionally probes nvidia-smi at container start and falls back to CPU (-ngl 0) even if GPU access was requested but isn't actually available, so a misconfigured toolkit degrades gracefully instead of failing the container.Why: keeps
docker compose upusable on any dev machine out of the box, while still letting GPU-equipped hosts opt in for meaningfully faster OCR/LLM inference.Wiring for the rest of the stack — field_mapping, translation, gateway, and frontend are added as compose services with their internal ports, env vars, and depends_on chains (gateway waits on ocr/translation/field_mapping; those wait on ollama/surya where relevant). New
ollama_modelsandsurya_modelsnamed volumes persist downloaded models across restarts.Model choice: docker-compose.yml defaults
OLLAMA_MODELtogemma4:e4b-it-qatfor both field_mapping and translation — this is the model tested against in this branch, including the keep-alive/timeout fix below. To use the unquantizedgemma4:e4binstead, change theOLLAMA_MODELdefault in docker-compose.yml for those two services (or override it via.env); note that a larger model will also change cold-load time and may need theOLLAMA_TIMEOUT_SECONDS/ keep-alive settings revisited.Onboarding
.env.examplefiles added at the repo root,field_mapping_poc/, andfrontend/, documenting the env vars each part of the stack expects (DB connection,COMPOSE_PROFILES, Vite API base URLs, etc.).document_processing/ocr/README.md— new section documenting how to run the service via compose, how to switch CPU/GPU profiles, and the GPU prerequisites (NVIDIA Container Toolkit, WSL2 GPU passthrough on Windows).Fixes bundled into this branch
Ollama model eviction causing /map timeouts (d293b13): Ollama was unloading
gemma4:e4b-it-qatafter its default 5-minute idle timeout, and reloading it took ~140–150s — longer than field_mapping's request timeout, so every call after an idle gap failed on all retries. Fixed by settingOLLAMA_KEEP_ALIVE=-1(never unload) and raising field_mapping'sOLLAMA_TIMEOUT_SECONDSto 300s to cover the first cold load.CRLF line endings breaking entrypoint.sh (baa515e): the script's shebang broke when checked out on Windows. Added
.gitattributes(*.sh text eol=lf) to force LF endings for shell scripts going forward.app/config.py: added
extra="ignore"to the Pydantic Settings config so the shared.envfile (now containing compose-wide vars likeCOMPOSE_PROFILES,POSTGRES_HOST_PORT) doesn't fail app startup validation.Database_setup.md: corrected the documented Postgres host port (55432 → 55439) to match what docker-compose.yml actually maps — the doc had drifted out of sync with the compose file.
How to test
To test the GPU path, set
COMPOSE_PROFILES=gpuin.envon a host with the NVIDIA Container Toolkit installed.# optional: use the unquantized model instead of the default gemma4:e4b-it-qat OLLAMA_MODEL=gemma4:e4b docker compose up --build