fix(2119): stop reporting empty --model formal-ai runs as successes

.changeset/formal-ai-empty-run-false-positives.md

@@ -0,0 +1,5 @@
+---
+'@link-assistant/hive-mind': patch
+---
+
+Stop reporting an empty `--model formal-ai` run as a success. Formal AI sessions are now attributed to Link.Assistant at $0.00, token usage is parsed for all six tools, the two duplicate auto-restart loops are one N/M budget that fails visibly when it is exhausted, and a pull request whose net diff is empty (or holds only the solver's own placeholder) is neither described as changed nor announced as ready to merge. Docker images now pin Formal AI 0.317.0 so the upstream workspace-effect and self-healing fixes are distributed with Hive Mind.

.prettierignore

@@ -36,6 +36,7 @@ docs/case-studies/issue-1881/data/
 docs/case-studies/issue-2080/data/
 docs/case-studies/issue-2084/data/
 docs/case-studies/issue-2085/data/
+docs/case-studies/issue-2119/data/
 docs/case-studies/issue-2109/source/*.json
 docs/case-studies/issue-2109/research/*.json
 docs/case-studies/issue-2117/data/

Dockerfile

@@ -18,7 +18,7 @@
 #
 # Build: docker build -t konard/hive-mind .
 
-ARG FORMAL_AI_VERSION=0.305.0
+ARG FORMAL_AI_VERSION=0.317.0
 # Bookworm's glibc 2.36 remains compatible with the Ubuntu 24.04 Box runtime.
 FROM rust:1.96-slim-bookworm AS formal-ai-builder
 ARG FORMAL_AI_VERSION

Dockerfile.dind

@@ -21,7 +21,7 @@
 # We pin 2.3.5 (latest patch on top of that fix).
 # Latest Box releases: https://github.com/link-foundation/box/releases
 
-ARG FORMAL_AI_VERSION=0.305.0
+ARG FORMAL_AI_VERSION=0.317.0
 # Bookworm's glibc 2.36 remains compatible with the Ubuntu 24.04 Box runtime.
 FROM rust:1.96-slim-bookworm AS formal-ai-builder
 ARG FORMAL_AI_VERSION

Dockerfile.formal-ai

@@ -4,7 +4,7 @@
 # isolated solve jobs have the same agentic CLIs. The pinned builder keeps this
 # file buildable before a new Hive Mind tag reaches every registry mirror.
 
-ARG FORMAL_AI_VERSION=0.305.0
+ARG FORMAL_AI_VERSION=0.317.0
 ARG HIVE_MIND_VERSION=latest
 # Bookworm's glibc 2.36 remains compatible with the Ubuntu 24.04 Box runtime.
 FROM rust:1.96-slim-bookworm AS formal-ai-builder

coolify/Dockerfile

@@ -18,7 +18,7 @@
 #
 # Build: docker build -f coolify/Dockerfile -t konard/hive-mind .
 
-ARG FORMAL_AI_VERSION=0.305.0
+ARG FORMAL_AI_VERSION=0.317.0
 # Bookworm's glibc 2.36 remains compatible with the Ubuntu 24.04 Box runtime.
 FROM rust:1.96-slim-bookworm AS formal-ai-builder
 ARG FORMAL_AI_VERSION

docs/case-studies/issue-2119/data/issue-2119-comments.json

@@ -0,0 +1 @@
+[]

docs/case-studies/issue-2119/data/issue-2119.json

@@ -0,0 +1,11 @@
+{
+  "author": { "id": "MDQ6VXNlcjE0MzE5MDQ=", "is_bot": false, "login": "konard", "name": "Konstantin Diachenko" },
+  "body": "Provider of the model is `Link.Assistant`, not `OpenCode Zen` or `Anthropic`.\n\nThe cost of the model tokens is 0$ (free).\n\nCarefully check all the GitHub comments and logs, and fix all other false positives, false negatives, errors and warnings.\n\nAuto-restart/resume on uncommitted changes and any other reason must stop after 5-th iteration, otherwise it will stuck infinitely, that must be applied to any model, as not only `formal-ai` may fail that way. Also as `Auto-restart triggered (iteration 1)` and `Auto-restart 1/5 Log` are different, that means we have some critical error in duplication of code. We should only use single auto-restart/resume system where it provides N/M as in `Auto-restart 1/5 Log`, so it is clear what is the limit. And after 5 we must actually stop (fail + auto-commit on fail recovery).\n\nSo the result will be actually visible.\n\nWe must solve all issues related to Hive Mind itself in this pull request, and report all issues related to Formal AI itself to github.com/link-assistant/formal-ai, and as always we should ask to solve it by generalization, not specialization, improving self learning and self healing meta algorithm. As Formal AI must be capable of coding and computer tasks.\n\nPull requests tested with `--model formal-ai`:\n- https://github.com/konard/test-hello-world-019fb330-00e1-73b9-955e-f357a1600d5b/pull/2\n- https://github.com/konard/test-hello-world-019fb331-c107-78c7-8ff6-9f127a3c593c/pull/2\n- https://github.com/konard/test-hello-world-019fb330-fa49-7c9d-a664-b7ea33bb698a/pull/2\n\nAlso find all other previous `konard/test-hello-world-*` examples, and we should make sure we have tests in Formal AI, that guarantee highest possible quality of implementation. So based on these examples and tests we will be able to build quality product out of Formal AI. Each good example now is the strongest foundation for the future development.\n\nEach pull request targeted different `--tool`: codex, claude, agent (our own https://github.com/link-assistant/agent if any issues with it also report).\n\nWe should also ensure uniform support for gemini, and qwen for the Formal AI and Hive Mind.\n\nWe need to collect all the logs, and use them to report issue to Formal AI (it should fit single big issue with all listed requirements to be fixed) with all the materials linked.\n\nWe need to download all logs and data related about the issue to this repository, make sure we compile that data to `./docs/case-studies/issue-{id}` folder, and use it to do deep case study analysis (also make sure to search online for additional facts and data), in which we will reconstruct timeline/sequence of events, list of each and all requirements from the issue, find root causes of the each problem, and propose possible solutions and solution plans for each requirement (we should also check known existing components/libraries, that solve similar problem or can help in solutions).\n\nIf there is not enough data to find actual root cause, add debug output and verbose mode if not present, that will allow us to find root cause on next iteration.\n\nIf issue related to any other repository/project, where we can report issues on GitHub, please do so. Each issue must contain reproducible examples, workarounds and suggestions for fix the issue in code. Also double check to fully apply requirements to entire codebase, so if we have issue in multiple places, it should be fixed in all them.\n\nPlease plan and execute everything in this single pull request, you have unlimited time and context, as context auto-compacts and you can continue indefinitely, until it is each and every requirement fully addressed, and everything is totally done.",
+  "createdAt": "2026-07-30T14:29:29Z",
+  "labels": [{ "id": "LA_kwDOPUU0qc8AAAACGYm6iw", "name": "bug", "description": "Something isn't working", "color": "d73a4a" }],
+  "number": 2119,
+  "state": "OPEN",
+  "title": "`--model formal-ai` not working on simplest hello world",
+  "updatedAt": "2026-07-30T14:44:56Z",
+  "url": "https://github.com/link-assistant/hive-mind/issues/2119"
+}

docs/case-studies/issue-2119/data/log-gists.json

@@ -0,0 +1,17 @@
+{
+  "019fb330-00e1-73b9-955e-f357a1600d5b": [
+    { "createdAt": "2026-07-30T14:14:38Z", "gist": "https://gist.githubusercontent.com/konard/465f5511052c5796806c6fc1d29c6b4c/raw/c2768221a87d91fc27cbcad2dc9bc9e45b2a6ecd/tmp-hive-mind-log-upload-TQuSSp-sanitized.log.txt", "title": "🤖 Solution Draft Log" },
+    { "createdAt": "2026-07-30T14:15:24Z", "gist": "https://gist.githubusercontent.com/konard/ef3290d4bc10e5d1116c6e1d79cfe9fc/raw/f4798e8bf7b0121f0e89ba82ceb233c85d55e950/tmp-hive-mind-log-upload-YU3TQL-sanitized.log.txt", "title": "🔄 Auto-restart 1/5 Log" },
+    { "createdAt": "2026-07-30T14:16:07Z", "gist": "https://gist.githubusercontent.com/konard/4f45c03c2bc13fabed47d8fea4a5194f/raw/08207cecd10edcfc34ff5b940a522831d92fcba1/tmp-hive-mind-log-upload-9twZ9s-sanitized.log.txt", "title": "🔄 Auto-restart 2/5 Log" },
+    { "createdAt": "2026-07-30T14:16:50Z", "gist": "https://gist.githubusercontent.com/konard/ebda1420a27cdaac0327cf3bfa515261/raw/26108136c6e9fc928e66a55fe06e6367f0cf4fb0/tmp-hive-mind-log-upload-6bihBU-sanitized.log.txt", "title": "🔄 Auto-restart 3/5 Log" },
+    { "createdAt": "2026-07-30T14:17:47Z", "gist": "https://gist.githubusercontent.com/konard/f6a134fb73adac137e7c200ab5d3db92/raw/cee22282afc4bcbdeaba88475f26a8806cd5ece2/tmp-hive-mind-log-upload-56SSDs-sanitized.log.txt", "title": "🔄 Auto-restart 4/5 Log" },
+    { "createdAt": "2026-07-30T14:18:35Z", "gist": "https://gist.githubusercontent.com/konard/9cd2e3a7e98a634375d2eed7049fb62d/raw/570dd2d36c791629e9a68001ac94e35d36841357/tmp-hive-mind-log-upload-tWAAPV-sanitized.log.txt", "title": "🔄 Auto-restart 5/5 Log" },
+    { "createdAt": "2026-07-30T14:21:33Z", "gist": "https://gist.githubusercontent.com/konard/385002afaadc89bd178b783864d88301/raw/1174c597b8947cbfe4ab938e802664507db3d994/tmp-hive-mind-log-upload-HRzjRr-sanitized.log.txt", "title": "🔄 Auto-restart-until-mergeable Log (iteration 1)" },
+    { "createdAt": "2026-07-30T14:24:25Z", "gist": "https://gist.githubusercontent.com/konard/037503c4af448467dd8fdc3eb3c48980/raw/3815a653380f2b28b482948233fccdc18f3e7403/tmp-hive-mind-log-upload-fEg1HP-sanitized.log.txt", "title": "🔄 Auto-restart-until-mergeable Log (iteration 2)" },
+    { "createdAt": "2026-07-30T14:27:17Z", "gist": "https://gist.githubusercontent.com/konard/1459e4acca0d459d305f28a6e9b215be/raw/00ea9281d3b3004012d03a99a151b4786203a546/tmp-hive-mind-log-upload-HB4zCM-sanitized.log.txt", "title": "🔄 Auto-restart-until-mergeable Log (iteration 3)" },
+    { "createdAt": "2026-07-30T14:30:11Z", "gist": "https://gist.githubusercontent.com/konard/48c447dc630ae2f8246bbe62ac835e71/raw/4f45d3bd037e82f2201771ee7d955b1b2e71af84/tmp-hive-mind-log-upload-ZzMtDW-sanitized.log.txt", "title": "🔄 Auto-restart-until-mergeable Log (iteration 4)" },
+    { "createdAt": "2026-07-30T14:33:02Z", "gist": "https://gist.githubusercontent.com/konard/a5f3e9b4dea22cdffe0c699fcf32e1c5/raw/0de8fc98d9e00bf957d61ce5e3ed627a5c069c1c/tmp-hive-mind-log-upload-0JuEzF-sanitized.log.txt", "title": "🔄 Auto-restart-until-mergeable Log (iteration 5)" }
+  ],
+  "019fb330-fa49-7c9d-a664-b7ea33bb698a": [{ "createdAt": "2026-07-30T14:20:18Z", "gist": "https://gist.githubusercontent.com/konard/16a9e51ac177a0ca8dd6f26b5148b1b4/raw/b2fb3eb4ede4871979e5443f643551b725141b90/tmp-hive-mind-log-upload-DAh7ws-sanitized.log.txt", "title": "🤖 Solution Draft Log" }],
+  "019fb331-c107-78c7-8ff6-9f127a3c593c": [{ "createdAt": "2026-07-30T14:25:59Z", "gist": "https://gist.githubusercontent.com/konard/4e56198bee2a177e71ddc41bdf5b5294/raw/9ef0b5a8748c953eaeb2c98b9eafe6603359c855/tmp-hive-mind-log-upload-WQPa4n-sanitized.log.txt", "title": "🚨 Solution Draft Failed" }]
+}

docs/case-studies/issue-2119/data/prs/019fb330-00e1-73b9-955e-f357a1600d5b/issue-1-comments.json

@@ -0,0 +1 @@
+[]

docs/case-studies/issue-2119/data/prs/019fb330-00e1-73b9-955e-f357a1600d5b/issue-1.json

@@ -0,0 +1 @@
+{"url":"https://api.github.com/repos/konard/test-hello-world-019fb330-00e1-73b9-955e-f357a1600d5b/issues/1","repository_url":"https://api.github.com/repos/konard/test-hello-world-019fb330-00e1-73b9-955e-f357a1600d5b","labels_url":"https://api.github.com/repos/konard/test-hello-world-019fb330-00e1-73b9-955e-f357a1600d5b/issues/1/labels{/name}","comments_url":"https://api.github.com/repos/konard/test-hello-world-019fb330-00e1-73b9-955e-f357a1600d5b/issues/1/comments","events_url":"https://api.github.com/repos/konard/test-hello-world-019fb330-00e1-73b9-955e-f357a1600d5b/issues/1/events","html_url":"https://github.com/konard/test-hello-world-019fb330-00e1-73b9-955e-f357a1600d5b/issues/1","id":5020167374,"node_id":"I_kwDOToRLvM8AAAABKzmszg","number":1,"title":"Implement Hello World in Scala","user":{"login":"konard","id":1431904,"node_id":"MDQ6VXNlcjE0MzE5MDQ=","avatar_url":"https://avatars.githubusercontent.com/u/1431904?v=4","gravatar_id":"","url":"https://api.github.com/users/konard","html_url":"https://github.com/konard","followers_url":"https://api.github.com/users/konard/followers","following_url":"https://api.github.com/users/konard/following{/other_user}","gists_url":"https://api.github.com/users/konard/gists{/gist_id}","starred_url":"https://api.github.com/users/konard/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/konard/subscriptions","organizations_url":"https://api.github.com/users/konard/orgs","repos_url":"https://api.github.com/users/konard/repos","events_url":"https://api.github.com/users/konard/events{/privacy}","received_events_url":"https://api.github.com/users/konard/received_events","type":"User","user_view_type":"public","site_admin":false},"labels":[],"state":"open","locked":false,"assignees":[],"milestone":null,"comments":0,"created_at":"2026-07-30T13:21:47Z","updated_at":"2026-07-30T13:21:47Z","closed_at":null,"assignee":null,"author_association":"OWNER","active_lock_reason":null,"sub_issues_summary":{"total":0,"completed":0,"percent_completed":0},"issue_dependencies_summary":{"blocked_by":0,"total_blocked_by":0,"blocking":0,"total_blocking":0},"body":"## Task\nPlease implement a \"Hello World\" program in Scala.\n\n## Requirements\n1. Create a file with the appropriate extension for Scala\n2. The program should print exactly: `Hello, World!`\n3. Add clear comments explaining the code\n4. Ensure the code follows Scala best practices and idioms\n5. If applicable, include build/run instructions in a comment at the top of the file\n6. **Create a GitHub Actions workflow that automatically runs and tests the program on every push and pull request**\n\n## Expected Output\nWhen the program runs, it should output:\n```\nHello, World!\n```\n\n## GitHub Actions Requirements\nThe CI/CD workflow should:\n- Trigger on push to main branch and on pull requests\n- Set up the appropriate Scala runtime/compiler\n- Run the Hello World program\n- Verify the output is exactly \"Hello, World!\"\n- Show a green check mark when tests pass\n\nExample workflow structure:\n- Checkout code\n- Setup Scala environment\n- Run the program\n- Assert output matches expected string\n\n## Additional Notes\n- The implementation should be simple and straightforward\n- Focus on clarity and correctness\n- Use the standard library only (no external dependencies unless absolutely necessary for Scala)\n- The GitHub Actions workflow should be in `.github/workflows/` directory\n- The workflow should have a meaningful name like `test-hello-world.yml`\n\n## Definition of Done\n- [ ] Program file created with correct extension\n- [ ] Code prints \"Hello, World!\" exactly\n- [ ] Code is properly commented\n- [ ] Code follows Scala conventions\n- [ ] Instructions for running the program are included (if needed)\n- [ ] GitHub Actions workflow created and passing\n- [ ] CI badge showing build status (optional but recommended)","closed_by":null,"reactions":{"url":"https://api.github.com/repos/konard/test-hello-world-019fb330-00e1-73b9-955e-f357a1600d5b/issues/1/reactions","total_count":0,"+1":0,"-1":0,"laugh":0,"hooray":0,"confused":0,"heart":0,"rocket":0,"eyes":0},"timeline_url":"https://api.github.com/repos/konard/test-hello-world-019fb330-00e1-73b9-955e-f357a1600d5b/issues/1/timeline","performed_via_github_app":null,"state_reason":null,"pinned_comment":null}

docs/case-studies/issue-2119/data/prs/019fb330-00e1-73b9-955e-f357a1600d5b/pr-2-review-comments.json

@@ -0,0 +1 @@
+[]

docs/case-studies/issue-2119/data/prs/019fb330-fa49-7c9d-a664-b7ea33bb698a/issue-1-comments.json

@@ -0,0 +1 @@
+[]

docs/case-studies/issue-2119/data/prs/019fb330-fa49-7c9d-a664-b7ea33bb698a/issue-1.json

@@ -0,0 +1 @@
+{"url":"https://api.github.com/repos/konard/test-hello-world-019fb330-fa49-7c9d-a664-b7ea33bb698a/issues/1","repository_url":"https://api.github.com/repos/konard/test-hello-world-019fb330-fa49-7c9d-a664-b7ea33bb698a","labels_url":"https://api.github.com/repos/konard/test-hello-world-019fb330-fa49-7c9d-a664-b7ea33bb698a/issues/1/labels{/name}","comments_url":"https://api.github.com/repos/konard/test-hello-world-019fb330-fa49-7c9d-a664-b7ea33bb698a/issues/1/comments","events_url":"https://api.github.com/repos/konard/test-hello-world-019fb330-fa49-7c9d-a664-b7ea33bb698a/issues/1/events","html_url":"https://github.com/konard/test-hello-world-019fb330-fa49-7c9d-a664-b7ea33bb698a/issues/1","id":5020176114,"node_id":"I_kwDOToRPqc8AAAABKznO8g","number":1,"title":"Implement Hello World in Kotlin","user":{"login":"konard","id":1431904,"node_id":"MDQ6VXNlcjE0MzE5MDQ=","avatar_url":"https://avatars.githubusercontent.com/u/1431904?v=4","gravatar_id":"","url":"https://api.github.com/users/konard","html_url":"https://github.com/konard","followers_url":"https://api.github.com/users/konard/followers","following_url":"https://api.github.com/users/konard/following{/other_user}","gists_url":"https://api.github.com/users/konard/gists{/gist_id}","starred_url":"https://api.github.com/users/konard/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/konard/subscriptions","organizations_url":"https://api.github.com/users/konard/orgs","repos_url":"https://api.github.com/users/konard/repos","events_url":"https://api.github.com/users/konard/events{/privacy}","received_events_url":"https://api.github.com/users/konard/received_events","type":"User","user_view_type":"public","site_admin":false},"labels":[],"state":"open","locked":false,"assignees":[],"milestone":null,"comments":0,"created_at":"2026-07-30T13:22:50Z","updated_at":"2026-07-30T13:22:50Z","closed_at":null,"assignee":null,"author_association":"OWNER","active_lock_reason":null,"sub_issues_summary":{"total":0,"completed":0,"percent_completed":0},"issue_dependencies_summary":{"blocked_by":0,"total_blocked_by":0,"blocking":0,"total_blocking":0},"body":"## Task\nPlease implement a \"Hello World\" program in Kotlin.\n\n## Requirements\n1. Create a file with the appropriate extension for Kotlin\n2. The program should print exactly: `Hello, World!`\n3. Add clear comments explaining the code\n4. Ensure the code follows Kotlin best practices and idioms\n5. If applicable, include build/run instructions in a comment at the top of the file\n6. **Create a GitHub Actions workflow that automatically runs and tests the program on every push and pull request**\n\n## Expected Output\nWhen the program runs, it should output:\n```\nHello, World!\n```\n\n## GitHub Actions Requirements\nThe CI/CD workflow should:\n- Trigger on push to main branch and on pull requests\n- Set up the appropriate Kotlin runtime/compiler\n- Run the Hello World program\n- Verify the output is exactly \"Hello, World!\"\n- Show a green check mark when tests pass\n\nExample workflow structure:\n- Checkout code\n- Setup Kotlin environment\n- Run the program\n- Assert output matches expected string\n\n## Additional Notes\n- The implementation should be simple and straightforward\n- Focus on clarity and correctness\n- Use the standard library only (no external dependencies unless absolutely necessary for Kotlin)\n- The GitHub Actions workflow should be in `.github/workflows/` directory\n- The workflow should have a meaningful name like `test-hello-world.yml`\n\n## Definition of Done\n- [ ] Program file created with correct extension\n- [ ] Code prints \"Hello, World!\" exactly\n- [ ] Code is properly commented\n- [ ] Code follows Kotlin conventions\n- [ ] Instructions for running the program are included (if needed)\n- [ ] GitHub Actions workflow created and passing\n- [ ] CI badge showing build status (optional but recommended)","closed_by":null,"reactions":{"url":"https://api.github.com/repos/konard/test-hello-world-019fb330-fa49-7c9d-a664-b7ea33bb698a/issues/1/reactions","total_count":0,"+1":0,"-1":0,"laugh":0,"hooray":0,"confused":0,"heart":0,"rocket":0,"eyes":0},"timeline_url":"https://api.github.com/repos/konard/test-hello-world-019fb330-fa49-7c9d-a664-b7ea33bb698a/issues/1/timeline","performed_via_github_app":null,"state_reason":null,"pinned_comment":null}