Skip to content

refactor(core): harden observation evidence pipeline - #3196

Draft
quanru wants to merge 1 commit into
feat/observation-report-sequencefrom
fix/observation-evidence-review
Draft

quanru wants to merge 1 commit into
feat/observation-report-sequencefrom
fix/observation-evidence-review

Conversation

@quanru

@quanru quanru commented Sep 28, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

  • make observation evidence an atomic typed model of screenshots and source-frame indices
  • move prompt and evidence orchestration out of the generic insight extraction flow
  • cap visual analysis at 50 evenly sampled source frames and honor cancellation
  • preserve subtle UI evidence with lossless intermediate rendering and readable two-line labels
  • make sequence prompts, documentation, warnings, and dump serialization match the bounded-evidence contract

Review findings addressed

  • corrected prompts that implied every captured frame was sent to the model
  • replaced synchronized optional arrays with atomic evidence entries
  • replaced broad screenshotSequence* serialization filtering with exact transient fields
  • bounded preprocessing work and propagated abort signals
  • removed repeated JPEG encoding before final grid generation
  • added direct verification that the orange measurement outline is present

Validation

  • pnpm --filter @midscene/core exec rstest run — 2,064 passed, 8 skipped
  • focused observation regression suite — 16 passed
  • pnpm --filter @midscene/core run build
  • Biome and git diff --check

Status

Draft/WIP while the prepatch release is being validated.

Prepatch release

@cloudflare-workers-and-pages

Copy link
Copy Markdown

Deploying midscene with  Cloudflare Pages  Cloudflare Pages

Latest commit: 4a87faa
Status: ✅  Deploy successful!
Preview URL: https://fd55b8f7.midscene.pages.dev
Branch Preview URL: https://fix-observation-evidence-rev.midscene.pages.dev

View logs

@quanru

quanru commented Oct 7, 2026

Copy link
Copy Markdown
Collaborator Author

跟进 observation evidence 改造。当前相对 feat/observation-report-sequence 无冲突,附着的 20 项检查通过,仍保留 Draft。PR 描述中有 core 定向测试、完整 core 测试和预发布版本。

合并前还需要确认 base 分支的落地安排,以及预发布在真实 observation 使用场景中的验收结果。请维护者先复审原子 evidence 表示、50 帧上限与取消传播的设计;本轮不将单元测试等同于真实模型效果验证。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant