Skip to content

test(registry): extend structural validation to full registry#671

Open
rainhotel wants to merge 2 commits into
harness:mainfrom
rainhotel:codex/test-full-registry-invariants
Open

test(registry): extend structural validation to full registry#671
rainhotel wants to merge 2 commits into
harness:mainfrom
rainhotel:codex/test-full-registry-invariants

Conversation

@rainhotel

@rainhotel rainhotel commented Jul 21, 2026

Copy link
Copy Markdown

Description

Extends universal structural invariants to the full registry, including opt-in toolsets, while keeping default exposure rules scoped to the default registry.

Current main registers both pipeline and pipeline_v1 simultaneously; HARNESS_PIPELINE_VERSION only controls the default preference. The tests therefore use one complete Registry rather than duplicating identical V0/V1 variants.

Closes #119.

Type of Change

  • Bug fix
  • New feature
  • Refactor
  • Documentation
  • Other — structural validation tests

Changes

  • run universal path, identifier, scope, HTTP, extractor, metadata, and operation-policy invariants against the full registry
  • derive opt-in resources from ToolsetDefinition.optIn
  • verify defaults, opt-in resources, pipeline, and pipeline_v1 are covered
  • preserve explicit default-only exposure assertions
  • inject a malformed opt-in fixture through RegistryOptions.additionalToolsets to exercise the same Registry and validator path
  • remove redundant opt-in-only structural checks

Validation

  • structural validation: 35/35
  • related registry tests: 208/208
  • pnpm standards:check: 77/77
  • pnpm typecheck: passed
  • pnpm build: passed
  • full suite with --testTimeout=30000: 2542 passed, 8 skipped
  • falsifiability check: temporarily invalidating an opt-in-only resource made the expanded invariant fail, and restoring it returned the suite to green

The default pnpm test command still hits the existing 5-second timeout in tests/search/manager.test.ts for the local semantic-search provider. The same failure was reproduced on a clean origin/main worktree, so it is not introduced by this change.

Checklist

  • pnpm test passes — blocked locally by the clean-main 5-second search timeout described above
  • pnpm typecheck passes
  • pnpm build passes
  • pnpm standards:check passes
  • pnpm docs:check not required — registry/tool counts are unchanged

Coding Standards

This PR changes tests only and does not add or modify Harness API coverage, tool registrations, endpoint policies, extractors, or resource definitions.

Copilot AI review requested due to automatic review settings July 21, 2026 08:03

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Extends the registry structural-validation test suite so that universal invariants (paths/params, scope, HTTP methods, extractors, operationPolicy, write-metadata, descriptions) are evaluated against the full Registry (defaults + opt-in toolsets), while keeping default-exposure semantics assertions scoped to the default Registry.

Changes:

  • Switch structural invariants to iterate over a single “full registry” instance built via HARNESS_TOOLSETS additive selection (covering opt-in toolsets and both pipeline/pipeline_v1).
  • Replace prior opt-in-only structural checks with full-registry invariants + targeted regression coverage (including an injected invalid opt-in toolset via RegistryOptions.additionalToolsets).
  • Add task tracking notes for Issue #119 work and lessons learned.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File Description
tests/registry/structural-validation.test.ts Runs universal structural invariants against the full registry; adds regression + exposure-semantics assertions.
tasks/todo.md Records Issue #119 execution plan and verification notes.
tasks/lessons.md Captures a lesson about revalidating historical helpers against current runtime behavior.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread tests/registry/structural-validation.test.ts Outdated
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
@CLAassistant

CLAassistant commented Jul 22, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@rainhotel

Copy link
Copy Markdown
Author

Hi @thisrohangupta, just checking in — is there anything you'd like me to adjust on this PR, or does it look good to
you as-is? Happy to make changes if needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Structural validation tests: extend invariants to full registry

4 participants