crap/quality score external-analyzer test functions at 0% coverage instead of excluding them

[jflowers]

Summary

When Gaze analyzes a non-Go project via an external analyzer, gaze crap and gaze quality assign CRAP/contract-coverage scores to the project's test functions as if they were production code. Because the analyzer's own coverage output instruments only its src/ package, every test function is scored at 0% line coverage, inflating module-wide CRAP and emitting hundreds of phantom add_tests remediation flags.

Repro (snake-eyes, Python analyzer)

# snake-eyes instruments only src/ via: --cov=snake_eyes
gaze crap --analyzer snake-eyes --language python ./src
gaze quality --analyzer snake-eyes --language python ./src

Observed on the zero-dot-force/snake-eyes repo:

• 993 functions analyzed; 724 are test functions.
• Module-wide CRAPload = 188; 172 of the 188 Q4 (dangerous) functions are test functions.
• Module-wide avg line coverage reads 24.9%, but src/-only coverage is 92.1% (src-only CRAPload = 16).
• Remediation breakdown reports 174 add_tests flags, essentially all test functions.

Root cause

The analyzer protocol's discover method already returns a source/test split (source_files / test_files), and the analyzer's complexity method returns entries for both sets. Gaze merges complexity entries with coverage data that only exists for source files. Test functions have no coverage entry, so they default to 0% coverage and are CRAP-scored as uncovered production code.

Expected behavior

For external analyzers, Gaze should either:

1. Exclude test_files from crap scoring (recommended) — test functions are not production code and have no coverage profile by design; or
2. Treat "no coverage entry" for a file classified as a test file as not-applicable rather than 0% (so complexity-only metrics, if any, don't feed the CRAP coverage penalty).

quality (test-quality) legitimately analyzes test functions, but its contract coverage for tests should also distinguish "test function, no contract expected" from "production function, contract unasserted" — currently both surface as 0% and pollute the over-specification/needs-assertions signals.

Impact

• External-analyzer CRAP/quality reports are dominated by noise from test functions.
• CI gates driven by gaze crap --max-crapload on Python/Rust/TS projects will trip on phantom violations.
• Users must manually filter "test functions with 0% coverage" out of every report.

Notes

• Go mode is unaffected because Go's -coverprofile instruments _test.go files, so Go test functions do carry coverage data. The gap is specific to external analyzers whose coverage scope excludes tests by convention.

[fullsend-ai-triage]

🤖 Finished Triage · ✅ Success · Started 4:53 PM UTC · Completed 4:57 PM UTC

Commit: 6d6be20 · View workflow run →

Runtime: claude · Model: opus → claude-opus-4-6 · Effort: high · Cost: $0.94

[fullsend-ai-triage]

Prerequisite Resolved

The prior triage identified #260 (HTML formatter foundation for gaze analyze) as a blocker. That issue is now closed, and the shared infrastructure is in place (internal/report/html.go, styles.go, analyze.html.tmpl). This issue is unblocked and ready for the spec-first workflow.

Implementation Notes

• Three new HTML templates needed: quality, crap, and combined report
• Reuse the existing styles.go inline CSS and html/template auto-escaping pattern
• Handle optional data gracefully: SSA-degraded packages, skipped BDD tests, baseline comparison (crap), empty result sets
• The crap template should render quadrant distribution, worst offenders with fix strategy labels, and remediation breakdown
• The quality template should render contract coverage, assertion mapping tables, gap hints, and over-specification metrics
• The report template combines pipeline step summaries (CRAP + quality + classify + docscan)

Proposed Test

func TestWriteQualityHTML_PopulatedReport(t *testing.T) {
    // Render a QualityReport with coverage, mappings, gaps.
    // Assert: valid HTML, no external resources, metrics displayed.
}

func TestWriteCrapHTML_EmptyResults(t *testing.T) {
    // Render with zero functions. Assert: no panic, empty-state message.
}

func TestWriteCrapHTML_WithBaselineComparison(t *testing.T) {
    // Render with ComparisonResult. Assert: deltas and violations shown.
}

Related

• Implement HTML report formatter #8 — parent issue (Implement HTML report formatter)
• feat: add self-contained HTML output for gaze analyze #260 — foundation work (closed)

---

Next steps:

• /fs-code — agent creates a PR to implement this issue
• /fs-code <your instruction> — agent implements with your specific guidance