Skip to content

COVERAGE_GUIDE.md

Latest commit

 

History

History
361 lines (268 loc) · 9.72 KB

COVERAGE_GUIDE.md

File metadata and controls

361 lines (268 loc) · 9.72 KB
Raw

Test Coverage Guide

Overview

This project enforces minimum test coverage thresholds to maintain code quality and prevent regression. Coverage is automatically checked on every pull request and push to main/develop branches.

Current Thresholds

All metrics must meet or exceed 60% coverage:

  • Branches: 60%
  • Functions: 60%
  • Lines: 60%
  • Statements: 60%

Running Coverage Locally

Generate Coverage Report

npm run test:coverage

This will:

  • Run all tests with coverage collection
  • Generate multiple report formats (text, HTML, LCOV, JSON)
  • Display coverage summary in terminal
  • Create detailed HTML report in coverage/lcov-report/index.html

View HTML Report

# After running coverage
open coverage/lcov-report/index.html  # macOS
start coverage/lcov-report/index.html # Windows
xdg-open coverage/lcov-report/index.html # Linux

The HTML report shows:

  • File-by-file coverage breakdown
  • Line-by-line coverage visualization
  • Uncovered code highlighted in red
  • Partially covered branches in yellow

CI/CD Integration

Automatic Checks

Coverage is enforced through GitHub Actions:

  1. On Pull Request: Coverage workflow runs automatically
  2. Threshold Check: Jest validates all metrics meet 60% minimum
  3. Build Status:
    • ✅ Pass: Coverage meets thresholds, PR can merge
    • ❌ Fail: Coverage below thresholds, PR blocked

Coverage Workflow

The .github/workflows/ci.yml test job:

  • Runs on every PR and push to main/master
  • Executes npm run test:coverage:ci
  • Runs npm run check-coverage to enforce thresholds
  • Generates a coverage diff comment on every pull request
  • Uploads coverage reports as artifacts (30-day retention)
  • Saves the LCOV report as a base artifact on pushes to main (90-day retention)

Understanding Coverage Metrics

Statements

Percentage of executable statements that were executed during tests.

Branches

Percentage of conditional branches (if/else, switch, ternary) that were tested.

Functions

Percentage of functions that were called during tests.

Lines

Percentage of code lines that were executed during tests.

PR Coverage Diff Reporting

Every pull request automatically receives a sticky comment showing how the PR changes overall and per-file coverage compared with the base branch.

How the LCOV Delta Parser Works

The scripts/check-coverage.js script includes three functions that power the diff report:

  1. parseLcovFile(lcovPath) — Reads an lcov.info file and returns a structured object:

    {
  files: {
    "src/services/StellarService.js": {
      lines:     { found: 120, hit: 90, details: { 12: 3, 13: 0, … } },
      branches:  { found: 40, hit: 25 },
      functions: { found: 15, hit: 12 }
    },
    …
  },
  totals: { lines: {…}, branches: {…}, functions: {…} }
}

    It processes SF:, DA:, BRDA:, BRF:, BRH:, FN:, FNDA:, FNF:, FNH:, LF:, and LH: records.

  2. computeCoverageDelta(baseLcov, headLcov) — Compares two parsed LCOV objects and returns:

    • overall — aggregate line-coverage percentage before/after/delta
    • files — per-file before/after/delta plus a list of uncovered line numbers
    • regressions — files whose coverage decreased
    • improvements — files whose coverage increased

  3. formatCoverageDiffComment(delta) — Renders the delta as a Markdown string containing the sticky comment marker <!-- coverage-diff-comment -->, an overall summary table, a per-file changes table, and an uncovered-lines section for regressed files.

The CI workflow invokes this with the --diff flag:

node scripts/check-coverage.js \
  --diff \
  --base=coverage/base-lcov.info \
  --head=coverage/lcov.info \
  --output=coverage/diff-comment.md

What the Sticky PR Comment Looks Like

<!-- coverage-diff-comment -->

## 📊 Coverage Report

### Overall Coverage

|  | Coverage |
|---|---|
| **Base branch** | 72.45% |
| **This PR**     | 69.10% |
| **Delta**       | 🔴 -3.35% |

### Summary

🔴 **2** file(s) regressed
🟢 **1** file(s) improved

### Files Changed

| File | Base | Head | Delta |
|---|---|---|---|
| `src/routes/donation.js` | 80.00% | 65.00% | 🔴 -15.00% |
| `src/services/StellarService.js` | 60.00% | 55.00% | 🔴 -5.00% |
| `src/utils/database.js` | 50.00% | 58.00% | 🟢 +8.00% |

### Uncovered Lines in Regressed Files

- `src/routes/donation.js`: lines 42, 43, 89, 102, 115
- `src/services/StellarService.js`: lines 200, 201, 202

---
_Coverage diff generated by [check-coverage.js](../scripts/check-coverage.js)_

How to Interpret the Diff Output

Indicator Meaning
🟢 positive delta Coverage increased — new code is well-tested
🔴 negative delta Coverage decreased — some new/changed code is untested
⚪ no change Coverage percentage is unchanged

The Uncovered Lines section lists the exact line numbers that are not executed by any test in the head snapshot. Use these with the HTML report (coverage/lcov-report/index.html) to locate the untested code quickly.

Green/Red Indicators

  • 🟢 improvement: The file's line-coverage percentage is higher in this PR than on the base branch.
  • 🔴 regression: The file's line-coverage percentage is lower in this PR. The CI job still passes as long as overall thresholds (60 %) are met, but the comment highlights the risk so reviewers can decide whether to request additional tests.

Sticky Comment Behaviour

The comment is identified by the hidden marker <!-- coverage-diff-comment -->. On each new push to the PR the CI job searches for an existing comment with that marker and updates it in place, so the PR thread is never cluttered with duplicate coverage comments.

Improving Coverage

1. Identify Uncovered Code

npm run test:coverage
# Look for files with low coverage percentages

2. Add Tests for Uncovered Areas

Focus on:

  • Critical business logic
  • Error handling paths
  • Edge cases
  • Conditional branches

3. Write Effective Tests

// Example: Testing both branches
describe('validateAmount', () => {
  it('should accept valid amounts', () => {
    expect(validateAmount(10)).toBe(true);
  });

  it('should reject negative amounts', () => {
    expect(validateAmount(-5)).toBe(false);
  });
});

4. Verify Improvement

npm run test:coverage
# Check that coverage percentages increased

Coverage Reports

Text Summary (Terminal)

Displayed after running tests, shows overall percentages.

HTML Report

Interactive report with file navigation and line-by-line coverage.

LCOV Report

Machine-readable format for CI/CD integration and coverage tools. The lcov.info file is also used by the PR diff reporter.

JSON Summary

Programmatic access to coverage data (coverage/coverage-summary.json).

Best Practices

Do's

  • ✅ Run coverage before submitting PRs
  • ✅ Test critical business logic thoroughly
  • ✅ Cover error handling paths
  • ✅ Test edge cases and boundary conditions
  • ✅ Aim for meaningful tests, not just coverage numbers

Don'ts

  • ❌ Don't write tests just to hit coverage numbers
  • ❌ Don't ignore failing coverage checks
  • ❌ Don't commit coverage reports to git (they're in .gitignore)
  • ❌ Don't lower thresholds to pass checks

Troubleshooting

Coverage Below Threshold

Problem: CI fails with "Coverage threshold not met"

Solution:

  1. Run npm run test:coverage locally
  2. Check which files have low coverage
  3. Add tests for uncovered code
  4. Verify coverage meets thresholds
  5. Commit and push changes

Coverage Report Not Generated

Problem: No coverage directory created

Solution:

# Clean and reinstall
rm -rf node_modules coverage
npm install
npm run test:coverage

Tests Pass But Coverage Fails

Problem: Tests succeed but coverage check fails

Solution: This is expected behaviour. Coverage thresholds are independent of test results. Add more tests to increase coverage.

PR Comment Not Appearing

Problem: No coverage diff comment on the pull request

Possible causes:

  • The coverage/lcov.info file was not produced (check the test job logs).
  • The workflow does not have issues: write permission — verify the repo Actions settings allow the default GITHUB_TOKEN to post comments.
  • The actions/github-script step failed — check the step output in the Actions run for error details.

Configuration Files

jest.config.js

coverageThreshold: {
  global: {
    branches:   60,
    functions:  60,
    lines:      60,
    statements: 60,
  },
}

package.json

{
  "scripts": {
    "test:coverage":    "jest --coverage",
    "test:coverage:ci": "jest --coverage --ci --maxWorkers=2",
    "check-coverage":   "node scripts/check-coverage.js"
  }
}

Future Improvements

Short Term

  • Increase thresholds to 70% as coverage improves
  • Add per-file coverage requirements for critical modules
  • Generate coverage badges for README

Medium Term

  • Target 70–80% overall coverage
  • Implement coverage diff reporting (show coverage change in PRs) ✅ DONE — sticky PR comment with LCOV delta parser
  • Add coverage trends tracking

Long Term

  • Achieve 80%+ coverage for production code
  • 90%+ coverage for critical business logic
  • Integrate with code review tools

Related Documentation

Support

If you encounter issues with coverage:

  1. Check this guide for troubleshooting steps
  2. Review the CI Pipeline docs
  3. Open an issue with coverage report output