Skip to content

problem: no notifications for new targets (JSON, Pulsar)#26

Open
splix wants to merge 1 commit into
masterfrom
feature/notification-format
Open

problem: no notifications for new targets (JSON, Pulsar)#26
splix wants to merge 1 commit into
masterfrom
feature/notification-format

Conversation

@splix

@splix splix commented Jun 10, 2026

Copy link
Copy Markdown
Member

solution: extend the Notification struct with new types

solution: extend the Notification struct with new types
Copilot AI review requested due to automatic review settings June 10, 2026 17:21

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Extends the archive notification payload to support non-file targets (JSON-per-field layouts and Pulsar streaming) by making location a typed object and emitting notifications based on writer-reported output locations.

Changes:

  • Replaced Notification.location: String with Notification.location: Location (tagged enum) and bumped schema version to /v2; added NotificationBuilder for constructing notifications from per-range locations.
  • Added TargetFileWriter::locations() and implemented it across storage backends so per-height targets (JSON, Pulsar) can report one location per height.
  • Updated the archiver pipeline to collect and publish multiple notifications per processing step (one per reported location), with tests/docs updates.

Reviewed changes

Copilot reviewed 16 out of 16 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
src/storage/pulsar.rs Accumulates broker receipts per height and reports Location::Pulsar via locations().
src/storage/objects.rs Tracks the session Range in the writer and reports Location::File via locations().
src/storage/mod.rs Adds TargetFileWriter::locations() contract and imports Location.
src/storage/json_objects.rs Tracks per-row produced JSON objects and reports per-height Location::Files.
src/storage/json_fs.rs Tracks per-row produced JSON files and reports per-height Location::Files; adds tests for per-height notification behavior.
src/storage/fs.rs Tracks the session Range in the writer and reports Location::File via locations().
src/notify/pulsar.rs Updates notifier test to use the new typed Location.
src/notify/mod.rs Introduces Location-typed notification schema (v2) + NotificationBuilder.
src/notify/location.rs New Location enum and supporting structs (FileGroup, MessageRef, etc.) with serde tagging and tests.
src/formats/json.rs Adds FileSlot tagging to JSON field outputs so writers can build FileGroup without filename parsing.
src/archiver/table.rs Switches tx/trace processing to generate multiple notifications from writer locations().
src/archiver/order.rs Updates test writers to satisfy the new locations() requirement.
src/archiver/mod.rs Updates ProcessOutcome to carry Vec<Notification> instead of Option<Notification>.
src/archiver/block.rs Switches block processing to generate multiple notifications from writer locations().
src/archiver/archiver.rs Publishes flattened notification streams (multiple per step) and updates helper extraction logic.
README.adoc Documents notification v2 schema and the new location shapes (file, files, pulsar).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/storage/json_fs.rs
@coveralls

Copy link
Copy Markdown

Coverage Report for CI Build 27293415880

Coverage increased (+0.6%) to 49.306%

Details

  • Coverage increased (+0.6%) from the base build.
  • Patch coverage: 49 uncovered changes across 6 files (60 of 109 lines covered, 55.05%).
  • 20 coverage regressions across 9 files.

Uncovered Changes

File Changed Covered %
src/archiver/archiver.rs 29 8 27.59%
src/archiver/table.rs 18 4 22.22%
src/archiver/block.rs 9 4 44.44%
src/storage/json_objects.rs 10 6 60.0%
src/notify/location.rs 16 13 81.25%
src/storage/pulsar.rs 11 9 81.82%
Total (9 files) 109 60 55.05%

Coverage Regressions

20 previously-covered lines in 9 files lost coverage.

File Lines Losing Coverage Coverage
src/formats/avro/mod.rs 6 79.22%
src/archiver/archiver.rs 5 28.69%
src/command/verify.rs 2 45.43%
src/metrics/mod.rs 2 10.53%
src/archiver/datakind.rs 1 80.43%
src/archiver/filenames.rs 1 78.57%
src/archiver/table.rs 1 27.89%
src/formats/json.rs 1 68.52%
src/notify/mod.rs 1 75.0%

Coverage Stats

Coverage Status
Relevant Lines: 2667
Covered Lines: 1315
Line Coverage: 49.31%
Coverage Strength: 1.57 hits per line

💛 - Coveralls

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants