Repository navigation
Cell Dissemination (Partial messages) #8314
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 166 commits
Commits
Show all changes
178 commits
Select commit
Hold shift + click to select a range
7914128
Implement Cell Dissemination
dknopik 424dc79
Merge branch 'unstable' into partial-columns
dknopik 303c638
fix compilation
dknopik ad85059
fix some lints
dknopik 7b8cb0d
impl get_blobs_v2 fallback
dknopik efa9366
update gossipsub
dknopik 56f7a1f
remove redundant log
dknopik 2a721d7
try to upgrade column using block immediately
dknopik 5709408
publish data after block availability
dknopik 02194da
fix remaining lints and release test compilation
dknopik bd8000d
implement caching of partial messages
dknopik bb575ad
update gossipsub
dknopik eeaf03b
add CLI flags for disable request and disable support
dknopik 76eb32e
Merge branch 'unstable' into partial-columns
dknopik 47a4119
update CLI docs
dknopik 65c236a
do not error for column on failed observation check
dknopik ae9f493
Fix get blobs v2 tests
dknopik 8442bf3
Reintroduce sanity check for get blobs v2
dknopik 7c68ba4
Only count actually received blobs
dknopik 09e220b
Log full vs partial
dknopik 7cbb5c6
fix and improve logging
dknopik b6626f6
update gossipsub
dknopik fb956b9
Merge branch 'refs/heads/unstable' into partial-columns
dknopik f0b7d8f
look for block in data availability checker
dknopik 99d3568
re-enable SSE
dknopik 646fef5
Properly handly observing in new_for_block_publishing
dknopik 6794e5f
Fix cache and better logging
dknopik e98dce0
Fix invalid `UnexpectedDataColumn` error
dknopik 5e1f05d
remove custom cache in favour of reprocessing queue
dknopik 7067c7a
Merge branch 'unstable' into partial-columns
dknopik 695c648
publish partials on http API block
dknopik 6fb3721
experimentally full eager publish for API blocks
dknopik 870fa0d
experimental libp2p changes
dknopik 995344e
joaos libp2p changes
dknopik 2d524f2
Merge branch 'unstable' into partial-columns
dknopik af1499c
fix test
dknopik 209235b
Merge branch 'unstable' into partial-columns
dknopik 90266e7
fix function path
dknopik 4429de4
update libp2p
dknopik 7f190fe
do not eagerly publish all columns on proposal
dknopik 2df27a1
update libp2p again
dknopik bd62eaa
once again update libp2p
dknopik 849e5bc
merge from unstable
dknopik b725f58
fix import
dknopik 79c17e7
update for libp2p fix
dknopik 96b391f
Fix inverted condition
dknopik 4b4f92e
Another update for libp2p to fix received msgs
dknopik 5b2ee00
fix kzg verification
dknopik 65854ef
actually dequeue partials
dknopik 3484cf6
update libp2p
dknopik b4c24ba
Merge unstable
dknopik 002a5bc
Cargo fmt
dknopik 88c9c08
Refactor
pawanjay176 1f8a90a
Merge from unstable
dknopik b68a967
add getBlobsV3 metrics
dknopik 42ed853
update libp2p
dknopik 1f79e90
add cell usefulness metric definitions
dknopik 0f9380d
Merge from unstable
dknopik 24faa31
update libp2p
dknopik f0579c8
merge pawans refactor
dknopik 3f55ade
add PartialDataColumnHeader
dknopik 1885e25
remove eager sending of cells
dknopik 75a69a2
remove VerifiablePartialDataColumn
dknopik b0485de
Use separate network flow for partial messages
dknopik f505073
add component for tracking sent headers
dknopik 74a23b5
make early partial handling slightly less awkward
dknopik 074b137
Arc the cells in a partial data column sidecar
dknopik 76b4b5c
Rename "DanglingPartialDataColumn"
dknopik 07ff6db
Remove GossipVerifiedPartialDataColumn
dknopik 262aca8
Arc less
dknopik 1f59e20
Add validation and caching of partial header
dknopik d42cbbe
Fix obvious mistakes
dknopik 803b115
Remove unnecessary explicit init
dknopik 011b1bb
Remove outdated todo
dknopik 7ce4d23
Fix fetch blobs tests
dknopik fcdfa4c
Undo buggy changes in block publish flow
dknopik d5c57f7
fix SSE
dknopik ba3b6b0
Implement getting blobs after header arrival
dknopik 9e49854
Implement 3 of 4 missing metrics
dknopik 821e7d2
add version byte prefix to group id
dknopik a4526af
update libp2p
dknopik 5edaceb
merge from unstable
dknopik d9726ff
add logging on send
dknopik b9e09a9
Improve logging
dknopik c817af7
fix metric name
dknopik 096fb78
avoid completing already completed columsn
dknopik eb0ac14
fix: always publish partials after attempted getblobsv2/v3 and avoid …
dknopik 6712492
fmt
dknopik 81cce2c
add moar logs
dknopik 7fa4502
fix compilation
dknopik f2863ca
Cherry pick c3fb9eb0 for devnet retention fix
dknopik 6251407
update libp2p
dknopik c2a7c40
invert CLI param
dknopik 166bbfb
clean up
dknopik af796a3
Do maintenance and removal for partial assembler
dknopik 6cebaab
merge from unstable
dknopik 39cf8ec
implement spec change: request bitmap
dknopik 57befbd
Remove column from assembly when complete
dknopik bb44b26
merge from unstable
dknopik c6494f2
log silent if no peers support partial
dknopik 58cdcf4
only send partial if enabled
dknopik d794d4f
update libp2p
dknopik 1b3ccdd
merge from unstable
dknopik a102f6c
update libp2p
dknopik 970ce62
implement optional metadata function
dknopik 2a93e79
implement reporting of failures to libp2p
dknopik e18f07e
merge from unstable
dknopik 9cbde0b
fix lookup on unknown parent
dknopik 0651fc3
update libp2p
dknopik dc88ad7
merge from unstable
dknopik 76c9e92
fix inverted logic for sending
dknopik 4a964c0
merge from unstable
dknopik 799b0cc
merge from unstable and update libp2p
dknopik bc00446
Add metric for headerless partial verification
dknopik 6e23ad5
ensure atomic and guaranteed setting of has_local_blobs
dknopik 9941777
move send log to more sensible location
dknopik a2643c9
update libp2p
dknopik a840639
merge from unstable
dknopik b75f503
refactor: encode without cloning, no cell level arc, better validatio…
dknopik 4aa00ac
improve logging around getBlobsV3
dknopik 0fd7554
update libp2p
dknopik 5442ff4
mark peer as block-aware after receiving message
dknopik 61048e6
optional partial assembler
dknopik 682cf25
merge from unstable
dknopik 2f4e7af
add remaining verification steps
dknopik 20226bb
fix places relying on header in sidecar
dknopik b65d569
fix build
dknopik 9fbbaf9
add unit tests
dknopik 30536e1
merge from unstable
dknopik bdc1f09
update libp2p
dknopik 44a4fd5
merge from unstable
dknopik 2e7ae4d
merge from unstable
dknopik 1aa75da
merge from unstable
dknopik 5eb1a1d
address review
dknopik e1a1b76
fix lint
dknopik 8a1b80c
fix getBlobs when partial messages are not enabled
dknopik 3b94615
track latest cell timestamp
dknopik 26c4099
merge from unstable
dknopik e4a72ad
update ssz_types
dknopik ad57d83
Arc/Box some more
dknopik ca4b5f0
Use ListEncodedOption
dknopik edfa71c
share harness across tests
dknopik 00e9c83
Rename InconsistentMaxCount
dknopik 40125cd
Introduce error for missing column assembler
dknopik a0ffc72
get rid of expects
dknopik 64970f0
merge from unstable
dknopik 8a88f88
Metadata requests field matches spec name
dknopik d55cd2d
fix horrendous bug
dknopik 5eb7a7a
add cli test
dknopik 98fffcc
reuse block root
dknopik 90ec7b3
check for 1 to 1 mapping of subnet and column
dknopik 9adf8d1
use self.sampling_columns_for_epoch
dknopik c8cb635
merge from unstable
dknopik 455c7a4
merge from unstable
dknopik fea44da
struct variant for UnknownParentPartialDataColumn
dknopik f00e38f
rename len into verify_len
dknopik 8812fd3
fix metric definitions
dknopik 5fc3abe
penalize on undecodable topic
dknopik 02ffa62
add GOSSIP_UNACCEPTED_MESSAGES_PER_CLIENT
dknopik 5312b9d
nit typo
dknopik f34d6ab
fix comment on propagate_partial_validation_failure
dknopik 618986e
fix comment on init
dknopik f698fc9
FAILED_PARTIAL_PUBLISHES_PER_MAIN_TOPIC
dknopik 99c87fe
move merge to wrapper
dknopik 9e1e390
fix copy paste
dknopik 3006499
merge from unstable
dknopik bcb272e
organize imports
dknopik f7f8619
better validation flow around determine_missing_cells
dknopik cb7749d
update libp2p and move to master branch
dknopik 26cdb4a
merge from unstable
dknopik 948aa89
fmt
dknopik f52f0b3
fmt
dknopik 784cb95
avoid creating sidecar and header on publish if disabled
dknopik d3a4145
Fix fmt
jimmygchen 4c5b6de
Merge remote-tracking branch 'origin/unstable' into fork/dknopik/part…
jimmygchen 806ecbb
Fix fmt
jimmygchen 0e075c5
Fix `PartialDataColumnSidecar::get()` blob index mapping.
jimmygchen 4cd67db
Fix syn duplicate in Cargo.lock (see 9f3873f2bf) and remove comment c…
jimmygchen File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just adding a comment here as a reminder to update before we merge