fix(eventhubs): treat a lost ownership claim as a normal outcome

[Johnathan W (j7nw4r)]

Summary

InMemoryCheckpointStore::claim_ownership returned an error when a competing instance rotated the ETag first. That error reached EventProcessor::run, which stopped every partition the instance owned. A lost claim is now a normal outcome, and the cycle continues without that partition.

Motivation

A lost claim is an ordinary result of load balancing under ProcessorStrategy::Balanced, and two instances that share one checkpoint store are the ordinary deployment shape. BlobCheckpointStore already treats the race as normal and omits the partition from its result. InMemoryCheckpointStore broke that contract, so one stale ETag cancelled the whole batch and the error travelled through load_balance and dispatch into run(). The same abort also discarded the rotated ETag of every partition that the batch had already renewed, which left the caller with records it could never renew again.

Changes

• InMemoryCheckpointStore::claim_ownership now skips a partition with a stale ETag, keeps the claims the batch already made, and returns the remaining partitions.
• A private try_update_ownership helper reports a lost claim as Ok(None) and reserves Err for a store failure, so no code path matches on a rendered error message.
• The lost claim logs at the debug level with the claim-conflict event field that BlobCheckpointStore uses.
• InMemoryCheckpointStore::update_ownership keeps its public signature and still returns an error for a stale ETag.
• The CheckpointStore::claim_ownership doc now states that a lost claim is not an error, and that an implementation must omit that partition from the returned vector.
• Added a ### Bugs Fixed entry to the crate CHANGELOG.

The production code in load_balancer.rs and processor.rs did not change, and there is no public API change.

Test plan

Four tests were added. Three of them failed before the fix.

• test_claim_ownership_lost_claim_is_not_an_error: a stale ETag returns Ok with no ownership, and the winner's record stays unchanged. Failed before the fix.
• test_claim_ownership_continues_past_a_lost_claim: a lost claim in the middle of a three-partition batch does not cancel the partitions around it, and the rotated ETags of the winners reach the caller. Failed before the fix.
• test_claim_ownership_invalid_ownership_still_errors: a validation failure still returns an error. This test passes before and after the fix. It guards against a fix that swallows every error in the loop, and it fails when claim_ownership discards errors instead of classifying them.
• load_balance_survives_a_lost_claim: load_balance returns Ok and owns no partition after a competing instance claims it first. Failed before the fix.

Every test runs offline. No live test was added, and no live test was run.

Results with CARGO_BUILD_JOBS=1 and --test-threads=1: 196 passed before the change, and 200 passed with 0 failed after it. cargo test --no-run --package azure_messaging_eventhubs with RUSTFLAGS=-Dwarnings exits 0. cargo fmt --check, cargo clippy --all-features --all-targets --no-deps, and cargo doc --no-deps each exit 0. The azure_messaging_eventhubs_checkpointstore_blob crate still builds.

Follow-up

No structured discriminator for a claim conflict is shared across CheckpointStore implementations. BlobCheckpointStore classifies the conflict with azure_core::Error::http_status() against StatusCode::PreconditionFailed and StatusCode::Conflict. InMemoryCheckpointStore produced ErrorKind::Other with a formatted message, which only a string match could classify. This change removes the need for a match, because the conflict never becomes an Error. The gap remains for a third-party CheckpointStore, which can still return an error for a lost claim and end run(). A proposed remedy is an ErrorKind::OwnershipLost variant on the crate's #[non_exhaustive] error enum, which load_balance can detect through the same source-chain downcast that find_link_stolen already uses.

The two stores now log the same event at different levels. BlobCheckpointStore uses info! and InMemoryCheckpointStore uses debug!, which is the level this issue asked for. The .NET SDK logs its equivalent OwnershipNotClaimable event at the Informational level. Aligning the two Rust stores on one level is a small follow-up if that is preferred.

Closes #5095

[azure-pipelines]

Azure Pipelines:
Successfully started running 1 pipeline(s).
3 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.