diff --git a/docs/reference/server/configuration.mdx b/docs/reference/server/configuration.mdx index 64f1a1abfd..e465b41031 100644 --- a/docs/reference/server/configuration.mdx +++ b/docs/reference/server/configuration.mdx @@ -354,10 +354,22 @@ pruning then deletes local history only after its segment is uploaded. Each run records **history.segmentTargetBytes** when it starts, so a new value applies only to runs that start after the node restarts with it. +Runs that already exist when offload turns on migrate, including runs that +started while it was off. The first command that leaves such a run running seals +the oldest part of its local history. Each later command of the run seals one +more segment, until a segment reaches the command itself. The run then continues +like a run that started with offload on. Events that the history window removed +before the migration stay removed. A run that is already closed is not migrated; +deleting it after its retention deletes its local history. A run that closes +during its migration keeps the rest of its history on local disk until it is +deleted. + Turning offload off stops sealing. History of segments sealed earlier stays on local disk until each segment is uploaded; then pruning returns to the window rule. Commands applied while offload is off belong to no segment, so turning it -on again starts a new segment after them. +on again starts a new segment after them. Turning offload off also ends every +migration; the rest of each such run's history follows the window rule like +those commands. The lease holder uploads each sealed segment to the primary object store, under the **history/** prefix of the bucket that holds the commit log. It uploads a diff --git a/server-zd/crates/dexzd-apply/README.md b/server-zd/crates/dexzd-apply/README.md index d538e78814..be1687c858 100644 --- a/server-zd/crates/dexzd-apply/README.md +++ b/server-zd/crates/dexzd-apply/README.md @@ -52,7 +52,8 @@ superseded inside it, or when the command closed the run. Sealing writes a SEALE with a superseded seq inside the range, and the run's `HistoryUploadPending` index row, and emits `HistorySegmentSealed`. A value revived later keeps its row, so the segment's content never changes. Runs that started while offload was off -keep no segment state. While the partition does not offload, apply seals nothing, +keep no segment state until they migrate; see History migration. While the +partition does not offload, apply seals nothing, and each command of a run with segment state moves the open segment past itself. The seqs and events of those commands belong to no segment, and their local rows follow window pruning, so a segment opened after offload is turned on again never @@ -74,6 +75,35 @@ run state limit. Run purge deletes the pending index row with the run. Retention segment, as it does for an unacknowledged outbox message, so purging never abandons an upload. +## History migration + +A run that started while its partition did not offload history has no segment +state. The first command that leaves such a run running while its partition +offloads starts the migration: it sets `history_migration_started_seq` to its +seq and opens the segment at the run's first seq and event 1. While the run +migrates, the open segment holds every unsealed seq up to the current command +and tracks no bytes, and each command of the run seals one segment of it. + +A migration segment ends at a whole seq: the first one where it holds at least +two history records and reaches the target bytes, 10,000 superseded values, or +10,000 history rows. Otherwise it ends at the command's own seq, which ends the +migration. Cutting it reads one bounded page of history metadata and of table G. +Because segments end only at whole seqs, a migration step is bounded by the +largest single command in the migrated range, as sealing is: one seq whose +command superseded 30,000 values seals all of them in one step. +Two records per segment let the migration gain on the run, since each command +adds one record. The segment holds the rows still stored in its seqs, so seqs +that window pruning deleted earlier are holes inside the reported pruned event +range. Its event range comes from the metadata `last_assigned_event_id`; records +written before that field are decoded instead. Pruning keeps every row above U +while the partition offloads, so nothing is deleted before its segment uploads. + +A closed run never starts migrating, so retention purges its local history +directly. A run that closes during its migration seals one more segment at its +closing command and then keeps the rest local until purge; a time travel that +reopens it continues the migration. Turning offload off ends a migration, and the +unsealed rest joins the gap of commands applied while offload is off. + ## Value and storage accounting Each run tracks encoded Flow and History row bytes, including keys. Live and superseded value bytes use the encoded value sizes in birth records; garbage rows identify the superseded portion. Apply updates these totals from each command's writes and deletes. diff --git a/server-zd/crates/dexzd-apply/src/command_handlers/run_cursor.rs b/server-zd/crates/dexzd-apply/src/command_handlers/run_cursor.rs index b3171878ee..48e3f38dd9 100644 --- a/server-zd/crates/dexzd-apply/src/command_handlers/run_cursor.rs +++ b/server-zd/crates/dexzd-apply/src/command_handlers/run_cursor.rs @@ -17,7 +17,7 @@ use crate::command_handlers::{ CommandScope, HandlerOutput, ReplayedCommandCounters, ReplayedCommandOutput, }; use crate::history; -use crate::history_offload::segment_seal::seal_history_segment_if_due; +use crate::history_offload::segment_migration::seal_or_migrate_history_segment; use crate::projections::history_event_links::{ HistoryEventIndex, PendingHistoryEvents, StepEventLink, resolve_step_event_links, with_recorded_event_ids, @@ -283,7 +283,7 @@ impl RunCursor { ); search_document::refresh_search_document(transaction, &self, scope)?; history::after_command(transaction, &mut self, window_events)?; - seal_history_segment_if_due(transaction, &mut self, scope.now_micros, has_closed_run)?; + seal_or_migrate_history_segment(transaction, &mut self, scope.now_micros, has_closed_run)?; self.rows.put_run_record(transaction, &self.record); if scope.is_replay { reconcile_run_storage(transaction, &mut self)?; diff --git a/server-zd/crates/dexzd-apply/src/history_offload/mod.rs b/server-zd/crates/dexzd-apply/src/history_offload/mod.rs index 6087a5db20..debd6703fd 100644 --- a/server-zd/crates/dexzd-apply/src/history_offload/mod.rs +++ b/server-zd/crates/dexzd-apply/src/history_offload/mod.rs @@ -9,5 +9,6 @@ //! Offloading run history to object storage, gated by a committed partition mode. pub(crate) mod offload_mode; +pub(crate) mod segment_migration; pub(crate) mod segment_seal; pub(crate) mod segment_upload; diff --git a/server-zd/crates/dexzd-apply/src/history_offload/segment_migration.rs b/server-zd/crates/dexzd-apply/src/history_offload/segment_migration.rs new file mode 100644 index 0000000000..49208caadd --- /dev/null +++ b/server-zd/crates/dexzd-apply/src/history_offload/segment_migration.rs @@ -0,0 +1,299 @@ +// Copyright (c) 2026 Super Durable, Inc. +// +// Licensed under the Sustainable Use License 1.0. +// You may not use this file except in compliance with the License. +// See the LICENSE file in the repository root. +// +// SPDX-License-Identifier: LicenseRef-Sustainable-Use-1.0 + +//! Migrating history a run recorded before it had segment state (plan section 5.17). + +use std::collections::BTreeMap; + +use dexzd_keyspace::row_keys::history_record_key; +use dexzd_keyspace::{ColumnFamily, RunTable, run_table_prefix}; +use dexzd_protocol::dex::FlowStatus; +use dexzd_protocol::dexzd::v1::{HistoryRecord, HistoryRowMetadata, RunRecord, ValueGarbageRecord}; + +use crate::ApplyError; +use crate::command_handlers::run_cursor::RunCursor; +use crate::history::genesis_seq; +use crate::history_offload::offload_mode::is_history_offload_enabled; +use crate::history_offload::segment_seal::{ + MAX_SEGMENT_SUPERSEDED_VALUES, SealedSegmentBounds, garbage_identity, + open_segment_after_command, seal_history_segment_if_due, segment_target_bytes, + write_sealed_segment, +}; +use crate::state_access::FlowTransaction; + +/// Bounds the history rows of one migration segment, so cutting it reads a bounded page. +const MAX_MIGRATION_SEGMENT_HISTORY_ROWS: usize = 10_000; +/// FlowStart records event 1 of every run. +const FIRST_EVENT_ID: i64 = 1; + +/// Seals a due open segment, or starts or continues migrating older history, sealing at most one. +/// +/// `has_closed_run` tells whether this command closed the run; `now_micros` is its proposal time. +pub(crate) fn seal_or_migrate_history_segment( + transaction: &mut FlowTransaction<'_>, + cursor: &mut RunCursor, + now_micros: i64, + has_closed_run: bool, +) -> Result<(), ApplyError> { + let record = &cursor.record; + if record.history_open_segment_first_seq == 0 { + // Closed runs never start: migrating them only uploads history that purge deletes next. + if is_running(record) && !record.is_purging && is_history_offload_enabled(transaction)? { + start_migration(transaction, cursor)?; + migrate_one_segment(transaction, cursor, now_micros)?; + } + return Ok(()); + } + if record.history_migration_started_seq == 0 { + return seal_history_segment_if_due(transaction, cursor, now_micros, has_closed_run); + } + if !is_history_offload_enabled(transaction)? { + // The unsealed rest joins the gap of commands applied while offload is off. + cursor.record.history_migration_started_seq = 0; + return seal_history_segment_if_due(transaction, cursor, now_micros, has_closed_run); + } + if record.is_purging || !(has_closed_run || is_running(record)) { + return Ok(()); + } + migrate_one_segment(transaction, cursor, now_micros) +} + +/// Opens the segment state of a run without one at its first seq and event 1. +fn start_migration( + transaction: &FlowTransaction<'_>, + cursor: &mut RunCursor, +) -> Result<(), ApplyError> { + let first_seq = genesis_seq(transaction, cursor.rows)?; + let record = &mut cursor.record; + record.history_migration_started_seq = cursor.seq; + record.history_open_segment_first_seq = first_seq; + record.history_open_segment_first_event_id = FIRST_EVENT_ID; + record.history_open_segment_bytes = 0; + record.history_open_segment_superseded_value_count = 0; + Ok(()) +} + +/// Seals the next migration segment; the migration ends once a segment reaches this command. +fn migrate_one_segment( + transaction: &mut FlowTransaction<'_>, + cursor: &mut RunCursor, + now_micros: i64, +) -> Result<(), ApplyError> { + let bounds = cut_migration_segment(transaction, cursor)?; + write_sealed_segment(transaction, cursor, &bounds, now_micros)?; + if bounds.last_seq == cursor.seq { + cursor.record.history_migration_started_seq = 0; + open_segment_after_command(cursor); + } else { + cursor.record.history_open_segment_first_seq = bounds.last_seq + 1; + cursor.record.history_open_segment_first_event_id = bounds.last_event_id + 1; + } + Ok(()) +} + +/// Stored history of one seq inside the migration range. +#[derive(Default)] +struct SeqHistory { + bytes: u64, + history_rows: usize, + superseded_values: u64, + record: Option, +} + +/// What a migration segment holds so far. +#[derive(Default)] +struct SegmentTotals { + bytes: u64, + history_rows: usize, + superseded_values: u64, + records: usize, +} + +impl SegmentTotals { + fn add(&mut self, seq_history: &SeqHistory) { + self.bytes = self.bytes.saturating_add(seq_history.bytes); + self.history_rows += seq_history.history_rows; + self.superseded_values += seq_history.superseded_values; + self.records += usize::from(seq_history.record.is_some()); + } + + /// Two records at least, so each command seals more history than it adds. + fn is_complete(&self, target_bytes: u64) -> bool { + self.records >= 2 + && (self.bytes >= target_bytes + || self.superseded_values >= MAX_SEGMENT_SUPERSEDED_VALUES + || self.history_rows >= MAX_MIGRATION_SEGMENT_HISTORY_ROWS) + } +} + +/// Returns the bounds of the next migration segment, which starts at the open segment and ends +/// at a whole seq: the first one where the segment holds two history records and reaches the +/// target bytes, the superseded value limit, or the history row limit; otherwise this command's +/// seq. When a bounded page ends first, the segment ends with that page or its second record. +fn cut_migration_segment( + transaction: &FlowTransaction<'_>, + cursor: &RunCursor, +) -> Result { + let first_seq = cursor.record.history_open_segment_first_seq; + let first_event_id = cursor.record.history_open_segment_first_event_id; + let current_seq = cursor.seq; + let mut seqs = BTreeMap::::new(); + let metadata_complete_through = scan_history_metadata(transaction, cursor, &mut seqs)?; + let garbage_complete_through = scan_superseded_values(transaction, cursor, &mut seqs)?; + let complete_through = metadata_complete_through.min(garbage_complete_through); + // The command's own record is stored, so the range holds at least one record. + let second_record_seq = seqs + .iter() + .filter(|(_, seq_history)| seq_history.record.is_some()) + .map(|(seq, _)| *seq) + .nth(1) + .unwrap_or(current_seq); + let target_bytes = segment_target_bytes(&cursor.record); + let mut totals = SegmentTotals::default(); + let cut_seq = seqs + .iter() + .take_while(|(seq, _)| **seq <= complete_through) + .find_map(|(seq, seq_history)| { + totals.add(seq_history); + totals.is_complete(target_bytes).then_some(*seq) + }); + // Whole seqs can pass the limits; the command that recorded such a seq wrote as many rows. + let last_seq = cut_seq.unwrap_or_else(|| complete_through.max(second_record_seq)); + let last_event_id = if last_seq == current_seq { + cursor.record.next_event_id.max(1) - 1 + } else { + newest_event_id_through(transaction, cursor, &seqs, last_seq, first_event_id - 1)? + }; + Ok(SealedSegmentBounds { + first_seq, + last_seq, + first_event_id, + last_event_id, + }) +} + +/// Adds the history records and checkpoints of the migration range to `seqs`, reading at most +/// one row more than the row limit. Returns the last seq whose rows were all read. +fn scan_history_metadata( + transaction: &FlowTransaction<'_>, + cursor: &RunCursor, + seqs: &mut BTreeMap, +) -> Result { + let rows = cursor.rows; + let prefix = run_table_prefix(rows.flow_hash, rows.run_id, RunTable::HistoryRowMetadata); + let limit = MAX_MIGRATION_SEGMENT_HISTORY_ROWS + 1; + let page = transaction.scan_messages::( + ColumnFamily::Flow, + &seq_bound(&prefix, cursor.record.history_open_segment_first_seq), + &seq_bound(&prefix, cursor.seq + 1), + limit, + "history metadata", + )?; + let is_truncated = page.len() == limit; + let mut last_seq = cursor.seq; + for (key, metadata) in page { + let suffix = key + .strip_prefix(prefix.as_slice()) + .filter(|suffix| suffix.len() == 13) + .ok_or_else(|| ApplyError::InvariantViolation { + detail: "history metadata key must have a 13-byte suffix".to_owned(), + })?; + let seq = u64::from_be_bytes(suffix[..8].try_into().expect("eight seq bytes")); + last_seq = seq; + let seq_history = seqs.entry(seq).or_default(); + seq_history.bytes = seq_history.bytes.saturating_add(metadata.row_size_bytes); + seq_history.history_rows += 1; + if suffix[8] == RunTable::HistoryRecord as u8 { + seq_history.record = Some(metadata); + } + } + Ok(if is_truncated { + last_seq - 1 + } else { + cursor.seq + }) +} + +/// Adds the values superseded inside the migration range to `seqs`, reading at most one more +/// than the superseded value limit. Returns the last seq whose values were all read. +fn scan_superseded_values( + transaction: &FlowTransaction<'_>, + cursor: &RunCursor, + seqs: &mut BTreeMap, +) -> Result { + let rows = cursor.rows; + let prefix = run_table_prefix(rows.flow_hash, rows.run_id, RunTable::ValueGarbage); + let limit = + usize::try_from(MAX_SEGMENT_SUPERSEDED_VALUES).expect("the value limit fits in usize") + 1; + let page = transaction.scan_messages::( + ColumnFamily::Flow, + &seq_bound(&prefix, cursor.record.history_open_segment_first_seq), + &seq_bound(&prefix, cursor.seq + 1), + limit, + "value garbage", + )?; + let is_truncated = page.len() == limit; + let mut last_seq = cursor.seq; + for (key, garbage) in page { + let (seq, _) = garbage_identity(&key, &prefix)?; + last_seq = seq; + let seq_history = seqs.entry(seq).or_default(); + seq_history.bytes = seq_history.bytes.saturating_add(garbage.size_bytes); + seq_history.superseded_values += 1; + } + Ok(if is_truncated { + last_seq - 1 + } else { + cursor.seq + }) +} + +/// Returns the newest event ID that the records in `seqs` through `last_seq` assigned, or +/// `before_range` when none did. +/// +/// Records older than `last_assigned_event_id` are decoded; events of pruned records then widen +/// the next range. +fn newest_event_id_through( + transaction: &FlowTransaction<'_>, + cursor: &RunCursor, + seqs: &BTreeMap, + last_seq: u64, + before_range: i64, +) -> Result { + let mut newest = before_range; + for (seq, seq_history) in seqs.range(..=last_seq) { + let Some(metadata) = &seq_history.record else { + continue; + }; + if metadata.last_assigned_event_id > 0 { + newest = newest.max(metadata.last_assigned_event_id); + continue; + } + let key = history_record_key(cursor.rows.flow_hash, cursor.rows.run_id, *seq); + let record = transaction + .get_message::(ColumnFamily::History, &key, "history record")? + .ok_or_else(|| ApplyError::InvariantViolation { + detail: format!("history record {seq} has metadata but no row"), + })?; + if let Some(event_id) = record.events.iter().map(|event| event.event_id).max() { + newest = newest.max(event_id); + } + } + Ok(newest) +} + +/// Returns the first key of `seq` in a run table whose keys start with an eight-byte seq. +fn seq_bound(prefix: &[u8], seq: u64) -> Vec { + let mut key = prefix.to_vec(); + key.extend_from_slice(&seq.to_be_bytes()); + key +} + +fn is_running(record: &RunRecord) -> bool { + record.status == FlowStatus::Running as i32 +} diff --git a/server-zd/crates/dexzd-apply/src/history_offload/segment_seal.rs b/server-zd/crates/dexzd-apply/src/history_offload/segment_seal.rs index 0da191a39d..1f2657e6f9 100644 --- a/server-zd/crates/dexzd-apply/src/history_offload/segment_seal.rs +++ b/server-zd/crates/dexzd-apply/src/history_offload/segment_seal.rs @@ -21,7 +21,8 @@ use dexzd_keyspace::row_keys::{ }; use dexzd_keyspace::{ColumnFamily, RunTable, ValueId, run_prefix, run_table_prefix}; use dexzd_protocol::dexzd::v1::{ - HistorySegmentRecord, HistorySegmentState, HistorySegmentValueRecord, ValueGarbageRecord, + HistorySegmentRecord, HistorySegmentState, HistorySegmentValueRecord, RunRecord, + ValueGarbageRecord, }; use prost::Message; @@ -77,14 +78,7 @@ pub(crate) fn seal_history_segment_if_due( record.history_open_segment_superseded_value_count = record .history_open_segment_superseded_value_count .saturating_add(additions.superseded_values); - let target_bytes = match record - .settings - .unwrap_or_default() - .history_segment_target_bytes - { - 0 => DEFAULT_HISTORY_SEGMENT_TARGET_BYTES, - configured => configured, - }; + let target_bytes = segment_target_bytes(record); let is_due = has_closed_run || record.history_open_segment_bytes >= target_bytes || record.history_open_segment_superseded_value_count >= MAX_SEGMENT_SUPERSEDED_VALUES; @@ -144,11 +138,37 @@ fn seal_open_segment( transaction: &mut FlowTransaction<'_>, cursor: &mut RunCursor, now_micros: i64, +) -> Result<(), ApplyError> { + let bounds = SealedSegmentBounds { + first_seq: cursor.record.history_open_segment_first_seq, + last_seq: cursor.seq, + first_event_id: cursor.record.history_open_segment_first_event_id, + last_event_id: cursor.record.next_event_id.max(1) - 1, + }; + write_sealed_segment(transaction, cursor, &bounds, now_micros)?; + open_segment_after_command(cursor); + Ok(()) +} + +/// Seq and event ID range of a segment that apply seals. +pub(crate) struct SealedSegmentBounds { + pub(crate) first_seq: u64, + pub(crate) last_seq: u64, + pub(crate) first_event_id: i64, + /// Smaller than `first_event_id` when the seqs assigned no event. + pub(crate) last_event_id: i64, +} + +/// Writes the SEALED segment, its value list, and the pending index row, then emits the effect. +pub(crate) fn write_sealed_segment( + transaction: &mut FlowTransaction<'_>, + cursor: &mut RunCursor, + bounds: &SealedSegmentBounds, + now_micros: i64, ) -> Result<(), ApplyError> { let rows = cursor.rows; - let first_seq = cursor.record.history_open_segment_first_seq; - let last_seq = cursor.seq; - let next_event_id = cursor.record.next_event_id.max(1); + let first_seq = bounds.first_seq; + let last_seq = bounds.last_seq; let garbage_prefix = run_table_prefix(rows.flow_hash, rows.run_id, RunTable::ValueGarbage); let mut start = garbage_prefix.clone(); start.extend_from_slice(&first_seq.to_be_bytes()); @@ -181,8 +201,8 @@ fn seal_open_segment( &HistorySegmentRecord { first_seq, last_seq, - first_event_id: cursor.record.history_open_segment_first_event_id, - last_event_id: next_event_id - 1, + first_event_id: bounds.first_event_id, + last_event_id: bounds.last_event_id, state: HistorySegmentState::Sealed as i32, sealed_at_micros: now_micros, superseded_value_count, @@ -196,14 +216,25 @@ fn seal_open_segment( history_upload_pending_index_key(partition, rows.flow_hash, rows.run_id), Vec::new(), ); - open_segment_after_command(cursor); let run_id = cursor.run_id(); cursor.add_effect(ApplyEffect::HistorySegmentSealed { run_id }); Ok(()) } +/// Returns the run's segment target bytes; runs older than the setting record zero. +pub(crate) fn segment_target_bytes(record: &RunRecord) -> u64 { + match record + .settings + .unwrap_or_default() + .history_segment_target_bytes + { + 0 => DEFAULT_HISTORY_SEGMENT_TARGET_BYTES, + configured => configured, + } +} + /// Opens an empty segment whose seqs and events start right after the command's. -fn open_segment_after_command(cursor: &mut RunCursor) { +pub(crate) fn open_segment_after_command(cursor: &mut RunCursor) { let first_seq = cursor.seq + 1; let record = &mut cursor.record; record.history_open_segment_first_seq = first_seq; @@ -213,7 +244,7 @@ fn open_segment_after_command(cursor: &mut RunCursor) { } /// Returns the superseded seq and value ID of a value garbage key. -fn garbage_identity(key: &[u8], prefix: &[u8]) -> Result<(u64, ValueId), ApplyError> { +pub(crate) fn garbage_identity(key: &[u8], prefix: &[u8]) -> Result<(u64, ValueId), ApplyError> { let suffix = key .strip_prefix(prefix) .filter(|suffix| suffix.len() == 20) diff --git a/server-zd/crates/dexzd-apply/tests/history_migration.rs b/server-zd/crates/dexzd-apply/tests/history_migration.rs new file mode 100644 index 0000000000..07bafa1198 --- /dev/null +++ b/server-zd/crates/dexzd-apply/tests/history_migration.rs @@ -0,0 +1,1218 @@ +// Copyright (c) 2026 Super Durable, Inc. +// +// Licensed under the Sustainable Use License 1.0. +// You may not use this file except in compliance with the License. +// See the LICENSE file in the repository root. +// +// SPDX-License-Identifier: LicenseRef-Sustainable-Use-1.0 + +//! Migrating the history of runs that started before their partition offloaded history. + +use std::collections::{BTreeMap, BTreeSet}; + +use dexzd_apply::read_model::{get_history_events, read_history_segment_content}; +use dexzd_apply::{ + ApplyContext, ApplyEffect, ApplyOutcome, FlowStateReader, StateReadError, StateRow, StateWrite, + apply_flow_command, +}; +use dexzd_history_segment::{ + HistorySegmentContent, decode_history_segment, encode_history_segment, +}; +use dexzd_keyspace::row_keys::{ + history_record_key, history_row_metadata_key, history_segment_value_prefix, + history_upload_pending_index_key, prefix_range, run_named_key, run_singleton_key, +}; +use dexzd_keyspace::{ + ColumnFamily, IndexKind, PartitionHasher, RunId, RunTable, TenantId, ValueId, run_prefix, + run_table_prefix, +}; +use dexzd_protocol::dex::{ + AttributeWrite, ChannelMessage, FlowResetType, FlowTimeoutPolicy, GetHistoryEventsResponse, + PublishToChannelRequest, ResetFlowRequest, SetAttributesRequest, StartFlowRequest, + StopFlowRequest, StopType, Value, WorkerTarget, value, +}; +use dexzd_protocol::dexzd::v1::{ + AttributeRecord, AttributesSetCommand, ChannelPublishCommand, CommandReply, FlowCommand, + FlowStartCommand, FlowStopCommand, HistoryRecord, HistoryRowMetadata, HistorySegmentRecord, + HistorySegmentState, HistorySegmentUploadedCommand, HistorySegmentValueRecord, + RetentionTimerId, RunPurgeCommand, RunRecord, RunSettings, SetHistoryOffloadModeCommand, + TimeTravelCommand, TimerId, TimersFiredCommand, command_reply, flow_command::Command, + stored_value, timer_id, +}; +use prost::Message; + +const SECOND: i64 = 1_000_000; +/// Most commands one migration may take in these tests before it must have caught up. +const MAX_MIGRATION_COMMANDS: usize = 60; + +#[derive(Default, Clone, PartialEq, Eq, Debug)] +struct MemoryState { + rows: BTreeMap<(ColumnFamily, Vec), Vec>, +} + +impl FlowStateReader for MemoryState { + fn get( + &self, + column_family: ColumnFamily, + key: &[u8], + ) -> Result>, StateReadError> { + Ok(self.rows.get(&(column_family, key.to_vec())).cloned()) + } + + fn scan( + &self, + column_family: ColumnFamily, + start: &[u8], + end: &[u8], + limit: usize, + ) -> Result, StateReadError> { + Ok(self + .rows + .range((column_family, start.to_vec())..(column_family, end.to_vec())) + .take(limit) + .map(|((_, key), value)| (key.clone(), value.clone())) + .collect()) + } +} + +impl MemoryState { + fn commit(&mut self, outcome: &ApplyOutcome) { + for write in &outcome.writes { + match write { + StateWrite::Put { + column_family, + key, + value, + } => { + self.rows + .insert((*column_family, key.clone()), value.clone()); + } + StateWrite::Delete { column_family, key } + | StateWrite::SingleDelete { column_family, key } => { + self.rows.remove(&(*column_family, key.clone())); + } + StateWrite::DeleteRange { + column_family, + start, + end, + } => { + let doomed: Vec<_> = self + .rows + .range((*column_family, start.clone())..(*column_family, end.clone())) + .map(|(key, _)| key.clone()) + .collect(); + for key in doomed { + self.rows.remove(&key); + } + } + } + } + } +} + +struct Harness { + state: MemoryState, + context: ApplyContext, + now_micros: i64, + flow_id: String, + run_id: Vec, + settings: RunSettings, + request_count: u32, +} + +impl Harness { + fn new(settings: RunSettings) -> Self { + let flow_id = "migrating-flow"; + let hasher = PartitionHasher::new(&[7; 16], 1024).expect("valid partition hasher"); + let flow_hash = hasher.flow_hash(TenantId::DEFAULT, flow_id); + Self { + state: MemoryState::default(), + context: ApplyContext { + partition: hasher.partition_of(flow_hash), + flow_hash, + }, + now_micros: 1_800_000_000 * SECOND, + flow_id: flow_id.to_owned(), + run_id: Vec::new(), + settings, + request_count: 0, + } + } + + /// A run that started while its partition did not offload, with `commands` commands since. + fn legacy(settings: RunSettings, commands: usize) -> Self { + let mut harness = Self::new(settings); + harness.start(); + for round in 0..commands { + harness.apply_round(&format!("legacy{round}")); + } + assert_eq!(harness.run_record().history_open_segment_first_seq, 0); + harness + } + + fn command(&mut self, body: Command) -> FlowCommand { + self.now_micros += 1000; + FlowCommand { + flow_id: self.flow_id.clone(), + run_id: if matches!( + body, + Command::FlowStart(_) | Command::SetHistoryOffloadMode(_) + ) { + Vec::new() + } else { + self.run_id.clone() + }, + expected_run_attempt: 0, + epoch: 1, + behavior_version: 1, + proposed_at_micros: self.now_micros, + command_id: (self.now_micros as u128).to_be_bytes().to_vec(), + tenant_id: 0, + is_transactional_rpc_in_flight: false, + command: Some(body), + } + } + + /// Applies twice on the committed state, requires identical outcomes, and commits. + fn apply(&mut self, body: Command) -> ApplyOutcome { + let command = self.command(body); + let outcome = + apply_flow_command(&self.context, &self.state, &command).expect("apply succeeds"); + let again = + apply_flow_command(&self.context, &self.state, &command).expect("apply succeeds"); + assert_eq!(outcome, again, "apply must be deterministic"); + self.state.commit(&outcome); + outcome + } + + fn set_offload_mode(&mut self, is_enabled: bool) { + let outcome = self.apply(Command::SetHistoryOffloadMode( + SetHistoryOffloadModeCommand { + enabled: is_enabled, + partition: u32::from(self.context.partition.0), + }, + )); + assert!(is_ok(&outcome.reply), "{:?}", outcome.reply); + } + + fn start(&mut self) -> ApplyOutcome { + let outcome = self.apply(Command::FlowStart(FlowStartCommand { + new_run_id: vec![9; 16], + request: Some(StartFlowRequest { + flow_id: self.flow_id.clone(), + flow_type: "MigratingFlow".to_owned(), + request_id: "start-1".to_owned(), + ..StartFlowRequest::default() + }), + settings: Some(self.settings), + resolved_timeout_policy: FlowTimeoutPolicy::Fail as i32, + worker_target: Some(WorkerTarget { is_pull_mode: true }), + ..FlowStartCommand::default() + })); + let Some(command_reply::Reply::StartFlow(reply)) = &outcome.reply.reply else { + panic!("the run starts: {:?}", outcome.reply); + }; + self.run_id = reply.run_id.clone(); + outcome + } + + /// Replaces the `profile` Attribute when `tag` ends with an even digit and publishes to a + /// Channel otherwise, so records alternate between superseding a value and adding an event. + fn apply_round(&mut self, tag: &str) -> ApplyOutcome { + let is_write = tag + .bytes() + .last() + .is_some_and(|digit| (digit - b'0').is_multiple_of(2)); + if is_write { + self.set_attribute("profile", large_text(tag, 300)) + } else { + self.publish("updates", &large_text(tag, 200)) + } + } + + fn set_attribute(&mut self, key: &str, text: String) -> ApplyOutcome { + self.set_attributes(&[(key.to_owned(), text)]) + } + + fn set_attributes(&mut self, values: &[(String, String)]) -> ApplyOutcome { + self.request_count += 1; + let request_id = format!("write-{}", self.request_count); + let outcome = self.apply(Command::AttributesSet(AttributesSetCommand { + request: Some(SetAttributesRequest { + attributes: values + .iter() + .map(|(key, text)| AttributeWrite { + key: key.clone(), + value: Some(string_value(text)), + ..AttributeWrite::default() + }) + .collect(), + request_id: request_id.clone(), + ..SetAttributesRequest::default() + }), + hard_delete_objects: Vec::new(), + request_fingerprint: request_id.into_bytes(), + })); + assert!(is_ok(&outcome.reply), "{:?}", outcome.reply); + outcome + } + + fn publish(&mut self, channel_name: &str, text: &str) -> ApplyOutcome { + self.request_count += 1; + let outcome = self.apply(Command::ChannelPublish(ChannelPublishCommand { + request: Some(PublishToChannelRequest { + messages: vec![ChannelMessage { + channel_name: channel_name.to_owned(), + value: Some(string_value(text)), + message_id: format!("message-{}", self.request_count), + }], + ..PublishToChannelRequest::default() + }), + })); + assert!(is_ok(&outcome.reply), "{:?}", outcome.reply); + outcome + } + + fn stop(&mut self) -> ApplyOutcome { + let outcome = self.apply(Command::FlowStop(FlowStopCommand { + request: Some(StopFlowRequest { + flow_id: self.flow_id.clone(), + stop_type: StopType::Cancel as i32, + ..StopFlowRequest::default() + }), + })); + assert!(is_ok(&outcome.reply), "{:?}", outcome.reply); + outcome + } + + fn travel_to_seq(&mut self, seq: u64) -> ApplyOutcome { + let timestamp_micros = self.record_metadata(seq).timestamp_micros; + let outcome = self.apply(Command::TimeTravel(TimeTravelCommand { + should_restore_running_as_pending: false, + request: Some(ResetFlowRequest { + flow_id: self.flow_id.clone(), + reset_type: FlowResetType::HistoryEventTime as i32, + reason: "test".to_owned(), + skip_writes_reapply: true, + history_event_time: (i128::from(timestamp_micros) * 1000).to_string(), + ..ResetFlowRequest::default() + }), + })); + assert!( + matches!( + outcome.reply.reply, + Some(command_reply::Reply::TimeTravel(_)) + ), + "{:?}", + outcome.reply + ); + outcome + } + + fn fire_retention(&mut self) -> ApplyOutcome { + self.now_micros = self.run_record().retention_deadline_micros; + self.apply(Command::TimersFired(TimersFiredCommand { + timers: vec![TimerId { + run_attempt: 0, + kind: Some(timer_id::Kind::Retention(RetentionTimerId {})), + }], + })) + } + + /// Acknowledges the oldest SEALED segment with the checksum of its encoded content. + fn acknowledge_oldest_sealed(&mut self) -> Option { + let uploaded_through_seq = self.run_record().history_uploaded_through_seq; + let segment = self + .segments() + .into_iter() + .find(|segment| segment.last_seq > uploaded_through_seq)?; + let encoded = encode_history_segment(&self.segment_content(segment.first_seq)) + .expect("the content encodes"); + let outcome = self.apply(Command::HistorySegmentUploaded( + HistorySegmentUploadedCommand { + first_seq: segment.first_seq, + last_seq: segment.last_seq, + content_checksum: encoded.content_checksum.to_vec(), + object_size_bytes: encoded.bytes.len() as u64, + }, + )); + assert!(is_ok(&outcome.reply), "{:?}", outcome.reply); + assert_eq!( + self.run_record().history_uploaded_through_seq, + segment.last_seq + ); + Some(segment) + } + + fn acknowledge_every_sealed(&mut self) { + while self.acknowledge_oldest_sealed().is_some() {} + assert!(!self.has_upload_pending_index_row()); + } + + /// Applies rounds until the run's migration caught up, requiring exactly one sealed segment + /// per command. Returns the commands the migration took. + fn migrate_to_completion(&mut self, tag: &str) -> usize { + for command_count in 1..=MAX_MIGRATION_COMMANDS { + let outcome = self.apply_round(&format!("{tag}{command_count}")); + assert_eq!( + sealed_effect_count(&outcome), + 1, + "every command of a migrating run seals one segment" + ); + assert_segment_chain(self); + if self.run_record().history_migration_started_seq == 0 { + return command_count; + } + } + panic!("the migration did not catch up within {MAX_MIGRATION_COMMANDS} commands"); + } + + fn run(&self) -> RunId { + RunId::from_slice(&self.run_id).expect("the harness has a run") + } + + fn get( + &self, + column_family: ColumnFamily, + key: &[u8], + ) -> Option { + self.state + .rows + .get(&(column_family, key.to_vec())) + .map(|bytes| RowMessage::decode(bytes.as_slice()).expect("row decodes")) + } + + fn run_record(&self) -> RunRecord { + let key = run_singleton_key(self.context.flow_hash, self.run(), RunTable::RunRecord); + self.get(ColumnFamily::Flow, &key) + .expect("the run record exists") + } + + fn last_seq(&self) -> u64 { + self.run_record().next_run_seq - 1 + } + + fn record_metadata(&self, seq: u64) -> HistoryRowMetadata { + self.get( + ColumnFamily::Flow, + &history_row_metadata_key( + self.context.flow_hash, + self.run(), + seq, + RunTable::HistoryRecord, + 0, + ), + ) + .unwrap_or_else(|| panic!("history record {seq} has metadata")) + } + + /// Seqs of the history records stored locally, ascending. + fn stored_record_seqs(&self) -> Vec { + let prefix = run_table_prefix(self.context.flow_hash, self.run(), RunTable::HistoryRecord); + let (start, end) = prefix_range(&prefix); + self.state + .rows + .range((ColumnFamily::History, start)..(ColumnFamily::History, end)) + .map(|((_, key), _)| { + u64::from_be_bytes(key[prefix.len()..].try_into().expect("eight-byte seq")) + }) + .collect() + } + + /// Event IDs of the history records stored locally. + fn stored_event_ids(&self) -> BTreeSet { + self.stored_record_seqs() + .into_iter() + .flat_map(|seq| { + let key = history_record_key(self.context.flow_hash, self.run(), seq); + let record: HistoryRecord = self + .get(ColumnFamily::History, &key) + .expect("the record is stored"); + record.events.into_iter().map(|event| event.event_id) + }) + .collect() + } + + fn segments(&self) -> Vec { + self.table_rows::(RunTable::HistorySegment) + .into_values() + .collect() + } + + fn segment_values(&self, first_seq: u64) -> BTreeMap { + let prefix = history_segment_value_prefix(self.context.flow_hash, self.run(), first_seq); + let (start, end) = prefix_range(&prefix); + self.state + .rows + .range((ColumnFamily::Flow, start)..(ColumnFamily::Flow, end)) + .map(|((_, key), value)| { + ( + ValueId::from_key_suffix(&key[prefix.len()..]).expect("value ID suffix"), + HistorySegmentValueRecord::decode(value.as_slice()).expect("row decodes"), + ) + }) + .collect() + } + + fn segment_content(&self, first_seq: u64) -> HistorySegmentContent { + let content = read_history_segment_content( + &self.state, + self.context.flow_hash, + self.run(), + first_seq, + ) + .expect("the segment content reads") + .expect("the segment exists"); + let encoded = encode_history_segment(&content).expect("the content encodes"); + assert_eq!( + decode_history_segment(&encoded.bytes).expect("the object decodes"), + content + ); + content + } + + fn table_rows( + &self, + table: RunTable, + ) -> BTreeMap, RowMessage> { + let prefix = run_table_prefix(self.context.flow_hash, self.run(), table); + let (start, end) = prefix_range(&prefix); + self.state + .rows + .range((ColumnFamily::Flow, start)..(ColumnFamily::Flow, end)) + .map(|((_, key), value)| { + ( + key[prefix.len()..].to_vec(), + RowMessage::decode(value.as_slice()).expect("row decodes"), + ) + }) + .collect() + } + + fn has_upload_pending_index_row(&self) -> bool { + self.state.rows.contains_key(&( + ColumnFamily::PartitionIndex, + history_upload_pending_index_key( + self.context.partition, + self.context.flow_hash, + self.run(), + ), + )) + } + + fn attribute_value_id(&self, key: &str) -> ValueId { + let row_key = run_named_key(self.context.flow_hash, self.run(), RunTable::Attribute, key); + let record: AttributeRecord = self + .get(ColumnFamily::Flow, &row_key) + .expect("the Attribute exists"); + match record.value.and_then(|stored| stored.kind) { + Some(stored_value::Kind::TableValue(reference)) => { + let value_id = reference.value_id.expect("table values have an ID"); + ValueId { + run_seq: value_id.run_seq, + index_in_command: value_id.index_in_command, + } + } + other => panic!("Attribute {key} is not a table value: {other:?}"), + } + } + + fn history_page(&self) -> GetHistoryEventsResponse { + get_history_events( + &self.state, + self.context.flow_hash, + self.run(), + 0, + 1000, + &[], + 0, + usize::MAX, + ) + .expect("history reads") + } + + /// Clears the event ID fields of every history record's metadata, as rows of binaries + /// older than `last_assigned_event_id` store them. + fn clear_metadata_event_ids(&mut self) { + let prefix = run_table_prefix( + self.context.flow_hash, + self.run(), + RunTable::HistoryRowMetadata, + ); + for ((column_family, key), value) in &mut self.state.rows { + let is_record_metadata = *column_family == ColumnFamily::Flow + && key.starts_with(&prefix) + && key.get(prefix.len() + 8) == Some(&(RunTable::HistoryRecord as u8)); + if !is_record_metadata { + continue; + } + let metadata = HistoryRowMetadata::decode(value.as_slice()).expect("row decodes"); + *value = HistoryRowMetadata { + last_assigned_event_id: 0, + max_event_id: 0, + ..metadata + } + .encode_to_vec(); + } + } +} + +/// Settings whose window prunes before the migration: an eight-event window pruned one event at a +/// time, a checkpoint every two commands, and segments of about two records. +fn legacy_settings() -> RunSettings { + RunSettings { + history_window_events: 8, + history_prune_chunk_events: 1, + max_retained_superseded_bytes: 256 << 20, + retention_after_close_micros: 7 * 24 * 3600 * SECOND, + wire_inline_threshold_bytes: 100, + max_value_bytes: 16 << 20, + checkpoint_row_write_bytes_threshold: 1 << 20, + checkpoint_command_threshold: 2, + request_dedup_capacity: 1000, + max_run_state_bytes: 1 << 30, + max_active_step_executions: 10_000, + max_child_flow_result_bytes: 16 << 20, + history_segment_target_bytes: 1_500, + } +} + +fn string_value(text: &str) -> Value { + Value { + kind: Some(value::Kind::StringValue(text.to_owned())), + } +} + +/// A string long enough to be stored as a table value. +fn large_text(tag: &str, length: usize) -> String { + let mut text = format!("{tag}:"); + while text.len() < length { + text.push_str(tag); + } + text +} + +fn is_ok(reply: &CommandReply) -> bool { + matches!(reply.reply, Some(command_reply::Reply::Ok(_))) +} + +fn sealed_effect_count(outcome: &ApplyOutcome) -> usize { + outcome + .effects + .iter() + .filter(|effect| matches!(effect, ApplyEffect::HistorySegmentSealed { .. })) + .count() +} + +fn requests_purge(outcome: &ApplyOutcome) -> bool { + outcome + .effects + .iter() + .any(|effect| matches!(effect, ApplyEffect::RunPurgeRequested { .. })) +} + +/// Asserts that the segments are contiguous from seq 1 and event 1 up to the open segment, that +/// U names the last uploaded one, and that each lists every value superseded inside it. +fn assert_segment_chain(harness: &Harness) { + let record = harness.run_record(); + let mut next_seq = 1; + let mut next_event_id = 1; + for segment in harness.segments() { + assert_eq!(segment.first_seq, next_seq, "{segment:?}"); + assert!(segment.first_seq <= segment.last_seq, "{segment:?}"); + assert_eq!(segment.first_event_id, next_event_id, "{segment:?}"); + assert_segment_state(harness, &segment, &record); + next_seq = segment.last_seq + 1; + next_event_id = segment.last_event_id + 1; + } + assert_eq!(record.history_open_segment_first_seq, next_seq); + assert_eq!(record.history_open_segment_first_event_id, next_event_id); +} + +fn assert_segment_state(harness: &Harness, segment: &HistorySegmentRecord, record: &RunRecord) { + let state = if segment.last_seq <= record.history_uploaded_through_seq { + HistorySegmentState::Uploaded + } else { + HistorySegmentState::Sealed + }; + assert_eq!(segment.state, state as i32, "{segment:?}"); + let values = harness.segment_values(segment.first_seq); + assert_eq!(segment.superseded_value_count, values.len() as u64); + assert!( + values + .values() + .all(|row| (segment.first_seq..=segment.last_seq).contains(&row.superseded_seq)) + ); + if state == HistorySegmentState::Sealed { + let content = harness.segment_content(segment.first_seq); + let stored: Vec = harness + .stored_record_seqs() + .into_iter() + .filter(|seq| (segment.first_seq..=segment.last_seq).contains(seq)) + .collect(); + let held: Vec = content.records.iter().map(|row| row.seq).collect(); + assert_eq!(held, stored, "a SEALED segment holds its stored records"); + assert_eq!(content.values.len(), values.len()); + } +} + +/// Asserts that a history read returns exactly the stored events and reports every other event +/// ID since event 1 as pruned. +fn assert_history_reads_match_stored_events(harness: &Harness) { + let page = harness.history_page(); + let returned: BTreeSet = page.events.iter().map(|event| event.event_id).collect(); + assert_eq!(returned, harness.stored_event_ids()); + let pruned = page + .pruned_event_id_range + .map_or(0..=0, |range| range.start_event_id..=range.end_event_id); + let next_event_id = harness.run_record().next_event_id; + for event_id in 1..next_event_id { + assert_ne!( + returned.contains(&event_id), + pruned.contains(&event_id), + "event {event_id} is returned or reported pruned, never both or neither" + ); + } +} + +/// Seqs of history records that window pruning deleted, between genesis and the newest one. +fn pruned_record_seqs(harness: &Harness) -> BTreeSet { + let stored: BTreeSet = harness.stored_record_seqs().into_iter().collect(); + (1..=harness.last_seq()) + .filter(|seq| !stored.contains(seq)) + .collect() +} + +#[test] +fn a_running_legacy_run_migrates_one_segment_per_command_until_caught_up() { + let mut harness = Harness::legacy(legacy_settings(), 30); + let pruned_before = pruned_record_seqs(&harness); + assert!( + !pruned_before.is_empty(), + "the window pruned before offload" + ); + assert!(!pruned_before.contains(&1), "FlowStart stays"); + let pruned_range = harness + .history_page() + .pruned_event_id_range + .expect("pruned events are reported"); + assert_history_reads_match_stored_events(&harness); + + harness.set_offload_mode(true); + let first = harness.apply_round("migrate0"); + let record = harness.run_record(); + assert_eq!(sealed_effect_count(&first), 1); + assert_eq!(record.history_migration_started_seq, harness.last_seq()); + assert_eq!(record.history_open_segment_bytes, 0); + assert_segment_chain(&harness); + let commands = 1 + harness.migrate_to_completion("migrate"); + assert!(commands >= 3, "the legacy history spans several segments"); + + let segments = harness.segments(); + let (last, migrated) = segments + .split_last() + .expect("the migration sealed segments"); + assert_eq!( + last.last_seq, + harness.last_seq(), + "it ends at the catching-up command" + ); + for segment in migrated { + let content = harness.segment_content(segment.first_seq); + assert!(content.records.len() >= 2, "{segment:?} holds two records"); + assert!( + content + .records + .iter() + .all(|row| !pruned_before.contains(&row.seq)), + "pruned records stay holes" + ); + } + assert_eq!( + harness.history_page().pruned_event_id_range, + Some(pruned_range), + "the migration deletes nothing, so earlier pruned events stay reported as pruned" + ); + assert_history_reads_match_stored_events(&harness); + + let after = harness.apply_round("after0"); + let record = harness.run_record(); + assert_eq!(record.history_migration_started_seq, 0); + assert_eq!( + sealed_effect_count(&after), + 0, + "the open segment follows its bytes" + ); + assert!(record.history_open_segment_bytes > 0); + + for round in 1..12 { + harness.acknowledge_every_sealed(); + harness.apply_round(&format!("after{round}")); + assert_history_reads_match_stored_events(&harness); + } + let pruned_after = harness + .history_page() + .pruned_event_id_range + .expect("pruning continues after uploads"); + assert!(pruned_after.end_event_id > pruned_range.end_event_id); +} + +#[test] +fn a_run_started_while_offload_was_off_migrates_once_it_is_on() { + let mut harness = Harness::new(legacy_settings()); + harness.set_offload_mode(true); + harness.set_offload_mode(false); + harness.start(); + for round in 0..20 { + harness.apply_round(&format!("off{round}")); + } + assert_eq!(harness.run_record().history_open_segment_first_seq, 0); + assert!(harness.segments().is_empty()); + + harness.set_offload_mode(true); + harness.migrate_to_completion("on"); + assert_eq!(harness.segments()[0].first_seq, 1); + harness.acknowledge_every_sealed(); + assert_segment_chain(&harness); + assert_history_reads_match_stored_events(&harness); +} + +#[test] +fn a_closed_legacy_run_is_not_migrated_and_its_purge_deletes_its_history() { + let mut harness = Harness::legacy(legacy_settings(), 12); + harness.stop(); + harness.set_offload_mode(true); + + let fired = harness.fire_retention(); + assert_eq!(sealed_effect_count(&fired), 0); + assert!(requests_purge(&fired), "nothing waits for an upload"); + let record = harness.run_record(); + assert_eq!(record.history_open_segment_first_seq, 0); + assert_eq!(record.history_migration_started_seq, 0); + assert!(harness.segments().is_empty()); + assert!(!harness.stored_record_seqs().is_empty()); + + let purge = harness.apply(Command::RunPurge(RunPurgeCommand {})); + assert!(is_ok(&purge.reply), "{:?}", purge.reply); + assert_every_run_row_purged(&harness); +} + +#[test] +fn a_run_that_closes_while_migrating_seals_once_more_then_keeps_the_rest_local() { + let mut harness = Harness::legacy(legacy_settings(), 30); + harness.set_offload_mode(true); + harness.apply_round("migrate0"); + let stop = harness.stop(); + assert_eq!( + sealed_effect_count(&stop), + 1, + "the closing command still migrates" + ); + let record = harness.run_record(); + assert_ne!( + record.history_migration_started_seq, 0, + "the history is too long to catch up by the close" + ); + let unsealed_from = record.history_open_segment_first_seq; + assert_segment_chain(&harness); + + let postponed = harness.fire_retention(); + assert_eq!( + sealed_effect_count(&postponed), + 0, + "a closed run stops migrating" + ); + assert!( + !requests_purge(&postponed), + "retention waits for the uploads" + ); + assert_eq!( + harness.run_record().history_open_segment_first_seq, + unsealed_from + ); + assert!( + harness + .stored_record_seqs() + .iter() + .any(|seq| *seq >= unsealed_from), + "the rest stays local" + ); + + harness.acknowledge_every_sealed(); + let fired = harness.fire_retention(); + assert_eq!(sealed_effect_count(&fired), 0); + assert!(requests_purge(&fired)); + let purge = harness.apply(Command::RunPurge(RunPurgeCommand {})); + assert!(is_ok(&purge.reply), "{:?}", purge.reply); + assert_every_run_row_purged(&harness); +} + +#[test] +fn time_travel_during_a_migration_restores_its_target_and_the_migration_continues() { + let mut harness = Harness::legacy(legacy_settings(), 30); + harness.set_offload_mode(true); + harness.apply_round("migrate0"); + assert_ne!(harness.run_record().history_migration_started_seq, 0); + let target_seq = harness.last_seq(); + assert_eq!(harness.attribute_value_id("profile").run_seq, target_seq); + harness.apply_round("migrate1"); + harness.apply_round("migrate2"); + + let travel = harness.travel_to_seq(target_seq); + let travel_seq = harness.last_seq(); + assert_eq!( + sealed_effect_count(&travel), + 1, + "the time travel migrates too" + ); + let record = harness.run_record(); + assert_eq!(record.current_attempt, 2); + assert_eq!(harness.attribute_value_id("profile").run_seq, target_seq); + assert_segment_chain(&harness); + + harness.migrate_to_completion("travelled"); + let segment = harness + .segments() + .into_iter() + .find(|segment| (segment.first_seq..=segment.last_seq).contains(&travel_seq)) + .expect("a segment holds the time travel"); + let content = harness.segment_content(segment.first_seq); + assert!(content.records.iter().any(|row| row.seq == travel_seq)); + assert!( + content + .checkpoints + .iter() + .any(|row| row.seq == travel_seq && row.attempt == 2), + "the new attempt's start checkpoint is offloaded with its command" + ); + harness.acknowledge_every_sealed(); + assert_segment_chain(&harness); +} + +#[test] +fn migration_event_ranges_do_not_need_last_assigned_event_ids() { + let mut intact = Harness::legacy(legacy_settings(), 30); + let mut older = Harness::legacy(legacy_settings(), 30); + older.clear_metadata_event_ids(); + for harness in [&mut intact, &mut older] { + harness.set_offload_mode(true); + harness.apply_round("migrate0"); + harness.migrate_to_completion("migrate"); + } + let ranges = |harness: &Harness| { + harness + .segments() + .into_iter() + .map(|segment| { + ( + segment.first_seq, + segment.last_seq, + segment.first_event_id, + segment.last_event_id, + ) + }) + .collect::>() + }; + assert!(ranges(&intact).len() >= 3); + assert_eq!(ranges(&older), ranges(&intact)); +} + +#[test] +fn commands_larger_than_the_target_still_let_the_migration_catch_up() { + let mut harness = Harness::legacy( + RunSettings { + history_window_events: 1_000, + history_segment_target_bytes: 100, + ..legacy_settings() + }, + 16, + ); + harness.set_offload_mode(true); + harness.apply_round("migrate0"); + let commands = 1 + harness.migrate_to_completion("migrate"); + assert!(commands <= 17, "each command seals at least two records"); + let segments = harness.segments(); + let (_, migrated) = segments.split_last().expect("segments"); + for segment in migrated { + assert_eq!( + harness.segment_content(segment.first_seq).records.len(), + 2, + "{segment:?} ends at its second record" + ); + } +} + +#[test] +fn the_superseded_value_limit_cuts_migration_segments() { + let keys: Vec = (0..6_000).map(|index| format!("item-{index}")).collect(); + let mut harness = Harness::new(RunSettings { + history_window_events: 1_000, + history_segment_target_bytes: 1 << 30, + ..legacy_settings() + }); + harness.start(); + for round in 0..4 { + let values: Vec<(String, String)> = keys + .iter() + .map(|key| (key.clone(), large_text(&format!("{key}-{round}"), 110))) + .collect(); + harness.set_attributes(&values); + } + let superseding_seqs: Vec = (3..=harness.last_seq()).collect(); + harness.set_offload_mode(true); + harness.apply_round("migrate0"); + harness.migrate_to_completion("migrate"); + + let segments = harness.segments(); + assert!(segments.len() >= 3, "{segments:?}"); + let first = &segments[0]; + assert_eq!( + first.last_seq, superseding_seqs[0], + "the limit cuts after 6,000 values" + ); + assert_eq!(first.superseded_value_count, 6_000); + let second = &segments[1]; + assert_eq!( + (second.first_seq, second.last_seq), + (superseding_seqs[1], superseding_seqs[2]), + "a segment ends at the seq that reaches the limit" + ); + assert_eq!(second.superseded_value_count, 12_000); + assert_segment_chain(&harness); +} + +#[test] +fn a_legacy_seq_past_the_superseded_value_limit_migrates_in_one_step() { + let keys: Vec = (0..12_000).map(|index| format!("item-{index}")).collect(); + let mut harness = Harness::new(RunSettings { + history_window_events: 1_000, + history_segment_target_bytes: 1 << 30, + ..legacy_settings() + }); + harness.start(); + for round in 0..2 { + let values: Vec<(String, String)> = keys + .iter() + .map(|key| (key.clone(), large_text(&format!("{key}-{round}"), 110))) + .collect(); + harness.set_attributes(&values); + } + let superseding_seq = harness.last_seq(); + let superseded: BTreeSet = harness + .table_rows::(RunTable::ValueGarbage) + .into_keys() + .filter(|suffix| suffix[..8] == superseding_seq.to_be_bytes()) + .map(|suffix| ValueId::from_key_suffix(&suffix[8..]).expect("value ID suffix")) + .collect(); + assert_eq!(superseded.len(), 12_000); + + harness.set_offload_mode(true); + harness.apply_round("migrate0"); + assert!( + harness + .segments() + .iter() + .all(|segment| segment.last_seq < superseding_seq), + "the first step stops before the seq past the limit" + ); + let step = harness.apply_round("migrate1"); + assert_eq!( + sealed_effect_count(&step), + 1, + "one step seals the whole seq" + ); + let segment = harness + .segments() + .into_iter() + .find(|segment| (segment.first_seq..=segment.last_seq).contains(&superseding_seq)) + .expect("a segment holds the seq"); + assert_eq!(segment.first_seq, superseding_seq); + let listed = harness.segment_values(segment.first_seq); + let listed_at_seq: BTreeSet = listed + .iter() + .filter(|(_, row)| row.superseded_seq == superseding_seq) + .map(|(value_id, _)| *value_id) + .collect(); + assert_eq!( + listed_at_seq, superseded, + "every value of the seq is listed" + ); + assert_eq!(segment.superseded_value_count, listed.len() as u64); + assert_eq!( + harness.segment_content(segment.first_seq).values.len(), + listed.len() + ); + harness.migrate_to_completion("migrate"); + assert_segment_chain(&harness); +} + +#[test] +fn turning_offload_off_ends_a_migration_and_its_rest_becomes_a_gap() { + let mut harness = Harness::legacy(legacy_settings(), 30); + harness.set_offload_mode(true); + harness.apply_round("migrate0"); + let record = harness.run_record(); + assert_ne!(record.history_migration_started_seq, 0); + let gap_first_seq = record.history_open_segment_first_seq; + harness.acknowledge_every_sealed(); + let migrated = harness.segments(); + + harness.set_offload_mode(false); + let off = harness.apply_round("off0"); + assert_eq!(sealed_effect_count(&off), 0); + let record = harness.run_record(); + assert_eq!( + record.history_migration_started_seq, 0, + "the migration ended" + ); + assert_eq!( + record.history_open_segment_first_seq, + harness.last_seq() + 1 + ); + for round in 1..14 { + harness.apply_round(&format!("off{round}")); + } + let gap_last_seq = harness.last_seq(); + let gap_last_event_id = harness.run_record().next_event_id - 1; + let gap = gap_first_seq..=gap_last_seq; + assert!( + pruned_record_seqs(&harness) + .iter() + .any(|seq| gap.contains(seq)), + "pruning while offload is off deleted rows of the gap" + ); + assert_history_reads_match_stored_events(&harness); + + harness.set_offload_mode(true); + assert_gap_is_skipped(&mut harness, &migrated, &gap, gap_last_event_id); +} + +#[test] +fn reenabling_offload_skips_the_gap_and_reads_report_its_pruned_events() { + let mut harness = Harness::new(legacy_settings()); + harness.set_offload_mode(true); + harness.start(); + for round in 0..12 { + harness.apply_round(&format!("on{round}")); + harness.acknowledge_every_sealed(); + } + let sealed = harness.segments(); + assert!(!sealed.is_empty()); + let gap_first_seq = harness.run_record().history_open_segment_first_seq; + + harness.set_offload_mode(false); + for round in 0..14 { + harness.apply_round(&format!("off{round}")); + } + let gap_last_seq = harness.last_seq(); + let gap_last_event_id = harness.run_record().next_event_id - 1; + let gap = gap_first_seq..=gap_last_seq; + assert!( + pruned_record_seqs(&harness) + .iter() + .any(|seq| gap.contains(seq)), + "pruning while offload is off deleted rows of the gap" + ); + + harness.set_offload_mode(true); + assert_gap_is_skipped(&mut harness, &sealed, &gap, gap_last_event_id); +} + +/// With offload on again after the gap, seals segments until one follows the gap, then asserts +/// that no segment holds a gap seq and that reads stay consistent while the window passes it. +fn assert_gap_is_skipped( + harness: &mut Harness, + before_gap: &[HistorySegmentRecord], + gap: &std::ops::RangeInclusive, + gap_last_event_id: i64, +) { + let mut round = 0; + while harness.segments().len() == before_gap.len() { + harness.apply_round(&format!("again{round}")); + assert_history_reads_match_stored_events(harness); + round += 1; + assert!(round < 20, "a segment seals after offload is on again"); + } + let segments = harness.segments(); + assert_eq!(&segments[..before_gap.len()], before_gap); + let reopened = &segments[before_gap.len()]; + assert_eq!(reopened.first_seq, gap.end() + 1); + assert_eq!(reopened.first_event_id, gap_last_event_id + 1); + // Content stays inside a segment's seqs, so disjoint seqs keep gap rows out of every object. + for segment in &segments { + assert!(segment.last_seq < *gap.start() || segment.first_seq > *gap.end()); + assert!( + harness + .segment_values(segment.first_seq) + .values() + .all(|row| !gap.contains(&row.superseded_seq)) + ); + } + let reopened_content = harness.segment_content(reopened.first_seq); + assert!( + reopened_content + .records + .iter() + .all(|row| !gap.contains(&row.seq)) + ); + assert!( + reopened_content + .checkpoints + .iter() + .all(|row| !gap.contains(&row.seq)) + ); + assert!( + harness + .stored_record_seqs() + .iter() + .any(|seq| gap.contains(seq)) + ); + for round in 0..30 { + harness.acknowledge_every_sealed(); + harness.apply_round(&format!("passing{round}")); + assert_history_reads_match_stored_events(harness); + } + let record = harness.run_record(); + assert!(record.history_uploaded_through_seq > *gap.end()); + assert!( + record.window_start_seq > *gap.end(), + "the window passed the gap" + ); + let pruned = harness + .history_page() + .pruned_event_id_range + .expect("pruned events are reported"); + assert!(pruned.end_event_id >= gap_last_event_id); + assert!( + harness + .stored_record_seqs() + .iter() + .all(|seq| *seq == 1 || !gap.contains(seq)), + "the window rule pruned the gap" + ); +} + +fn assert_every_run_row_purged(harness: &Harness) { + assert!( + harness + .state + .rows + .keys() + .all( + |(column_family, key)| *column_family != ColumnFamily::PartitionIndex + || key.get(2) != Some(&(IndexKind::HistoryUploadPending as u8)) + ) + ); + let run_prefix = run_prefix(harness.context.flow_hash, harness.run()); + assert!( + harness + .state + .rows + .keys() + .all( + |(column_family, key)| *column_family == ColumnFamily::PartitionIndex + || !key.starts_with(&run_prefix) + ), + "the purge deleted every row of the run" + ); +} diff --git a/server-zd/crates/dexzd-apply/tests/history_segments.rs b/server-zd/crates/dexzd-apply/tests/history_segments.rs index 9d8ab37374..22762d41b9 100644 --- a/server-zd/crates/dexzd-apply/tests/history_segments.rs +++ b/server-zd/crates/dexzd-apply/tests/history_segments.rs @@ -1088,28 +1088,6 @@ fn reenabling_offload_opens_a_fresh_segment_after_the_commands_applied_while_off harness.encode_segment(reopened.first_seq); } -#[test] -fn runs_started_before_offload_keep_no_segment_state_and_keep_their_history() { - let mut harness = Harness::new(small_window_settings(1)); - harness.start(); - harness.set_offload_mode(true); - for round in 0..12 { - let outcome = harness.set_attribute("profile", large_text(&format!("v{round}"), 300)); - assert_eq!(sealed_effect_count(&outcome), 0); - } - assert_eq!(harness.run_record().history_open_segment_first_seq, 0); - assert!(harness.segments().is_empty()); - assert_eq!( - harness.history_record_count(), - harness.last_seq() as usize, - "nothing is uploaded, so nothing is pruned" - ); - assert!( - harness.run_record().window_start_seq > 1, - "the window still moves" - ); -} - #[test] fn window_pruning_deletes_only_uploaded_rows() { let mut harness = Harness::offloading(small_window_settings(2_000)); diff --git a/server-zd/crates/dexzd-engine/src/effects/tests/history_migration_upload_tests.rs b/server-zd/crates/dexzd-engine/src/effects/tests/history_migration_upload_tests.rs new file mode 100644 index 0000000000..9ebc8cbf2b --- /dev/null +++ b/server-zd/crates/dexzd-engine/src/effects/tests/history_migration_upload_tests.rs @@ -0,0 +1,252 @@ +// Copyright (c) 2026 Super Durable, Inc. +// +// Licensed under the Sustainable Use License 1.0. +// You may not use this file except in compliance with the License. +// See the LICENSE file in the repository root. +// +// SPDX-License-Identifier: LicenseRef-Sustainable-Use-1.0 + +//! Uploads of history that a run recorded before its partition offloaded history. + +use std::time::Duration; + +use dexzd_apply::FlowStateReader; +use dexzd_apply::read_model::read_history_segment_content; +use dexzd_history_segment::{ + decode_history_segment, encode_history_segment, history_run_object_key_prefix, + history_segment_object_key, +}; +use dexzd_keyspace::row_keys::prefix_range; +use dexzd_keyspace::{ColumnFamily, RunId, RunTable, TenantId, run_table_prefix}; +use dexzd_object_store::ObjectStore; +use dexzd_protocol::dex::{AttributeWrite, SetAttributesRequest, StopFlowRequest, StopType}; +use dexzd_protocol::dexzd::v1::{ + AttributesSetCommand, FlowCommand, FlowStopCommand, HistorySegmentRecord, HistorySegmentState, + RunSettings, SetHistoryOffloadModeCommand, flow_command, +}; +use prost::Message; + +use super::{EffectHarness, is_error_reply, settings, start_request, string_value}; +use crate::flow_coordination::CommitUrgency; + +/// Rounds of 100 ms that a wait allows; paused time makes them instant. +const WAIT_ROUNDS: usize = 3_000; +const FLOW_ID: &str = "legacy-order"; + +#[tokio::test(start_paused = true)] +async fn a_run_started_while_offload_was_off_uploads_all_its_history_once_offload_is_on() { + let harness = EffectHarness::new(); + harness.rebuild_all_partitions().await; + let run_id = harness.start_flow( + start_request(FLOW_ID), + RunSettings { + history_segment_target_bytes: 1, + ..settings() + }, + ); + for round in 0..8 { + write_notes(&harness, run_id, round); + } + assert_eq!( + harness + .run_record(FLOW_ID, run_id) + .history_open_segment_first_seq, + 0 + ); + assert!(segments(&harness, run_id).is_empty()); + + set_history_offload_mode(&harness, true); + let mut commands = 0; + loop { + write_notes(&harness, run_id, 8 + commands); + commands += 1; + if harness + .run_record(FLOW_ID, run_id) + .history_migration_started_seq + == 0 + { + break; + } + assert!(commands < 30, "the migration catches up"); + } + assert!(commands >= 2, "the recorded history took several commands"); + stop_run(&harness, run_id); + + wait_until(&harness, "every segment is uploaded", || { + let record = harness.run_record(FLOW_ID, run_id); + segments(&harness, run_id).iter().all(|segment| { + segment.state == HistorySegmentState::Uploaded as i32 + && segment.last_seq <= record.history_uploaded_through_seq + }) && record.history_uploaded_through_seq == record.next_run_seq - 1 + }) + .await; + assert_uploaded_chain_from_the_first_seq(&harness, run_id).await; + harness.shutdown().await; +} + +/// Requires contiguous segments from seq 1 and event 1, each stored as one object whose content +/// and checksum match committed state, and no other object of the run. +async fn assert_uploaded_chain_from_the_first_seq(harness: &EffectHarness, run_id: RunId) { + let run = harness.run_location(FLOW_ID, run_id); + let mut next_seq = 1; + let mut next_event_id = 1; + let mut expected_keys = Vec::new(); + for segment in segments(harness, run_id) { + assert_eq!( + (segment.first_seq, segment.first_event_id), + (next_seq, next_event_id), + "{segment:?}" + ); + next_seq = segment.last_seq + 1; + next_event_id = segment.last_event_id + 1; + let key = history_segment_object_key( + TenantId::DEFAULT, + run.flow_hash, + run_id, + segment.first_seq, + segment.last_seq, + ); + let object = harness + .durable + .objects + .get(&key) + .await + .expect("the in-memory store reads") + .unwrap_or_else(|| panic!("{key} exists")) + .bytes; + let content = read_history_segment_content( + &harness.flow_effect_operations.snapshot(), + run.flow_hash, + run_id, + segment.first_seq, + ) + .expect("the segment content reads") + .expect("the segment exists"); + assert_eq!( + decode_history_segment(&object).expect("the object decodes"), + content + ); + let encoded = encode_history_segment(&content).expect("committed content encodes"); + assert_eq!(segment.content_checksum, encoded.content_checksum.to_vec()); + expected_keys.push(key); + } + let prefix = history_run_object_key_prefix(TenantId::DEFAULT, run.flow_hash, run_id); + let stored_keys: Vec = harness + .durable + .objects + .list(&prefix, None, 1000) + .await + .expect("the in-memory store lists") + .objects + .into_iter() + .map(|object| object.key) + .collect(); + assert_eq!(stored_keys, expected_keys); +} + +fn write_notes(harness: &EffectHarness, run_id: RunId, round: usize) { + commit( + harness, + run_id, + flow_command::Command::AttributesSet(AttributesSetCommand { + request: Some(SetAttributesRequest { + flow_id: FLOW_ID.to_owned(), + attributes: vec![AttributeWrite { + key: "notes".to_owned(), + value: Some(string_value(&format!("v{round} {}", "x".repeat(200)))), + ..AttributeWrite::default() + }], + request_id: format!("write-{round}"), + ..SetAttributesRequest::default() + }), + hard_delete_objects: Vec::new(), + request_fingerprint: vec![1], + }), + ); +} + +fn stop_run(harness: &EffectHarness, run_id: RunId) { + commit( + harness, + run_id, + flow_command::Command::FlowStop(FlowStopCommand { + request: Some(StopFlowRequest { + flow_id: FLOW_ID.to_owned(), + stop_type: StopType::Terminate as i32, + ..StopFlowRequest::default() + }), + }), + ); +} + +fn commit(harness: &EffectHarness, run_id: RunId, body: flow_command::Command) { + let reply = harness + .flow_effect_operations + .commit( + &EffectHarness::flow(FLOW_ID), + FlowCommand { + flow_id: FLOW_ID.to_owned(), + run_id: run_id.as_bytes().to_vec(), + command: Some(body), + ..FlowCommand::default() + }, + CommitUrgency::Prompt, + ) + .expect("the command commits"); + assert!(!is_error_reply(&reply), "the command failed: {reply:?}"); +} + +/// Commits the offload mode to every partition, as a node with `history.offloadEnabled` does. +fn set_history_offload_mode(harness: &EffectHarness, is_enabled: bool) { + for (partition, flow_id) in harness + .hasher + .partition_command_flow_ids() + .into_iter() + .enumerate() + { + let reply = harness + .flow_effect_operations + .commit( + &EffectHarness::flow(&flow_id), + FlowCommand { + flow_id: flow_id.clone(), + command: Some(flow_command::Command::SetHistoryOffloadMode( + SetHistoryOffloadModeCommand { + enabled: is_enabled, + partition: u32::try_from(partition).expect("few partitions"), + }, + )), + ..FlowCommand::default() + }, + CommitUrgency::Prompt, + ) + .expect("the mode commits"); + assert!(!is_error_reply(&reply), "the mode failed: {reply:?}"); + } +} + +fn segments(harness: &EffectHarness, run_id: RunId) -> Vec { + let run = harness.run_location(FLOW_ID, run_id); + let prefix = run_table_prefix(run.flow_hash, run.run_id, RunTable::HistorySegment); + let (start, end) = prefix_range(&prefix); + harness + .flow_effect_operations + .snapshot() + .scan(ColumnFamily::Flow, &start, &end, usize::MAX) + .expect("memory reads never fail") + .into_iter() + .map(|(_, bytes)| HistorySegmentRecord::decode(bytes.as_slice()).expect("decodes")) + .collect() +} + +/// Waits until `condition` holds, letting tasks and paused time advance. +async fn wait_until(harness: &EffectHarness, what: &str, condition: impl Fn() -> bool) { + for _ in 0..WAIT_ROUNDS { + harness.settle().await; + if condition() { + return; + } + tokio::time::sleep(Duration::from_millis(100)).await; + } + panic!("timed out waiting until {what}"); +} diff --git a/server-zd/crates/dexzd-engine/src/effects/tests/mod.rs b/server-zd/crates/dexzd-engine/src/effects/tests/mod.rs index 75e5bf27f1..7ff157fa4c 100644 --- a/server-zd/crates/dexzd-engine/src/effects/tests/mod.rs +++ b/server-zd/crates/dexzd-engine/src/effects/tests/mod.rs @@ -14,6 +14,7 @@ mod deadline_tests; mod dispatch_tests; mod handout_tests; mod hard_delete_tests; +mod history_migration_upload_tests; mod history_upload_tests; mod keyword_index_tests; mod outbox_and_purge_tests; diff --git a/server-zd/crates/dexzd-engine/tests/format_compat/golden_replay.rs b/server-zd/crates/dexzd-engine/tests/format_compat/golden_replay.rs index 8cc025b76a..b416639eb9 100644 --- a/server-zd/crates/dexzd-engine/tests/format_compat/golden_replay.rs +++ b/server-zd/crates/dexzd-engine/tests/format_compat/golden_replay.rs @@ -29,7 +29,7 @@ use crate::testdata_tree::{ }; /// Digest of every file of each version, README included. Regenerate v1 before 1.0; afterward versions are immutable: add `v`. -const PINNED_VERSION_DIGESTS: &[(&str, &str)] = &[("v1", "7f19d73154a70a9ebc4b50879ece2457")]; +const PINNED_VERSION_DIGESTS: &[(&str, &str)] = &[("v1", "e96ae4da37f20885d85eeba5b4f30682")]; /// The restoring node starts this long after the last recorded command. pub(crate) const RESTART_DELAY_MICROS: i64 = 60 * 60 * 1_000_000; diff --git a/server-zd/crates/dexzd-engine/tests/format_compat/workload.rs b/server-zd/crates/dexzd-engine/tests/format_compat/workload.rs index 9263e0d501..e460d2b8cd 100644 --- a/server-zd/crates/dexzd-engine/tests/format_compat/workload.rs +++ b/server-zd/crates/dexzd-engine/tests/format_compat/workload.rs @@ -135,6 +135,7 @@ impl Workload<'_> { .await; self.set_history_offload_modes(&[PartitionId(0)], false) .await; + self.legacy_runs_record_history_without_offload().await; } /// `history-flow`, in partition 1, which offloads history: a replaced Attribute supersedes a @@ -2030,6 +2031,8 @@ pub(crate) async fn put_new_object(store: &InMemoryObjectStore, key: &str, body: ); } +mod legacy_history; + mod load_blobs_purge_tests { use std::sync::Arc; diff --git a/server-zd/crates/dexzd-engine/tests/format_compat/workload/legacy_history.rs b/server-zd/crates/dexzd-engine/tests/format_compat/workload/legacy_history.rs new file mode 100644 index 0000000000..c252b94505 --- /dev/null +++ b/server-zd/crates/dexzd-engine/tests/format_compat/workload/legacy_history.rs @@ -0,0 +1,629 @@ +// Copyright (c) 2026 Super Durable, Inc. +// +// Licensed under the Sustainable Use License 1.0. +// You may not use this file except in compliance with the License. +// See the LICENSE file in the repository root. +// +// SPDX-License-Identifier: LicenseRef-Sustainable-Use-1.0 + +//! Runs recorded while their partition did not offload history, and their migration once a +//! node restored from the recording turns offload on (plan section 5.17). + +use std::collections::{BTreeMap, BTreeSet}; +use std::sync::Arc; + +use dexzd_apply::read_model::{get_history_events, resolve_run_id}; +use dexzd_keyspace::{ValueId, run_prefix}; +use dexzd_protocol::dex::{FlowStatus, GetHistoryEventsResponse}; +use dexzd_protocol::dexzd::v1::{AttributeRecord, HistoryRecord, RunRecord, stored_value}; + +use super::*; +use crate::compat_node::{engine_node_config, start_engine_node}; +use crate::format_decoding::decode_objects; +use crate::golden_replay::RESTART_DELAY_MICROS; +use crate::testdata_tree::{OBJECTS_DIRECTORY, load_objects, read_files, testdata_versions}; + +/// Never offloads history in the recording. +const NEVER_OFFLOADED_PARTITION: PartitionId = PartitionId(2); +/// Turns offload on at the end of the recording, while `legacy-migrating` runs. +const LATE_OFFLOADED_PARTITION: PartitionId = PartitionId(3); +const RUNNING_FLOW_PREFIX: &str = "legacy-running"; +const CLOSED_FLOW_PREFIX: &str = "legacy-closed"; +const MIGRATING_FLOW_PREFIX: &str = "legacy-migrating"; +/// Enough commands for the eight-event window to prune and for several migration segments. +const LEGACY_ROUNDS: usize = 30; +/// Most commands a migration may take in these tests. +const MAX_MIGRATION_COMMANDS: usize = 60; +const RESTORE_SEED: u64 = 0x5eed_2026_1107; + +impl Workload<'_> { + /// `legacy-running`, `legacy-closed`, and `legacy-migrating`: runs with a small window + /// and small segments that record history while their partition does not offload. + pub(super) async fn legacy_runs_record_history_without_offload(&self) { + let running = self + .start_legacy_run( + RUNNING_FLOW_PREFIX, + NEVER_OFFLOADED_PARTITION, + legacy_run_settings(), + ) + .await; + for round in 0..LEGACY_ROUNDS { + self.apply_legacy_round(&running, round).await; + } + let closed = self + .start_legacy_run( + CLOSED_FLOW_PREFIX, + NEVER_OFFLOADED_PARTITION, + RunSettings { + retention_after_close_micros: 60 * SECOND_MICROS, + ..legacy_run_settings() + }, + ) + .await; + for round in 0..4 { + self.apply_legacy_round(&closed, round).await; + } + self.commit(&closed, cancel_command(&closed)).await; + let migrating = self + .start_legacy_run( + MIGRATING_FLOW_PREFIX, + LATE_OFFLOADED_PARTITION, + legacy_run_settings(), + ) + .await; + for round in 0..LEGACY_ROUNDS { + self.apply_legacy_round(&migrating, round).await; + } + self.set_history_offload_modes(&[LATE_OFFLOADED_PARTITION], true) + .await; + for round in LEGACY_ROUNDS..LEGACY_ROUNDS + 2 { + let outcome = self.apply_legacy_round(&migrating, round).await; + assert_eq!(sealed_segment_count(&outcome), 1, "the run migrates"); + } + } + + async fn start_legacy_run( + &self, + prefix: &str, + partition: PartitionId, + settings: RunSettings, + ) -> FlowRun { + let flow_id = self.legacy_flow_id(prefix, partition); + self.start_flow( + StartFlowRequest { + flow_id: flow_id.clone(), + flow_type: "LegacyFlow".to_owned(), + request_id: format!("start-{flow_id}"), + ..StartFlowRequest::default() + }, + settings, + ) + .await + } + + fn legacy_flow_id(&self, prefix: &str, partition: PartitionId) -> String { + (0..) + .map(|index| format!("{prefix}-{index}")) + .find(|flow_id| self.context(flow_id).partition == partition) + .expect("some Flow ID hashes to the partition") + } + + /// Replaces `profile` in even rounds and publishes to `updates` in odd rounds, so records + /// alternate between superseding a value and recording an event. + async fn apply_legacy_round(&self, run: &FlowRun, round: usize) -> ApplyOutcome { + let text = format!("{LONG_PROFILE_TEXT} Round {round}."); + if round.is_multiple_of(2) { + let request_id = format!("{}-write-{round}", run.flow_id); + let request = SetAttributesRequest { + flow_id: run.flow_id.clone(), + attributes: vec![attribute("profile", string_value(&text))], + request_id: request_id.clone(), + ..SetAttributesRequest::default() + }; + self.commit( + run, + Command::AttributesSet(AttributesSetCommand { + request_fingerprint: request_fingerprint(&request), + request: Some(request), + hard_delete_objects: Vec::new(), + }), + ) + .await + } else { + let message_id = format!("{}-message-{round}", run.flow_id); + self.commit( + run, + publish_command(vec![channel_message("updates", &message_id, &text)]), + ) + .await + } + } + + async fn legacy_run(&self, prefix: &str, partition: PartitionId) -> FlowRun { + let flow_id = self.legacy_flow_id(prefix, partition); + let flow_hash = self.context(&flow_id).flow_hash; + let view = self + .node + .committer() + .committed_view() + .expect("open a committed view"); + let lookup_flow_id = flow_id.clone(); + let run_id = tokio::task::spawn_blocking(move || { + resolve_run_id(&view, flow_hash, TenantId::DEFAULT, &lookup_flow_id, "") + .expect("the Flow directory reads") + .expect("the recorded Flow has a run") + }) + .await + .expect("the read task completes"); + FlowRun { flow_id, run_id } + } + + async fn run_record_of(&self, run: &FlowRun) -> RunRecord { + let mut records = self.run_rows::(run, RunTable::RunRecord).await; + assert_eq!(records.len(), 1, "{} has one run record", run.flow_id); + records.remove(0) + } + + async fn history_segments(&self, run: &FlowRun) -> Vec { + self.run_rows::(run, RunTable::HistorySegment) + .await + } + + /// Committed history records of the run in the History column family, by seq. + async fn stored_history_records(&self, run: &FlowRun) -> BTreeMap { + let flow_hash = self.context(&run.flow_id).flow_hash; + let prefix = run_table_prefix(flow_hash, run.run_id, RunTable::HistoryRecord); + let (start, end) = prefix_range(&prefix); + let view = self + .node + .committer() + .committed_view() + .expect("open a committed view"); + let rows = tokio::task::spawn_blocking(move || { + view.scan(ColumnFamily::History, &start, &end, usize::MAX) + .expect("scan committed history") + }) + .await + .expect("the scan task completes"); + rows.into_iter() + .map(|(_, value)| { + let record = HistoryRecord::decode(value.as_slice()).expect("records decode"); + (record.seq, record) + }) + .collect() + } + + async fn history_events(&self, run: &FlowRun) -> GetHistoryEventsResponse { + let flow_hash = self.context(&run.flow_id).flow_hash; + let run_id = run.run_id; + let view = self + .node + .committer() + .committed_view() + .expect("open a committed view"); + tokio::task::spawn_blocking(move || { + get_history_events(&view, flow_hash, run_id, 0, 1000, &[], 0, usize::MAX) + .expect("history reads") + }) + .await + .expect("the read task completes") + } + + async fn segment_content(&self, run: &FlowRun, first_seq: u64) -> HistorySegmentContent { + let flow_hash = self.context(&run.flow_id).flow_hash; + let run_id = run.run_id; + let view = self + .node + .committer() + .committed_view() + .expect("open a committed view"); + tokio::task::spawn_blocking(move || { + read_history_segment_content(&view, flow_hash, run_id, first_seq) + .expect("the segment content reads") + .expect("the segment exists") + }) + .await + .expect("the read task completes") + } + + /// Acknowledges every SEALED segment oldest first, as the uploader would. + async fn acknowledge_sealed_segments(&self, run: &FlowRun) { + loop { + let uploaded_through_seq = self.run_record_of(run).await.history_uploaded_through_seq; + let Some(segment) = self + .history_segments(run) + .await + .into_iter() + .find(|segment| segment.last_seq > uploaded_through_seq) + else { + return; + }; + let content = self.segment_content(run, segment.first_seq).await; + let encoded = encode_history_segment(&content).expect("the segment encodes"); + self.commit( + run, + Command::HistorySegmentUploaded(HistorySegmentUploadedCommand { + first_seq: segment.first_seq, + last_seq: segment.last_seq, + content_checksum: encoded.content_checksum.to_vec(), + object_size_bytes: encoded.bytes.len() as u64, + }), + ) + .await; + } + } + + /// Applies rounds from `first_round` until the migration caught up, requiring one sealed + /// segment per command. Returns the commands it took. + async fn migrate_to_completion(&self, run: &FlowRun, first_round: usize) -> usize { + for command_count in 1..=MAX_MIGRATION_COMMANDS { + let outcome = self + .apply_legacy_round(run, first_round + command_count) + .await; + assert_eq!(sealed_segment_count(&outcome), 1, "one segment per command"); + let record = self.run_record_of(run).await; + assert_segment_chain(&self.history_segments(run).await, &record); + if record.history_migration_started_seq == 0 { + return command_count; + } + } + panic!("the migration did not catch up within {MAX_MIGRATION_COMMANDS} commands"); + } +} + +fn legacy_run_settings() -> RunSettings { + RunSettings { + history_window_events: 8, + history_prune_chunk_events: 1, + checkpoint_command_threshold: 2, + history_segment_target_bytes: 1_500, + ..workload_run_settings() + } +} + +fn cancel_command(run: &FlowRun) -> Command { + Command::FlowStop(FlowStopCommand { + request: Some(StopFlowRequest { + flow_id: run.flow_id.clone(), + reason: "legacy run closed".to_owned(), + stop_type: StopType::Cancel as i32, + ..StopFlowRequest::default() + }), + }) +} + +fn sealed_segment_count(outcome: &ApplyOutcome) -> usize { + outcome + .effects + .iter() + .filter(|effect| matches!(effect, ApplyEffect::HistorySegmentSealed { .. })) + .count() +} + +/// Requires segments contiguous from seq 1 and event 1, followed by the open segment. +fn assert_segment_chain(segments: &[HistorySegmentRecord], record: &RunRecord) { + let mut next_seq = 1; + let mut next_event_id = 1; + for segment in segments { + assert_eq!( + (segment.first_seq, segment.first_event_id), + (next_seq, next_event_id), + "{segment:?}" + ); + assert!(segment.first_seq <= segment.last_seq, "{segment:?}"); + next_seq = segment.last_seq + 1; + next_event_id = segment.last_event_id + 1; + } + assert_eq!( + ( + record.history_open_segment_first_seq, + record.history_open_segment_first_event_id + ), + (next_seq, next_event_id) + ); +} + +fn profile_value_seq(attributes: &[AttributeRecord]) -> u64 { + let [profile] = attributes else { + panic!("the legacy run holds only its profile: {attributes:?}"); + }; + match profile + .value + .as_ref() + .and_then(|stored| stored.kind.as_ref()) + { + Some(stored_value::Kind::TableValue(reference)) => { + let value_id = reference + .value_id + .as_ref() + .expect("table values have an ID"); + ValueId { + run_seq: value_id.run_seq, + index_in_command: value_id.index_in_command, + } + .run_seq + } + other => panic!("the profile is not a table value: {other:?}"), + } +} + +/// A node restored from the newest recorded testdata version, driven with workload helpers. +struct RestoredNode { + node: EngineNode, + clock: Arc, + ids: Arc, + object_store: Arc, + hasher: PartitionHasher, +} + +impl RestoredNode { + async fn start() -> Self { + let version = testdata_versions().pop().expect("testdata holds a version"); + let decoded = decode_objects(&read_files(&version.directory.join(OBJECTS_DIRECTORY))); + let object_store = Arc::new(load_objects(&version.directory).await); + let recorded_node = decoded + .manifest + .nodes + .values() + .next() + .expect("the manifest names the recording node"); + let last_command_micros = decoded + .commands + .iter() + .map(|command| command.proposed_at_micros) + .max() + .expect("the log holds commands"); + let clock = Arc::new(ScriptedClock::starting_at( + last_command_micros + RESTART_DELAY_MICROS, + )); + let ids = Arc::new(SeededIdSource::new(RESTORE_SEED, clock.clone())); + let config = engine_node_config( + &recorded_node.node_id, + recorded_node.incarnation, + decoded.cluster.partition_count, + ); + let node = + start_engine_node(&config, object_store.clone(), clock.clone(), ids.clone()).await; + let hasher = PartitionHasher::new( + &decoded.cluster.partition_hash_key, + decoded.cluster.partition_count, + ) + .expect("the recorded cluster has a valid partition hasher"); + Self { + node, + clock, + ids, + object_store, + hasher, + } + } + + fn workload(&self) -> Workload<'_> { + Workload { + node: &self.node, + clock: &self.clock, + ids: &self.ids, + object_store: &self.object_store, + hasher: &self.hasher, + } + } + + async fn shutdown(self) { + self.node + .shutdown() + .await + .expect("the node releases its lease"); + } +} + +#[tokio::test(start_paused = true)] +async fn a_recorded_running_run_migrates_one_segment_per_command_once_offload_is_on() { + let restored = RestoredNode::start().await; + let workload = restored.workload(); + let run = workload + .legacy_run(RUNNING_FLOW_PREFIX, NEVER_OFFLOADED_PARTITION) + .await; + let record = workload.run_record_of(&run).await; + assert_eq!(record.status, FlowStatus::Running as i32); + assert_eq!(record.history_open_segment_first_seq, 0, "no segment state"); + let stored_before = workload.stored_history_records(&run).await; + let pruned_seqs: BTreeSet = (1..record.next_run_seq) + .filter(|seq| !stored_before.contains_key(seq)) + .collect(); + assert!(!pruned_seqs.is_empty(), "the window pruned before offload"); + let before = workload.history_events(&run).await; + let pruned_range = before + .pruned_event_id_range + .expect("pruned events are reported"); + + workload + .set_history_offload_modes(&[NEVER_OFFLOADED_PARTITION], true) + .await; + let first = workload.apply_legacy_round(&run, LEGACY_ROUNDS).await; + assert_eq!(sealed_segment_count(&first), 1); + let record = workload.run_record_of(&run).await; + assert_eq!( + record.history_migration_started_seq, + record.next_run_seq - 1 + ); + let commands = 1 + workload.migrate_to_completion(&run, LEGACY_ROUNDS).await; + assert!(commands >= 3, "the recorded history spans several segments"); + + let segments = workload.history_segments(&run).await; + let stored = workload.stored_history_records(&run).await; + for segment in &segments { + let content = workload.segment_content(&run, segment.first_seq).await; + let held: Vec = content.records.iter().map(|row| row.seq).collect(); + let expected: Vec = stored + .range(segment.first_seq..=segment.last_seq) + .map(|(seq, _)| *seq) + .collect(); + assert_eq!(held, expected, "{segment:?} holds the stored records"); + assert!(held.iter().all(|seq| !pruned_seqs.contains(seq))); + } + assert_eq!(segments[0].first_seq, 1); + workload.acknowledge_sealed_segments(&run).await; + + let after = workload.history_events(&run).await; + assert_eq!( + after.pruned_event_id_range, + Some(pruned_range), + "events pruned before the migration are still reported as pruned" + ); + let before_ids: Vec = before.events.iter().map(|event| event.event_id).collect(); + let after_ids: Vec = after.events.iter().map(|event| event.event_id).collect(); + assert!( + after_ids.starts_with(&before_ids), + "the migration keeps every event" + ); + assert!( + after_ids + .iter() + .all(|event_id| *event_id == 1 || *event_id > pruned_range.end_event_id) + ); + restored.shutdown().await; +} + +#[tokio::test(start_paused = true)] +async fn a_recorded_closed_run_is_not_migrated_and_its_purge_deletes_its_local_history() { + let restored = RestoredNode::start().await; + let workload = restored.workload(); + let run = workload + .legacy_run(CLOSED_FLOW_PREFIX, NEVER_OFFLOADED_PARTITION) + .await; + let record = workload.run_record_of(&run).await; + assert_eq!(record.status, FlowStatus::Canceled as i32); + assert!(!workload.stored_history_records(&run).await.is_empty()); + + workload + .set_history_offload_modes(&[NEVER_OFFLOADED_PARTITION], true) + .await; + let fired = workload.fire_due_timers(&run, SECOND_MICROS).await; + assert_eq!(sealed_segment_count(&fired), 0); + assert!( + fired + .effects + .iter() + .any(|effect| matches!(effect, ApplyEffect::RunPurgeRequested { .. })), + "retention waits for no upload" + ); + let record = workload.run_record_of(&run).await; + assert_eq!(record.history_open_segment_first_seq, 0); + assert!(workload.history_segments(&run).await.is_empty()); + + workload + .commit(&run, Command::RunPurge(RunPurgeCommand {})) + .await; + let flow_hash = workload.context(&run.flow_id).flow_hash; + let (start, end) = prefix_range(&run_prefix(flow_hash, run.run_id)); + let view = restored + .node + .committer() + .committed_view() + .expect("open a committed view"); + let remaining = tokio::task::spawn_blocking(move || { + [ + ColumnFamily::Flow, + ColumnFamily::History, + ColumnFamily::Value, + ] + .into_iter() + .map(|column_family| { + view.scan(column_family, &start, &end, usize::MAX) + .expect("scan committed rows") + .len() + }) + .sum::() + }) + .await + .expect("the scan task completes"); + assert_eq!(remaining, 0, "the purge deleted the run's local history"); + restored.shutdown().await; +} + +#[tokio::test(start_paused = true)] +async fn time_travel_during_the_migration_of_a_recorded_run_is_correct() { + let restored = RestoredNode::start().await; + let workload = restored.workload(); + let run = workload + .legacy_run(RUNNING_FLOW_PREFIX, NEVER_OFFLOADED_PARTITION) + .await; + workload + .set_history_offload_modes(&[NEVER_OFFLOADED_PARTITION], true) + .await; + workload.apply_legacy_round(&run, LEGACY_ROUNDS).await; + let record = workload.run_record_of(&run).await; + assert_ne!(record.history_migration_started_seq, 0); + let target_seq = record.next_run_seq - 1; + let attributes = workload + .run_rows::(&run, RunTable::Attribute) + .await; + assert_eq!(profile_value_seq(&attributes), target_seq); + let target_timestamp_micros = + workload.stored_history_records(&run).await[&target_seq].timestamp_micros; + workload.apply_legacy_round(&run, LEGACY_ROUNDS + 1).await; + workload.apply_legacy_round(&run, LEGACY_ROUNDS + 2).await; + + let travel = workload + .commit( + &run, + Command::TimeTravel(TimeTravelCommand { + should_restore_running_as_pending: false, + request: Some(ResetFlowRequest { + flow_id: run.flow_id.clone(), + reset_type: FlowResetType::HistoryEventTime as i32, + reason: "travel during the migration".to_owned(), + skip_writes_reapply: true, + history_event_time: (i128::from(target_timestamp_micros) * 1000).to_string(), + ..ResetFlowRequest::default() + }), + }), + ) + .await; + assert!( + matches!( + travel.reply.reply, + Some(command_reply::Reply::TimeTravel(ref reply)) if reply.run_attempt == 2 + ), + "{:?}", + travel.reply + ); + assert_eq!( + sealed_segment_count(&travel), + 1, + "the time travel migrates too" + ); + let travel_seq = workload.run_record_of(&run).await.next_run_seq - 1; + let attributes = workload + .run_rows::(&run, RunTable::Attribute) + .await; + assert_eq!( + profile_value_seq(&attributes), + target_seq, + "the travel restored the profile" + ); + + workload + .migrate_to_completion(&run, LEGACY_ROUNDS + 2) + .await; + let segment = workload + .history_segments(&run) + .await + .into_iter() + .find(|segment| (segment.first_seq..=segment.last_seq).contains(&travel_seq)) + .expect("a segment holds the time travel"); + let content = workload.segment_content(&run, segment.first_seq).await; + assert!(content.records.iter().any(|row| row.seq == travel_seq)); + assert!( + content + .checkpoints + .iter() + .any(|row| row.seq == travel_seq && row.attempt == 2), + "the second attempt's start checkpoint is offloaded with the time travel" + ); + workload.acknowledge_sealed_segments(&run).await; + let record = workload.run_record_of(&run).await; + assert_eq!(record.current_attempt, 2); + assert_segment_chain(&workload.history_segments(&run).await, &record); + restored.shutdown().await; +} diff --git a/server-zd/crates/dexzd-simulation/src/apply_checker.rs b/server-zd/crates/dexzd-simulation/src/apply_checker.rs index 7a8d2a1340..b173d0f732 100644 --- a/server-zd/crates/dexzd-simulation/src/apply_checker.rs +++ b/server-zd/crates/dexzd-simulation/src/apply_checker.rs @@ -22,10 +22,11 @@ use prost::Message; use crate::invalid_client_inputs::{check_invalid_client_input_rejected, invalid_client_input}; use crate::invariants::{ - CheckerMemory, CloseBarrierCounts, InvariantViolation, OffloadGapsByRun, WindowPointState, - check_close_barrier, check_deletions_were_uploaded, check_flow, check_fresh_snapshot, - check_open_segment_transitions, check_outbox_writes, check_sealed_segment_encodings, - check_window_value_content, remember_ack, remember_window_points, + CheckerMemory, CloseBarrierCounts, HistoryMigrationCounts, InvariantViolation, + OffloadGapsByRun, WindowPointState, check_close_barrier, check_deletions_were_uploaded, + check_flow, check_fresh_snapshot, check_history_migration, check_open_segment_transitions, + check_outbox_writes, check_sealed_segment_encodings, check_window_value_content, remember_ack, + remember_window_points, }; use crate::memory_state::MemoryFlowState; use crate::state_view::{FlowView, RunSelection}; @@ -51,6 +52,7 @@ pub struct ApplyChecker { history_offload_off_time_travels: usize, /// Every run an admitted transactional RPC ever held, for checks of recovered stores. ever_rpc_held_runs: BTreeSet, + history_migration_counts: HistoryMigrationCounts, } /// When the checker retained one window point, in applied command counts. @@ -93,6 +95,7 @@ impl ApplyChecker { close_barrier_counts: CloseBarrierCounts::default(), history_offload_off_time_travels: 0, ever_rpc_held_runs: BTreeSet::new(), + history_migration_counts: HistoryMigrationCounts::default(), } } @@ -189,6 +192,9 @@ impl ApplyChecker { &mut self.memory.rpc_held_runs, )?; self.close_barrier_counts.merge(&close_barrier_counts); + let migration_counts = + check_history_migration(&command.flow_id, &outcome, &previous_records, &view)?; + self.history_migration_counts.merge(&migration_counts); self.ever_rpc_held_runs .extend(self.memory.rpc_held_runs.iter().copied()); let pruned_points = remember_window_points( @@ -237,6 +243,11 @@ impl ApplyChecker { self.close_barrier_counts } + /// History migrations that started, caught up, or ended because offload turned off. + pub fn history_migration_counts(&self) -> HistoryMigrationCounts { + self.history_migration_counts + } + /// Every run whose pending close an admitted transactional RPC held at some point. pub(crate) fn ever_rpc_held_runs(&self) -> &BTreeSet { &self.ever_rpc_held_runs diff --git a/server-zd/crates/dexzd-simulation/src/apply_simulation.rs b/server-zd/crates/dexzd-simulation/src/apply_simulation.rs index 6468fb6abc..85e8e265c3 100644 --- a/server-zd/crates/dexzd-simulation/src/apply_simulation.rs +++ b/server-zd/crates/dexzd-simulation/src/apply_simulation.rs @@ -15,7 +15,7 @@ use prost::Message; use xxhash_rust::xxh3::xxh3_128; use crate::apply_checker::ApplyChecker; -use crate::invariants::{CloseBarrierCounts, InvariantViolation}; +use crate::invariants::{CloseBarrierCounts, HistoryMigrationCounts, InvariantViolation}; use crate::model::{ClosedErasePhase, CommandGenerator}; /// Replays allowed while shrinking one failure. @@ -49,6 +49,8 @@ pub struct SimulationSummary { pub history_segments_sealed: usize, /// TimeTravel commands applied to a run with segment state while offload was off. pub history_offload_off_time_travels: usize, + /// History migrations of runs that predate their segment state. + pub history_migrations: HistoryMigrationCounts, } impl SimulationSummary { @@ -66,6 +68,7 @@ impl SimulationSummary { self.permission_mapping_commands += other.permission_mapping_commands; self.history_segments_sealed += other.history_segments_sealed; self.history_offload_off_time_travels += other.history_offload_off_time_travels; + self.history_migrations.merge(&other.history_migrations); for (kind, count) in &other.applied_by_kind { *self.applied_by_kind.entry(kind).or_default() += count; } @@ -233,6 +236,7 @@ pub fn run_seed( summary.size_rejections = checker.size_rejections().clone(); summary.close_barrier = checker.close_barrier_counts(); summary.history_offload_off_time_travels = checker.history_offload_off_time_travels(); + summary.history_migrations = checker.history_migration_counts(); summary.final_row_count = checker.state().row_count(); Ok(summary) } diff --git a/server-zd/crates/dexzd-simulation/src/invariants/history_migration_invariants.rs b/server-zd/crates/dexzd-simulation/src/invariants/history_migration_invariants.rs new file mode 100644 index 0000000000..459d68b459 --- /dev/null +++ b/server-zd/crates/dexzd-simulation/src/invariants/history_migration_invariants.rs @@ -0,0 +1,154 @@ +// Copyright (c) 2026 Super Durable, Inc. +// +// Licensed under the Sustainable Use License 1.0. +// You may not use this file except in compliance with the License. +// See the LICENSE file in the repository root. +// +// SPDX-License-Identifier: LicenseRef-Sustainable-Use-1.0 + +//! Migration of history that runs recorded before they had segment state (plan section 5.17): +//! bounded work per command, and segment state that matches the migration. + +use std::collections::BTreeMap; + +use dexzd_apply::{ApplyEffect, ApplyOutcome}; +use dexzd_keyspace::RunId; +use dexzd_protocol::dex::FlowStatus; +use dexzd_protocol::dexzd::v1::RunRecord; + +use crate::invariants::InvariantViolation; +use crate::state_view::{FlowView, RunView}; + +/// Migrations that started, caught up, and ended because offload turned off. +#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)] +pub struct HistoryMigrationCounts { + /// Runs without segment state that started migrating. + pub started: usize, + /// Migrations whose last segment reached the command that sealed it. + pub completed: usize, + /// Migrations that ended because their partition stopped offloading. + pub abandoned: usize, +} + +impl HistoryMigrationCounts { + /// Adds another count to this one. + pub fn merge(&mut self, other: &Self) { + self.started += other.started; + self.completed += other.completed; + self.abandoned += other.abandoned; + } +} + +/// Returns whether the command at `seq` must start or continue the run's migration: it took a +/// run seq while offload was on and left a run running whose history predates its segment state. +pub(crate) fn must_migrate( + previous: &RunRecord, + current: &RunRecord, + view: &FlowView, + seq: u64, +) -> bool { + let has_older_history = + previous.history_open_segment_first_seq == 0 || previous.history_migration_started_seq != 0; + view.is_history_offload_enabled + && has_older_history + && current.next_run_seq == seq + 1 + && current.status == FlowStatus::Running as i32 + && !current.is_purging +} + +/// Returns whether the command at `seq` started or continued the run's migration while offload +/// was on: it sealed one segment at the open segment's start, and the open segment moved right +/// after that segment, which holds two history records, or after the command once it caught up. +pub(crate) fn is_migration_step( + previous: &RunRecord, + current: &RunRecord, + run: &RunView, + view: &FlowView, + seq: u64, +) -> bool { + let (first_seq, first_event_id) = if previous.history_open_segment_first_seq == 0 { + ( + run.attempts.get(&1).map_or(1, |attempt| attempt.start_seq), + 1, + ) + } else if previous.history_migration_started_seq != 0 { + ( + previous.history_open_segment_first_seq, + previous.history_open_segment_first_event_id, + ) + } else { + return false; + }; + let Some(segment) = run.history_segments.get(&first_seq) else { + return false; + }; + let has_caught_up = segment.last_seq == seq + && current.history_migration_started_seq == 0 + && current.history_open_segment_first_seq == seq + 1 + && current.history_open_segment_first_event_id == current.next_event_id.max(1); + let segment_records = run + .history + .iter() + .filter(|(record_seq, _)| (segment.first_seq..=segment.last_seq).contains(record_seq)) + .count(); + let has_moved_after_segment = segment.last_seq < seq + && segment_records >= 2 + && current.history_migration_started_seq != 0 + && current.history_open_segment_first_seq == segment.last_seq + 1 + && current.history_open_segment_first_event_id == segment.last_event_id + 1; + view.is_history_offload_enabled + && segment.first_event_id == first_event_id + && current.history_open_segment_bytes == 0 + && current.history_open_segment_superseded_value_count == 0 + && (has_caught_up || has_moved_after_segment) +} + +/// Requires at most one sealed segment per run and command (H11), and an open segment without +/// tracked bytes while a run migrates (H12). Returns the migrations the command moved. +pub(crate) fn check_history_migration( + flow_id: &str, + outcome: &ApplyOutcome, + previous_records: &BTreeMap, + view: &FlowView, +) -> Result { + let mut seals = BTreeMap::::new(); + for effect in &outcome.effects { + if let ApplyEffect::HistorySegmentSealed { run_id } = effect { + *seals.entry(*run_id).or_default() += 1; + } + } + if let Some((run_id, count)) = seals.iter().find(|(_, count)| **count > 1) { + return Err(InvariantViolation::new( + "H11-one-seal-per-command", + format!("Flow {flow_id} run {run_id} sealed {count} segments in one command"), + )); + } + let mut counts = HistoryMigrationCounts::default(); + for (run_id, previous) in previous_records { + let Some(record) = view.runs.get(run_id).and_then(|run| run.record.as_ref()) else { + continue; + }; + if record.history_migration_started_seq != 0 + && (record.history_open_segment_bytes != 0 + || record.history_open_segment_superseded_value_count != 0) + { + return Err(InvariantViolation::new( + "H12-migration-segment-state", + format!("Flow {flow_id} run {run_id} tracks open segment bytes while migrating"), + )); + } + let was_migrating = previous.history_migration_started_seq != 0; + let is_migrating = record.history_migration_started_seq != 0; + let has_started = previous.history_open_segment_first_seq == 0 + && record.history_open_segment_first_seq != 0; + counts.started += usize::from(has_started); + if (was_migrating || has_started) && !is_migrating { + if view.is_history_offload_enabled { + counts.completed += 1; + } else { + counts.abandoned += 1; + } + } + } + Ok(counts) +} diff --git a/server-zd/crates/dexzd-simulation/src/invariants/history_segment_invariants.rs b/server-zd/crates/dexzd-simulation/src/invariants/history_segment_invariants.rs index 39f7ec163f..9ea6d73fda 100644 --- a/server-zd/crates/dexzd-simulation/src/invariants/history_segment_invariants.rs +++ b/server-zd/crates/dexzd-simulation/src/invariants/history_segment_invariants.rs @@ -20,6 +20,7 @@ use dexzd_protocol::dexzd::v1::{ FlowCommand, HistorySegmentState, RunRecord, flow_command::Command, }; +use crate::invariants::history_migration_invariants::{is_migration_step, must_migrate}; use crate::invariants::{InvariantViolation, RunContext}; use crate::memory_state::MemoryFlowState; use crate::state_view::{FlowView, RunView}; @@ -75,17 +76,21 @@ pub(crate) fn check_open_segment_transitions( record.history_open_segment_superseded_value_count, ) }; + let seq = previous.next_run_seq.max(1); + let is_migration_required = must_migrate(previous, current, view, seq); if previous.history_open_segment_first_seq == 0 { - if current.history_open_segment_first_seq != 0 { + if (current.history_open_segment_first_seq != 0 || is_migration_required) + && !is_migration_step(previous, current, run, view, seq) + { return Err(violation( "H9-open-segment-transition", &context, - "an existing run without segment state gained it", + "an existing run without segment state gained it without migrating, or a \ + running one did not start migrating", )); } continue; } - let seq = previous.next_run_seq.max(1); if current.next_run_seq != seq + 1 { if open(current) != open(previous) { return Err(violation( @@ -126,11 +131,20 @@ pub(crate) fn check_open_segment_transitions( let is_kept = current.history_open_segment_first_seq == first_seq && current.history_open_segment_first_event_id == previous.history_open_segment_first_event_id; - if !(is_kept || (moved_after_command && sealed_now)) { + let has_migration_step = is_migration_step(previous, current, run, view, seq); + let is_valid = if is_migration_required { + has_migration_step + } else { + is_kept || (moved_after_command && sealed_now) || has_migration_step + }; + if !is_valid { return Err(violation( "H9-open-segment-transition", &context, - format!("seq {seq} moved the open segment without sealing it"), + format!( + "seq {seq} moved the open segment without sealing it, or a running \ + migration made no progress" + ), )); } } diff --git a/server-zd/crates/dexzd-simulation/src/invariants/mod.rs b/server-zd/crates/dexzd-simulation/src/invariants/mod.rs index e3604c247c..59971d1b4e 100644 --- a/server-zd/crates/dexzd-simulation/src/invariants/mod.rs +++ b/server-zd/crates/dexzd-simulation/src/invariants/mod.rs @@ -11,6 +11,7 @@ mod attribute_sync_invariants; mod directory_invariants; mod dispatch_queue_invariants; +mod history_migration_invariants; mod history_segment_invariants; mod lock_invariants; mod search_order_invariants; @@ -28,6 +29,8 @@ pub use attribute_sync_invariants::CloseBarrierCounts; pub(crate) use attribute_sync_invariants::{ AttributeSyncAcks, check_close_barrier, check_outbox_writes, remember_ack, }; +pub use history_migration_invariants::HistoryMigrationCounts; +pub(crate) use history_migration_invariants::check_history_migration; pub(crate) use history_segment_invariants::{ OffloadGapsByRun, check_deletions_were_uploaded, check_open_segment_transitions, check_sealed_segment_encodings, diff --git a/server-zd/crates/dexzd-simulation/src/lib.rs b/server-zd/crates/dexzd-simulation/src/lib.rs index 21f6e5cdc0..994493fd9f 100644 --- a/server-zd/crates/dexzd-simulation/src/lib.rs +++ b/server-zd/crates/dexzd-simulation/src/lib.rs @@ -38,8 +38,8 @@ pub use invalid_client_inputs::{ check_invalid_client_input_rejected, invalid_client_input, }; pub use invariants::{ - CheckerMemory, CloseBarrierCounts, GUARDED_ATTRIBUTE_KEYS, InvariantViolation, check_flow, - check_fresh_snapshot, + CheckerMemory, CloseBarrierCounts, GUARDED_ATTRIBUTE_KEYS, HistoryMigrationCounts, + InvariantViolation, check_flow, check_fresh_snapshot, }; pub use memory_state::MemoryFlowState; pub use model::{ diff --git a/server-zd/crates/dexzd-simulation/tests/apply_invariants.rs b/server-zd/crates/dexzd-simulation/tests/apply_invariants.rs index 1b952a71d1..27c8e0dd6a 100644 --- a/server-zd/crates/dexzd-simulation/tests/apply_invariants.rs +++ b/server-zd/crates/dexzd-simulation/tests/apply_invariants.rs @@ -144,6 +144,11 @@ fn apply_keeps_every_invariant_over_random_command_sequences() { total.history_segments_sealed > 0, "the seed set never sealed a history segment: {total:#?}" ); + let migrations = total.history_migrations; + assert!( + migrations.started > 0 && migrations.completed > 0 && migrations.abandoned > 0, + "the seed set never started, completed, and abandoned a history migration: {total:#?}" + ); assert!( total.history_offload_off_time_travels > 0, "the seed set never time traveled a run while its partition did not offload: {total:#?}" diff --git a/server-zd/proto/dexzd/v1/state.proto b/server-zd/proto/dexzd/v1/state.proto index 0333a6e6ef..78f9b3ef5a 100644 --- a/server-zd/proto/dexzd/v1/state.proto +++ b/server-zd/proto/dexzd/v1/state.proto @@ -164,6 +164,9 @@ message RunRecord { uint64 history_open_segment_superseded_value_count = 62; // Last seq of the newest uploaded history segment (U); zero before the first upload. uint64 history_uploaded_through_seq = 63; + // Seq that started migrating older history; zero when not migrating. The open segment then + // tracks no bytes. + uint64 history_migration_started_seq = 64; // First event ID the open history segment can hold: the run's next event ID when it opened. int64 history_open_segment_first_event_id = 66; } diff --git a/server-zd/testdata/v1/README.md b/server-zd/testdata/v1/README.md index 884a0849a1..f516e8e398 100644 --- a/server-zd/testdata/v1/README.md +++ b/server-zd/testdata/v1/README.md @@ -79,6 +79,13 @@ Then the node took a snapshot. After the snapshot: run's only history segment with that value listed, and a HistorySegmentUploaded acknowledgment marks the segment uploaded and removes the run's pending index row. - Partition 0 disables history offload, which deletes its mode record. +- `legacy-running` and `legacy-closed`, in partition 2, which never offloads history, run with an + eight-event window, a checkpoint every two commands, and 1,500-byte segments. Thirty commands + that alternately replace a long `profile` Attribute and publish to a Channel let the window + prune `legacy-running`, which stays running without segment state. `legacy-closed` applies four + such commands and is canceled; its 60-second retention timer stays pending. +- `legacy-migrating`, in partition 3, records the same thirty commands. Then partition 3 enables + history offload, and each of the run's next two commands seals one migration segment. Every `FlowCommand` kind of behavior version 1 appears in the log. Hand-out commands outside `handout-sync` and `handout-travel` carry no task ID, as an older log does. Log cleanup did not run, diff --git a/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000118 b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000118 new file mode 100644 index 0000000000..ebb74c15f7 Binary files /dev/null and b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000118 differ diff --git a/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000119 b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000119 new file mode 100644 index 0000000000..2444349dd4 Binary files /dev/null and b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000119 differ diff --git a/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000120 b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000120 new file mode 100644 index 0000000000..f13d4d99f3 Binary files /dev/null and b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000120 differ diff --git a/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000121 b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000121 new file mode 100644 index 0000000000..76ea84af22 Binary files /dev/null and b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000121 differ diff --git a/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000122 b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000122 new file mode 100644 index 0000000000..9b3a032b17 Binary files /dev/null and b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000122 differ diff --git a/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000123 b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000123 new file mode 100644 index 0000000000..10d28d08c7 Binary files /dev/null and b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000123 differ diff --git a/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000124 b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000124 new file mode 100644 index 0000000000..919a5f3aae Binary files /dev/null and b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000124 differ diff --git a/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000125 b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000125 new file mode 100644 index 0000000000..f5c13d1edc Binary files /dev/null and b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000125 differ diff --git a/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000126 b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000126 new file mode 100644 index 0000000000..5483c5a8c6 Binary files /dev/null and b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000126 differ diff --git a/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000127 b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000127 new file mode 100644 index 0000000000..76ae6c2bea Binary files /dev/null and b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000127 differ diff --git a/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000128 b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000128 new file mode 100644 index 0000000000..de27bcf559 Binary files /dev/null and b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000128 differ diff --git a/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000129 b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000129 new file mode 100644 index 0000000000..c4d2c69ea2 Binary files /dev/null and b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000129 differ diff --git a/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000130 b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000130 new file mode 100644 index 0000000000..2112becf23 Binary files /dev/null and b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000130 differ diff --git a/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000131 b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000131 new file mode 100644 index 0000000000..9854102c55 Binary files /dev/null and b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000131 differ diff --git a/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000132 b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000132 new file mode 100644 index 0000000000..c7ced09abd Binary files /dev/null and b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000132 differ diff --git a/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000133 b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000133 new file mode 100644 index 0000000000..b416e40623 Binary files /dev/null and b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000133 differ diff --git a/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000134 b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000134 new file mode 100644 index 0000000000..209061b34b Binary files /dev/null and b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000134 differ diff --git a/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000135 b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000135 new file mode 100644 index 0000000000..5da5fcc78d Binary files /dev/null and b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000135 differ diff --git a/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000136 b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000136 new file mode 100644 index 0000000000..a51755032b Binary files /dev/null and b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000136 differ diff --git a/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000137 b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000137 new file mode 100644 index 0000000000..60167093d4 Binary files /dev/null and b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000137 differ diff --git a/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000138 b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000138 new file mode 100644 index 0000000000..8bc95e8f30 Binary files /dev/null and b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000138 differ diff --git a/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000139 b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000139 new file mode 100644 index 0000000000..72a36c9491 Binary files /dev/null and b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000139 differ diff --git a/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000140 b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000140 new file mode 100644 index 0000000000..2069a2028b Binary files /dev/null and b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000140 differ diff --git a/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000141 b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000141 new file mode 100644 index 0000000000..1f1068a042 Binary files /dev/null and b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000141 differ diff --git a/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000142 b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000142 new file mode 100644 index 0000000000..271b3bcbb4 Binary files /dev/null and b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000142 differ diff --git a/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000143 b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000143 new file mode 100644 index 0000000000..bfc0479c8e Binary files /dev/null and b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000143 differ diff --git a/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000144 b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000144 new file mode 100644 index 0000000000..442e4a07a6 Binary files /dev/null and b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000144 differ diff --git a/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000145 b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000145 new file mode 100644 index 0000000000..157635b725 Binary files /dev/null and b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000145 differ diff --git a/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000146 b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000146 new file mode 100644 index 0000000000..14482c2e6e Binary files /dev/null and b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000146 differ diff --git a/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000147 b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000147 new file mode 100644 index 0000000000..969105d6c9 Binary files /dev/null and b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000147 differ diff --git a/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000148 b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000148 new file mode 100644 index 0000000000..646cfd2e9b Binary files /dev/null and b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000148 differ diff --git a/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000149 b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000149 new file mode 100644 index 0000000000..3e8118f2cd Binary files /dev/null and b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000149 differ diff --git a/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000150 b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000150 new file mode 100644 index 0000000000..2706cda7e5 Binary files /dev/null and b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000150 differ diff --git a/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000151 b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000151 new file mode 100644 index 0000000000..ec5d45f204 Binary files /dev/null and b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000151 differ diff --git a/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000152 b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000152 new file mode 100644 index 0000000000..715eb4a2f4 Binary files /dev/null and b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000152 differ diff --git a/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000153 b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000153 new file mode 100644 index 0000000000..8d019589e7 Binary files /dev/null and b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000153 differ diff --git a/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000154 b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000154 new file mode 100644 index 0000000000..a9317f6437 Binary files /dev/null and b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000154 differ diff --git a/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000155 b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000155 new file mode 100644 index 0000000000..ed67e36bb6 Binary files /dev/null and b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000155 differ diff --git a/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000156 b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000156 new file mode 100644 index 0000000000..162c43ee6a Binary files /dev/null and b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000156 differ diff --git a/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000157 b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000157 new file mode 100644 index 0000000000..89befa8f21 Binary files /dev/null and b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000157 differ diff --git a/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000158 b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000158 new file mode 100644 index 0000000000..90baabb221 Binary files /dev/null and b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000158 differ diff --git a/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000159 b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000159 new file mode 100644 index 0000000000..3f1a1cf729 Binary files /dev/null and b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000159 differ diff --git a/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000160 b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000160 new file mode 100644 index 0000000000..eb85827039 Binary files /dev/null and b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000160 differ diff --git a/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000161 b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000161 new file mode 100644 index 0000000000..cab070d863 Binary files /dev/null and b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000161 differ diff --git a/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000162 b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000162 new file mode 100644 index 0000000000..917cf161e9 Binary files /dev/null and b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000162 differ diff --git a/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000163 b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000163 new file mode 100644 index 0000000000..cb315891cf Binary files /dev/null and b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000163 differ diff --git a/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000164 b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000164 new file mode 100644 index 0000000000..e9cd820c77 Binary files /dev/null and b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000164 differ diff --git a/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000165 b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000165 new file mode 100644 index 0000000000..c4478add18 Binary files /dev/null and b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000165 differ diff --git a/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000166 b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000166 new file mode 100644 index 0000000000..0e76aa4f44 Binary files /dev/null and b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000166 differ diff --git a/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000167 b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000167 new file mode 100644 index 0000000000..aa9cb0ed42 Binary files /dev/null and b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000167 differ diff --git a/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000168 b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000168 new file mode 100644 index 0000000000..94982e5c20 Binary files /dev/null and b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000168 differ diff --git a/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000169 b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000169 new file mode 100644 index 0000000000..fc83026a83 Binary files /dev/null and b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000169 differ diff --git a/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000170 b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000170 new file mode 100644 index 0000000000..f65493f0de Binary files /dev/null and b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000170 differ diff --git a/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000171 b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000171 new file mode 100644 index 0000000000..569226bd5b Binary files /dev/null and b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000171 differ diff --git a/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000172 b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000172 new file mode 100644 index 0000000000..12faa3359a Binary files /dev/null and b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000172 differ diff --git a/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000173 b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000173 new file mode 100644 index 0000000000..a50a05200a Binary files /dev/null and b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000173 differ diff --git a/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000174 b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000174 new file mode 100644 index 0000000000..24415b5e65 Binary files /dev/null and b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000174 differ diff --git a/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000175 b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000175 new file mode 100644 index 0000000000..2b6714c057 Binary files /dev/null and b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000175 differ diff --git a/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000176 b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000176 new file mode 100644 index 0000000000..940692e7cb Binary files /dev/null and b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000176 differ diff --git a/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000177 b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000177 new file mode 100644 index 0000000000..51b558af0f Binary files /dev/null and b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000177 differ diff --git a/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000178 b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000178 new file mode 100644 index 0000000000..23000e4459 Binary files /dev/null and b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000178 differ diff --git a/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000179 b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000179 new file mode 100644 index 0000000000..546815871f Binary files /dev/null and b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000179 differ diff --git a/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000180 b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000180 new file mode 100644 index 0000000000..3fd3432451 Binary files /dev/null and b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000180 differ diff --git a/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000181 b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000181 new file mode 100644 index 0000000000..4b4735c708 Binary files /dev/null and b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000181 differ diff --git a/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000182 b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000182 new file mode 100644 index 0000000000..ba78903d5f Binary files /dev/null and b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000182 differ diff --git a/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000183 b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000183 new file mode 100644 index 0000000000..7b056a5083 Binary files /dev/null and b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000183 differ diff --git a/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000184 b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000184 new file mode 100644 index 0000000000..ce8a731aae Binary files /dev/null and b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000184 differ diff --git a/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000185 b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000185 new file mode 100644 index 0000000000..787fad0cc4 Binary files /dev/null and b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000185 differ diff --git a/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000186 b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000186 new file mode 100644 index 0000000000..eedcf53a0e Binary files /dev/null and b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000186 differ diff --git a/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000187 b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000187 new file mode 100644 index 0000000000..6440aa1702 Binary files /dev/null and b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000187 differ diff --git a/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000188 b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000188 new file mode 100644 index 0000000000..9d049fd734 Binary files /dev/null and b/server-zd/testdata/v1/objects/log/node-a/0000000001/00000000000000000188 differ diff --git a/server-zd/testdata/v1/state.txt b/server-zd/testdata/v1/state.txt index f62c379312..984659830d 100644 --- a/server-zd/testdata/v1/state.txt +++ b/server-zd/testdata/v1/state.txt @@ -1,6 +1,6 @@ # column family, key hex, value: =hex or #length:xxh3-128 flow 00c77b2df1ff0c810100000000000000006163636f756e742d73796e63 =0a310a1001a3185ced3a76f5860d3203b7c6fd9f100118a0f689d8f2a299032a1273746172742d6163636f756e742d73796e63 -flow 00c77b2df1ff0c810201a3185ced3a76f5860d3203b7c6fd9f01 #193:8822af52e8329a77f4d88feea93b661d +flow 00c77b2df1ff0c810201a3185ced3a76f5860d3203b7c6fd9f01 #203:128446b10be7b0a6a8d897afe01d22b0 flow 00c77b2df1ff0c810201a3185ced3a76f5860d3203b7c6fd9f0200000001 =0801200128a0f689d8f2a299033a0101 flow 00c77b2df1ff0c810201a3185ced3a76f5860d3203b7c6fd9f036e6f746573 =0a0e0a0c1a0a6e6f742073796e6365642001 flow 00c77b2df1ff0c810201a3185ced3a76f5860d3203b7c6fd9f03706c616e732f6261736963 =0a090a071a05747269616c1a0208012001 @@ -13,6 +13,7 @@ flow 00c77b2df1ff0c810201a3185ced3a76f5860d3203b7c6fd9f1900000000000000011600000 flow 00c77b2df1ff0c810201a3185ced3a76f5860d3203b7c6fd9f1900000000000000021500000000 =08f70110011801200128b09799d8f2a29903300240025001 flow 00c77b2df1ff0c810201a3185ced3a76f5860d3203b7c6fd9f1900000000000000031500000000 =087f20012880cfd9daf2a2990330025002 flow 00c77b2df1ff0c810201a3185ced3a76f5860d3203b7c6fd9f1900000000000000041500000000 =08890120012890f0e8daf2a2990330025002 +flow 00c77b2df1ff0c810201a3185ced3a76f5860d3203b7c6fd9f1b0000000000000001 =080110031801200228013080cfd9daf2a29903 flow 0808577b47782b9201000000000000000068616e646f75742d73796e63 =0a3a0a1001a3185e0b5c7f468a30ab8a1e4c9f30100218f0df80fbf2a2990320a0828582f3a299032a1273746172742d68616e646f75742d73796e63 flow 0808577b47782b920201a3185e0b5c7f468a30ab8a1e4c9f3001 #267:77e78141f0e76bea745920c7d239c113 flow 0808577b47782b920201a3185e0b5c7f468a30ab8a1e4c9f300200000001 =0801200128f0df80fbf2a299033a0101 @@ -45,7 +46,7 @@ flow 0808577b47782b920201a3185e0b5c7f468a30ab8a1e4c9f301900000000000000131500000 flow 0808577b47782b920201a3185e0b5c7f468a30ab8a1e4c9f301900000000000000141500000000 =089c031001200128a0828582f3a29903300440044a005008 flow 0808577b47782b920201a3185e0b5c7f468a30ab8a1e4c9f301b0000000000000001 =0801101418012004280130a0828582f3a29903 flow 0f0b8625e5f9724701000000000000000066756c66696c6c6d656e742d706172656e74 =0a390a1001a3185ce75e7c9ba774c48afeb866c0100218c0afaed7f2a2990320b0a093d9f2a299032a1173746172742d706172656e742d666c6f77 -flow 0f0b8625e5f972470201a3185ce75e7c9ba774c48afeb866c001 #263:badc593c1169517a4191d8d6ed8b5d0d +flow 0f0b8625e5f972470201a3185ce75e7c9ba774c48afeb866c001 #269:511451298b15a1f20e4641b162d3f946 flow 0f0b8625e5f972470201a3185ce75e7c9ba774c48afeb866c00200000001 =0801200128c0afaed7f2a299033a0101 flow 0f0b8625e5f972470201a3185ce75e7c9ba774c48afeb866c006537061776e537465702d31 #70:e33fbe11e90df82c2011677432edea75 flow 0f0b8625e5f972470201a3185ce75e7c9ba774c48afeb866c007537061776e53746570 =0801 @@ -64,6 +65,8 @@ flow 0f0b8625e5f972470201a3185ce75e7c9ba774c48afeb866c01900000000000000051500000 flow 0f0b8625e5f972470201a3185ce75e7c9ba774c48afeb866c01900000000000000061500000000 =089803100120012880bde5d8f2a2990330023a1a0a0b537061776e537465702d311209537061776e5374657018025002 flow 0f0b8625e5f972470201a3185ce75e7c9ba774c48afeb866c01900000000000000071500000000 =089e01200128a0ff83d9f2a2990330025003 flow 0f0b8625e5f972470201a3185ce75e7c9ba774c48afeb866c01900000000000000081500000000 =08ca031001200128b0a093d9f2a29903300440044a005003 +flow 0f0b8625e5f972470201a3185ce75e7c9ba774c48afeb866c01b0000000000000001 =080110061801200228013080bde5d8f2a29903 +flow 0f0b8625e5f972470201a3185ce75e7c9ba774c48afeb866c01b0000000000000007 =0807100818032004280130b0a093d9f2a29903 flow 30cdf0c2388267440100000000000000006f726465722d6173796e63 =0a390a1001a3185cf9ec7d97b2eddbc862722e0c100218f0a4d0d9f2a2990320c0ca9cdaf2a299032a1173746172742d6f726465722d6173796e63 flow 30cdf0c2388267440201a3185cf9ec7d97b2eddbc862722e0c01 #260:a03e25f4e25c534ada3008325763d848 flow 30cdf0c2388267440201a3185cf9ec7d97b2eddbc862722e0c0200000001 =0801200128f0a4d0d9f2a299033a0101 @@ -188,6 +191,36 @@ flow 7becb50ef3da384a0201a3185e588277649c53bb00e0d05cb81900000000000000031500000 flow 7becb50ef3da384a0201a3185e588277649c53bb00e0d05cb81900000000000000041500000000 =08b301100120012890fce384f3a29903300440044a005003 flow 7becb50ef3da384a0201a3185e588277649c53bb00e0d05cb81b0000000000000001 =080110041801200428023090fce384f3a29903380142105c676d28d5c85784a8c10badd6d715bc48a008 flow 7becb50ef3da384a0201a3185e588277649c53bb00e0d05cb81c0000000000000001000000000000000200000000 =0803109301 +flow 88235d7e7155ba860100000000000000006c65676163792d636c6f7365642d32 =0a3d0a1001a3185e7ca476349f597dc4a1062f54100618b0e2ea88f3a29903208088b789f3a299032a1573746172742d6c65676163792d636c6f7365642d32 +flow 88235d7e7155ba860201a3185e7ca476349f597dc4a1062f5401 #209:e76adefc99ca07fa18501e62d95f7500 +flow 88235d7e7155ba860201a3185e7ca476349f597dc4a1062f540200000001 =0801200128b0e2ea88f3a299033a03010305 +flow 88235d7e7155ba860201a3185e7ca476349f597dc4a1062f540370726f66696c65 =0a0b12090a020804109c0118012004 +flow 88235d7e7155ba860201a3185e7ca476349f597dc4a1062f540400077570646174657300000000002dc6c0 =0a077570646174657312196c65676163792d636c6f7365642d322d6d6573736167652d311a0b12090a020803109c01180120c08db701 +flow 88235d7e7155ba860201a3185e7ca476349f597dc4a1062f540400077570646174657300000000004c4b40 =0a077570646174657312196c65676163792d636c6f7365642d322d6d6573736167652d331a0b12090a020805109c01180120c096b102 +flow 88235d7e7155ba860201a3185e7ca476349f597dc4a1062f540575706461746573 =0802 +flow 88235d7e7155ba860201a3185e7ca476349f597dc4a1062f540a2a00 =0a022a0010809685a6f3a29903 +flow 88235d7e7155ba860201a3185e7ca476349f597dc4a1062f540d026c65676163792d636c6f7365642d322d77726974652d30 =0a10006fe65e4a432b1510a7f79f0d7ff3c112022a001802 +flow 88235d7e7155ba860201a3185e7ca476349f597dc4a1062f540d026c65676163792d636c6f7365642d322d77726974652d32 =0a101c0adb45f679bbf4d47791aa22e1375412022a001804 +flow 88235d7e7155ba860201a3185e7ca476349f597dc4a1062f5411000000000000000200000000 =089c0110021804 +flow 88235d7e7155ba860201a3185e7ca476349f597dc4a1062f5411000000000000000300000000 =089c011003 +flow 88235d7e7155ba860201a3185e7ca476349f597dc4a1062f5411000000000000000400000000 =089c011004 +flow 88235d7e7155ba860201a3185e7ca476349f597dc4a1062f5411000000000000000500000000 =089c011005 +flow 88235d7e7155ba860201a3185e7ca476349f597dc4a1062f54120000000000000003000000000300077570646174657300000000002dc6c0 = +flow 88235d7e7155ba860201a3185e7ca476349f597dc4a1062f541200000000000000040000000002000770726f66696c65 = +flow 88235d7e7155ba860201a3185e7ca476349f597dc4a1062f54120000000000000005000000000300077570646174657300000000004c4b40 = +flow 88235d7e7155ba860201a3185e7ca476349f597dc4a1062f54130000000000000004000000000000000200000000 =089c01 +flow 88235d7e7155ba860201a3185e7ca476349f597dc4a1062f5417 #69:f42a7baa6b9b35c94630eb229d555fc8 +flow 88235d7e7155ba860201a3185e7ca476349f597dc4a1062f54180000000000000002 =026c65676163792d636c6f7365642d322d77726974652d30 +flow 88235d7e7155ba860201a3185e7ca476349f597dc4a1062f54180000000000000004 =026c65676163792d636c6f7365642d322d77726974652d32 +flow 88235d7e7155ba860201a3185e7ca476349f597dc4a1062f541900000000000000011500000000 =08ee021001200128b0e2ea88f3a29903300140015000 +flow 88235d7e7155ba860201a3185e7ca476349f597dc4a1062f541900000000000000011600000001 =082e +flow 88235d7e7155ba860201a3185e7ca476349f597dc4a1062f541900000000000000021500000000 =08b40310011801200128c083fa88f3a29903300240025001 +flow 88235d7e7155ba860201a3185e7ca476349f597dc4a1062f541900000000000000031500000000 =08b10310011801200128d0a48989f3a29903300340035002 +flow 88235d7e7155ba860201a3185e7ca476349f597dc4a1062f541900000000000000031600000001 =08ac01 +flow 88235d7e7155ba860201a3185e7ca476349f597dc4a1062f541900000000000000041500000000 =08ba0310011801200128e0c59889f3a29903300440045003 +flow 88235d7e7155ba860201a3185e7ca476349f597dc4a1062f541900000000000000051500000000 =08b20310011801200128f0e6a789f3a29903300540055004 +flow 88235d7e7155ba860201a3185e7ca476349f597dc4a1062f541900000000000000051600000001 =08fb01 +flow 88235d7e7155ba860201a3185e7ca476349f597dc4a1062f541900000000000000061500000000 =08b30110012001288088b789f3a29903300640064a005005 flow 957748ea772a492d01000000000000000070726f66696c652d726576696577 =0a3c0a1001a3185ce1827fd69291cf63846a12f3100218e0e8d2d6f2a2990320e08cbbdaf2a299032a1473746172742d70726f66696c652d726576696577 flow 957748ea772a492d0201a3185ce1827fd69291cf63846a12f301 #262:24f914ad660701729ddb7ab496a751a3 flow 957748ea772a492d0201a3185ce1827fd69291cf63846a12f30200000001 =0801200128e0e8d2d6f2a299033a0101 @@ -228,6 +261,113 @@ flow 9b749ea64ab099e80201a3185cea4c7d5982c3e7ad08b681721900000000000000011600000 flow 9b749ea64ab099e80201a3185cea4c7d5982c3e7ad08b681721900000000000000021500000000 =08b501200128e0fac6d8f2a2990330015001 flow 9b749ea64ab099e80201a3185cea4c7d5982c3e7ad08b681721900000000000000031500000000 =08c5031001200128f09bd6d8f2a29903300340034a005001 flow 9b749ea64ab099e80201a3185cea4c7d5982c3e7ad08b681721900000000000000041500000000 =08a10120012890def4d8f2a2990330035003 +flow a8ecaec88da2f4f00100000000000000006c65676163792d72756e6e696e672d33 =0a350a1001a3185e5e5e7db7b414da763fa4bf16100118c0df9185f3a299032a1673746172742d6c65676163792d72756e6e696e672d33 +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1601 #206:b285d4c48769cec9a6b8b9659cc5d02f +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf160200000001 =0801200128c0df9185f3a299033a060117191b1d1f +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf160370726f66696c65 =0a0b12090a02081e109d011801201e +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf160400077570646174657300000000002dc6c0 =0a0775706461746573121a6c65676163792d72756e6e696e672d332d6d6573736167652d311a0b12090a020803109c01180120c08db701 +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf160400077570646174657300000000004c4b40 =0a0775706461746573121a6c65676163792d72756e6e696e672d332d6d6573736167652d331a0b12090a020805109c01180120c096b102 +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf160400077570646174657300000000006acfc0 =0a0775706461746573121a6c65676163792d72756e6e696e672d332d6d6573736167652d351a0b12090a020807109c01180120c09fab03 +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf16040007757064617465730000000000895440 =0a0775706461746573121a6c65676163792d72756e6e696e672d332d6d6573736167652d371a0b12090a020809109c01180120c0a8a504 +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf16040007757064617465730000000000a7d8c0 =0a0775706461746573121a6c65676163792d72756e6e696e672d332d6d6573736167652d391a0b12090a02080b109c01180120c0b19f05 +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf16040007757064617465730000000000c65d40 =0a0775706461746573121b6c65676163792d72756e6e696e672d332d6d6573736167652d31311a0b12090a02080d109d01180120c0ba9906 +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf16040007757064617465730000000000e4e1c0 =0a0775706461746573121b6c65676163792d72756e6e696e672d332d6d6573736167652d31331a0b12090a02080f109d01180120c0c39307 +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf16040007757064617465730000000001036640 =0a0775706461746573121b6c65676163792d72756e6e696e672d332d6d6573736167652d31351a0b12090a020811109d01180120c0cc8d08 +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1604000775706461746573000000000121eac0 =0a0775706461746573121b6c65676163792d72756e6e696e672d332d6d6573736167652d31371a0b12090a020813109d01180120c0d58709 +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf16040007757064617465730000000001406f40 =0a0775706461746573121b6c65676163792d72756e6e696e672d332d6d6573736167652d31391a0b12090a020815109d01180120c0de810a +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf160400077570646174657300000000015ef3c0 =0a0775706461746573121b6c65676163792d72756e6e696e672d332d6d6573736167652d32311a0b12090a020817109d01180120c0e7fb0a +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf160400077570646174657300000000017d7840 =0a0775706461746573121b6c65676163792d72756e6e696e672d332d6d6573736167652d32331a0b12090a020819109d01180120c0f0f50b +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf160400077570646174657300000000019bfcc0 =0a0775706461746573121b6c65676163792d72756e6e696e672d332d6d6573736167652d32351a0b12090a02081b109d01180120c0f9ef0c +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf16040007757064617465730000000001ba8140 =0a0775706461746573121b6c65676163792d72756e6e696e672d332d6d6573736167652d32371a0b12090a02081d109d01180120c082ea0d +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf16040007757064617465730000000001d905c0 =0a0775706461746573121b6c65676163792d72756e6e696e672d332d6d6573736167652d32391a0b12090a02081f109d01180120c08be40e +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf160575706461746573 =080f +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf160d026c65676163792d72756e6e696e672d332d77726974652d30 =0a10665a3fb408c367e7aa4fd6868ac3ba0a12022a001802 +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf160d026c65676163792d72756e6e696e672d332d77726974652d3130 =0a10f45ec5385707a06abe06d0bf76480ebb12022a00180c +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf160d026c65676163792d72756e6e696e672d332d77726974652d3132 =0a103685a84168107295396a24f2b843b22912022a00180e +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf160d026c65676163792d72756e6e696e672d332d77726974652d3134 =0a10d961c523d8c17117cca03124a9ad354712022a001810 +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf160d026c65676163792d72756e6e696e672d332d77726974652d3136 =0a105b0906bc34f285109f6c99ca3329503012022a001812 +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf160d026c65676163792d72756e6e696e672d332d77726974652d3138 =0a101851c26846967522a4f5f96a3d0edc7e12022a001814 +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf160d026c65676163792d72756e6e696e672d332d77726974652d32 =0a100cd5b98c9571c4f15662c5288a5d262112022a001804 +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf160d026c65676163792d72756e6e696e672d332d77726974652d3230 =0a10471d0cd231b7d1ee124049ae91d9acbd12022a001816 +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf160d026c65676163792d72756e6e696e672d332d77726974652d3232 =0a10dc0276a7ebf561653bd804cc337f656d12022a001818 +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf160d026c65676163792d72756e6e696e672d332d77726974652d3234 =0a100f4df85838015a332a46cf87fd3f594612022a00181a +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf160d026c65676163792d72756e6e696e672d332d77726974652d3236 =0a1006c71312830e970f0dc5823ae99b9a7312022a00181c +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf160d026c65676163792d72756e6e696e672d332d77726974652d3238 =0a10abb9585719b3d6e74d3c59b098f08d6312022a00181e +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf160d026c65676163792d72756e6e696e672d332d77726974652d34 =0a106efa41cf05816c2a12714ce5121b81ad12022a001806 +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf160d026c65676163792d72756e6e696e672d332d77726974652d36 =0a106ebfe05210ed0c554a4b1d083595edb012022a001808 +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf160d026c65676163792d72756e6e696e672d332d77726974652d38 =0a10d1ae27f3217ced88fcec580cc87b612312022a00180a +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1611000000000000000300000000 =089c011003 +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1611000000000000000500000000 =089c011005 +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1611000000000000000700000000 =089c011007 +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1611000000000000000900000000 =089c011009 +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1611000000000000000b00000000 =089c01100b +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1611000000000000000d00000000 =089d01100d +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1611000000000000000f00000000 =089d01100f +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1611000000000000001100000000 =089d011011 +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1611000000000000001300000000 =089d011013 +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1611000000000000001500000000 =089d011015 +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1611000000000000001600000000 =089d0110161818 +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1611000000000000001700000000 =089d011017 +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1611000000000000001800000000 =089d011018181a +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1611000000000000001900000000 =089d011019 +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1611000000000000001a00000000 =089d01101a181c +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1611000000000000001b00000000 =089d01101b +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1611000000000000001c00000000 =089d01101c181e +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1611000000000000001d00000000 =089d01101d +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1611000000000000001e00000000 =089d01101e +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1611000000000000001f00000000 =089d01101f +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf16120000000000000003000000000300077570646174657300000000002dc6c0 = +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf16120000000000000005000000000300077570646174657300000000004c4b40 = +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf16120000000000000007000000000300077570646174657300000000006acfc0 = +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1612000000000000000900000000030007757064617465730000000000895440 = +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1612000000000000000b00000000030007757064617465730000000000a7d8c0 = +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1612000000000000000d00000000030007757064617465730000000000c65d40 = +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1612000000000000000f00000000030007757064617465730000000000e4e1c0 = +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1612000000000000001100000000030007757064617465730000000001036640 = +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf161200000000000000130000000003000775706461746573000000000121eac0 = +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1612000000000000001500000000030007757064617465730000000001406f40 = +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf16120000000000000017000000000300077570646174657300000000015ef3c0 = +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf16120000000000000019000000000300077570646174657300000000017d7840 = +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1612000000000000001b000000000300077570646174657300000000019bfcc0 = +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1612000000000000001d00000000030007757064617465730000000001ba8140 = +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1612000000000000001e0000000002000770726f66696c65 = +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1612000000000000001f00000000030007757064617465730000000001d905c0 = +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf16130000000000000018000000000000001600000000 =089d01 +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1613000000000000001a000000000000001800000000 =089d01 +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1613000000000000001c000000000000001a00000000 =089d01 +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1613000000000000001e000000000000001c00000000 =089d01 +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1617 =08011a106c65676163792d72756e6e696e672d33221001a3185e5e5e7db7b414da763fa4bf162a0a4c6567616379466c6f77300138c0df9185f3a29903 +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf16180000000000000002 =026c65676163792d72756e6e696e672d332d77726974652d30 +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf16180000000000000004 =026c65676163792d72756e6e696e672d332d77726974652d32 +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf16180000000000000006 =026c65676163792d72756e6e696e672d332d77726974652d34 +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf16180000000000000008 =026c65676163792d72756e6e696e672d332d77726974652d36 +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1618000000000000000a =026c65676163792d72756e6e696e672d332d77726974652d38 +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1618000000000000000c =026c65676163792d72756e6e696e672d332d77726974652d3130 +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1618000000000000000e =026c65676163792d72756e6e696e672d332d77726974652d3132 +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf16180000000000000010 =026c65676163792d72756e6e696e672d332d77726974652d3134 +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf16180000000000000012 =026c65676163792d72756e6e696e672d332d77726974652d3136 +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf16180000000000000014 =026c65676163792d72756e6e696e672d332d77726974652d3138 +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf16180000000000000016 =026c65676163792d72756e6e696e672d332d77726974652d3230 +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf16180000000000000018 =026c65676163792d72756e6e696e672d332d77726974652d3232 +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1618000000000000001a =026c65676163792d72756e6e696e672d332d77726974652d3234 +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1618000000000000001c =026c65676163792d72756e6e696e672d332d77726974652d3236 +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1618000000000000001e =026c65676163792d72756e6e696e672d332d77726974652d3238 +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf161900000000000000011500000000 =08ee021001200128c0df9185f3a29903300140015000 +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf161900000000000000011600000001 =082e +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf161900000000000000171500000000 =08b90310011801200128a0b8e187f3a29903301740175016 +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf161900000000000000171600000001 =08d307 +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf161900000000000000181500000000 =08c00310011801200128b0d9f087f3a29903301840185017 +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf161900000000000000191500000000 =08b30310011801200128c0faff87f3a29903301940195018 +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf161900000000000000191600000001 =08a408 +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1619000000000000001a1500000000 =08bf0310011801200128d09b8f88f3a29903301a401a5019 +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1619000000000000001b1500000000 =08b90310011801200128e0bc9e88f3a29903301b401b501a +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1619000000000000001b1600000001 =08f508 +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1619000000000000001c1500000000 =08c00310011801200128f0ddad88f3a29903301c401c501b +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1619000000000000001d1500000000 =08b3031001180120012880ffbc88f3a29903301d401d501c +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1619000000000000001d1600000001 =08c609 +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1619000000000000001e1500000000 =08bf031001180120012890a0cc88f3a29903301e401e501d +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1619000000000000001f1500000000 =08b90310011801200128a0c1db88f3a29903301f401f501e +flow a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1619000000000000001f1600000001 =08970a flow bd8ac614675559d00100000000000000007175657565642d77616974696e67 =0a330a1001a3185e47e87c9ba5bc71cdf3a0f30a100118d0e5b282f3a299032a1473746172742d7175657565642d77616974696e67 flow bd8ac614675559d00201a3185e47e87c9ba5bc71cdf3a0f30a01 #177:bd3f67b6ec434f0df3659debe7e5287e flow bd8ac614675559d00201a3185e47e87c9ba5bc71cdf3a0f30a0200000001 =0801200128d0e5b282f3a299033a0101 @@ -290,6 +430,126 @@ flow d879b97faa5ef0cf0201a3185c50007c4ab7515b188df65b7b19000000000000000d1500000 flow d879b97faa5ef0cf0201a3185c50007c4ab7515b188df65b7b19000000000000000e1500000000 =08da0310011801200128c0c1a2d9f2a29903300740075009 flow d879b97faa5ef0cf0201a3185c50007c4ab7515b188df65b7b19000000000000000f1500000000 =089501200128d0e2b1d9f2a299033007500a flow d879b97faa5ef0cf0201a3185c50007c4ab7515b188df65b7b1900000000000000101500000000 =0892041001200128e083c1d9f2a29903300940094a00500a +flow fe9d7958c1defd0d0100000000000000006c65676163792d6d6967726174696e672d3136 =0a380a1001a3185e82807eb09cd50af4d0a22d3a10011890a9c689f3a299032a1973746172742d6c65676163792d6d6967726174696e672d3136 +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a01 #221:851f9ab04dc81a933f0f2966d3eb19bc +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a0200000001 =080120012890a9c689f3a299033a070117191b1d1f21 +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a0370726f66696c65 =0a0b12090a020820109d0118012020 +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a0400077570646174657300000000002dc6c0 =0a0775706461746573121d6c65676163792d6d6967726174696e672d31362d6d6573736167652d311a0b12090a020803109c01180120c08db701 +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a0400077570646174657300000000004c4b40 =0a0775706461746573121d6c65676163792d6d6967726174696e672d31362d6d6573736167652d331a0b12090a020805109c01180120c096b102 +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a0400077570646174657300000000006acfc0 =0a0775706461746573121d6c65676163792d6d6967726174696e672d31362d6d6573736167652d351a0b12090a020807109c01180120c09fab03 +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a040007757064617465730000000000895440 =0a0775706461746573121d6c65676163792d6d6967726174696e672d31362d6d6573736167652d371a0b12090a020809109c01180120c0a8a504 +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a040007757064617465730000000000a7d8c0 =0a0775706461746573121d6c65676163792d6d6967726174696e672d31362d6d6573736167652d391a0b12090a02080b109c01180120c0b19f05 +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a040007757064617465730000000000c65d40 =0a0775706461746573121e6c65676163792d6d6967726174696e672d31362d6d6573736167652d31311a0b12090a02080d109d01180120c0ba9906 +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a040007757064617465730000000000e4e1c0 =0a0775706461746573121e6c65676163792d6d6967726174696e672d31362d6d6573736167652d31331a0b12090a02080f109d01180120c0c39307 +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a040007757064617465730000000001036640 =0a0775706461746573121e6c65676163792d6d6967726174696e672d31362d6d6573736167652d31351a0b12090a020811109d01180120c0cc8d08 +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a04000775706461746573000000000121eac0 =0a0775706461746573121e6c65676163792d6d6967726174696e672d31362d6d6573736167652d31371a0b12090a020813109d01180120c0d58709 +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a040007757064617465730000000001406f40 =0a0775706461746573121e6c65676163792d6d6967726174696e672d31362d6d6573736167652d31391a0b12090a020815109d01180120c0de810a +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a0400077570646174657300000000015ef3c0 =0a0775706461746573121e6c65676163792d6d6967726174696e672d31362d6d6573736167652d32311a0b12090a020817109d01180120c0e7fb0a +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a0400077570646174657300000000017d7840 =0a0775706461746573121e6c65676163792d6d6967726174696e672d31362d6d6573736167652d32331a0b12090a020819109d01180120c0f0f50b +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a0400077570646174657300000000019bfcc0 =0a0775706461746573121e6c65676163792d6d6967726174696e672d31362d6d6573736167652d32351a0b12090a02081b109d01180120c0f9ef0c +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a040007757064617465730000000001ba8140 =0a0775706461746573121e6c65676163792d6d6967726174696e672d31362d6d6573736167652d32371a0b12090a02081d109d01180120c082ea0d +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a040007757064617465730000000001d905c0 =0a0775706461746573121e6c65676163792d6d6967726174696e672d31362d6d6573736167652d32391a0b12090a02081f109d01180120c08be40e +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a040007757064617465730000000001f78a40 =0a0775706461746573121e6c65676163792d6d6967726174696e672d31362d6d6573736167652d33311a0b12090a020821109d01180120c094de0f +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a0575706461746573 =0810 +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a0d026c65676163792d6d6967726174696e672d31362d77726974652d30 =0a109a730bb6e96d47ac0dcd1e85fdaa6ca812022a001802 +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a0d026c65676163792d6d6967726174696e672d31362d77726974652d3130 =0a10ce4dca5d7c9346eb9fb3416e5c2ec56712022a00180c +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a0d026c65676163792d6d6967726174696e672d31362d77726974652d3132 =0a10e178d244826ce9d70517c3972ccafe2f12022a00180e +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a0d026c65676163792d6d6967726174696e672d31362d77726974652d3134 =0a100cd0e51f4e907643199468c80a7853ec12022a001810 +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a0d026c65676163792d6d6967726174696e672d31362d77726974652d3136 =0a10a0129ac76600a61728db1919e490db1c12022a001812 +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a0d026c65676163792d6d6967726174696e672d31362d77726974652d3138 =0a104d2b800105d96003cea6599bb10ea75e12022a001814 +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a0d026c65676163792d6d6967726174696e672d31362d77726974652d32 =0a101fb879f26f450f0a9ecbfc7ab63ab25312022a001804 +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a0d026c65676163792d6d6967726174696e672d31362d77726974652d3230 =0a10ee3508b3f7649c870514c6bf7111093e12022a001816 +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a0d026c65676163792d6d6967726174696e672d31362d77726974652d3232 =0a108ab8eee3965c3026920e7cadbc4077e012022a001818 +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a0d026c65676163792d6d6967726174696e672d31362d77726974652d3234 =0a10cb4007564446d23e6d271f2c4f1655c712022a00181a +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a0d026c65676163792d6d6967726174696e672d31362d77726974652d3236 =0a1054384b8435c41288cb9c31c3de70c61112022a00181c +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a0d026c65676163792d6d6967726174696e672d31362d77726974652d3238 =0a10390d58697493305a43d03676c6b7c4c012022a00181e +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a0d026c65676163792d6d6967726174696e672d31362d77726974652d3330 =0a1027fa82a7dbeb1c4f570ea9930079a9b512022a001820 +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a0d026c65676163792d6d6967726174696e672d31362d77726974652d34 =0a10be5e3030b11b26aa91f170b937b141b612022a001806 +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a0d026c65676163792d6d6967726174696e672d31362d77726974652d36 =0a10605312b6f87e68a83e53840be3dadbcd12022a001808 +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a0d026c65676163792d6d6967726174696e672d31362d77726974652d38 =0a1081881d100c51f513a1ad65d6810ab31c12022a00180a +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a11000000000000000300000000 =089c011003 +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a11000000000000000500000000 =089c011005 +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a11000000000000000700000000 =089c011007 +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a11000000000000000900000000 =089c011009 +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a11000000000000000b00000000 =089c01100b +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a11000000000000000d00000000 =089d01100d +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a11000000000000000f00000000 =089d01100f +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a11000000000000001100000000 =089d011011 +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a11000000000000001300000000 =089d011013 +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a11000000000000001500000000 =089d011015 +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a11000000000000001600000000 =089d0110161818 +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a11000000000000001700000000 =089d011017 +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a11000000000000001800000000 =089d011018181a +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a11000000000000001900000000 =089d011019 +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a11000000000000001a00000000 =089d01101a181c +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a11000000000000001b00000000 =089d01101b +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a11000000000000001c00000000 =089d01101c181e +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a11000000000000001d00000000 =089d01101d +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a11000000000000001e00000000 =089d01101e1820 +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a11000000000000001f00000000 =089d01101f +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a11000000000000002000000000 =089d011020 +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a11000000000000002100000000 =089d011021 +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a120000000000000003000000000300077570646174657300000000002dc6c0 = +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a120000000000000005000000000300077570646174657300000000004c4b40 = +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a120000000000000007000000000300077570646174657300000000006acfc0 = +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a12000000000000000900000000030007757064617465730000000000895440 = +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a12000000000000000b00000000030007757064617465730000000000a7d8c0 = +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a12000000000000000d00000000030007757064617465730000000000c65d40 = +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a12000000000000000f00000000030007757064617465730000000000e4e1c0 = +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a12000000000000001100000000030007757064617465730000000001036640 = +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a1200000000000000130000000003000775706461746573000000000121eac0 = +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a12000000000000001500000000030007757064617465730000000001406f40 = +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a120000000000000017000000000300077570646174657300000000015ef3c0 = +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a120000000000000019000000000300077570646174657300000000017d7840 = +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a12000000000000001b000000000300077570646174657300000000019bfcc0 = +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a12000000000000001d00000000030007757064617465730000000001ba8140 = +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a12000000000000001f00000000030007757064617465730000000001d905c0 = +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a1200000000000000200000000002000770726f66696c65 = +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a12000000000000002100000000030007757064617465730000000001f78a40 = +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a130000000000000018000000000000001600000000 =089d01 +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a13000000000000001a000000000000001800000000 =089d01 +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a13000000000000001c000000000000001a00000000 =089d01 +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a13000000000000001e000000000000001c00000000 =089d01 +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a130000000000000020000000000000001e00000000 =089d01 +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a17 =08011a136c65676163792d6d6967726174696e672d3136221001a3185e82807eb09cd50af4d0a22d3a2a0a4c6567616379466c6f7730013890a9c689f3a29903 +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a180000000000000002 =026c65676163792d6d6967726174696e672d31362d77726974652d30 +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a180000000000000004 =026c65676163792d6d6967726174696e672d31362d77726974652d32 +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a180000000000000006 =026c65676163792d6d6967726174696e672d31362d77726974652d34 +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a180000000000000008 =026c65676163792d6d6967726174696e672d31362d77726974652d36 +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a18000000000000000a =026c65676163792d6d6967726174696e672d31362d77726974652d38 +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a18000000000000000c =026c65676163792d6d6967726174696e672d31362d77726974652d3130 +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a18000000000000000e =026c65676163792d6d6967726174696e672d31362d77726974652d3132 +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a180000000000000010 =026c65676163792d6d6967726174696e672d31362d77726974652d3134 +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a180000000000000012 =026c65676163792d6d6967726174696e672d31362d77726974652d3136 +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a180000000000000014 =026c65676163792d6d6967726174696e672d31362d77726974652d3138 +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a180000000000000016 =026c65676163792d6d6967726174696e672d31362d77726974652d3230 +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a180000000000000018 =026c65676163792d6d6967726174696e672d31362d77726974652d3232 +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a18000000000000001a =026c65676163792d6d6967726174696e672d31362d77726974652d3234 +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a18000000000000001c =026c65676163792d6d6967726174696e672d31362d77726974652d3236 +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a18000000000000001e =026c65676163792d6d6967726174696e672d31362d77726974652d3238 +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a180000000000000020 =026c65676163792d6d6967726174696e672d31362d77726974652d3330 +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a1900000000000000011500000000 =08ff02100120012890a9c689f3a29903300140015000 +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a1900000000000000011600000001 =082e +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a1900000000000000171500000000 =08c20310011801200128f081968cf3a29903301740175016 +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a1900000000000000171600000001 =08f407 +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a1900000000000000181500000000 =08c3031001180120012880a3a58cf3a29903301840185017 +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a1900000000000000191500000000 =08c1031001180120012890c4b48cf3a29903301940195018 +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a1900000000000000191600000001 =08c808 +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a19000000000000001a1500000000 =08c90310011801200128a0e5c38cf3a29903301a401a5019 +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a19000000000000001b1500000000 =08c20310011801200128b086d38cf3a29903301b401b501a +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a19000000000000001b1600000001 =089c09 +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a19000000000000001c1500000000 =08c30310011801200128c0a7e28cf3a29903301c401c501b +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a19000000000000001d1500000000 =08c10310011801200128d0c8f18cf3a29903301d401d501c +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a19000000000000001d1600000001 =08f009 +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a19000000000000001e1500000000 =08c90310011801200128e0e9808df3a29903301e401e501d +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a19000000000000001f1500000000 =08c20310011801200128f08a908df3a29903301f401f501e +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a19000000000000001f1600000001 =08c40a +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a1900000000000000201500000000 =08c8031001180120012890cdae8df3a2990330204020501f +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a1900000000000000211500000000 =08c20310011801200128a0eebd8df3a29903302140215020 +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a1900000000000000211600000001 =08980b +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a1b0000000000000001 =080110171801201728013090cdae8df3a29903 +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a1b0000000000000018 =0818101918182019280130a0eebd8df3a299033801 +flow fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a1c0000000000000018000000000000001600000000 =0818109d01 history 00c77b2df1ff0c810201a3185ced3a76f5860d3203b7c6fd9f150000000000000001 #514:0cbf0fc00a8d2835a0270bab23b5b27d history 00c77b2df1ff0c810201a3185ced3a76f5860d3203b7c6fd9f150000000000000002 #213:6e320f01951a5cbcc616238dd1cbe3f9 history 00c77b2df1ff0c810201a3185ced3a76f5860d3203b7c6fd9f150000000000000003 #93:a823d7c055c3a44c410c4f7741b7c4a2 @@ -371,6 +631,15 @@ history 7becb50ef3da384a0201a3185e588277649c53bb00e0d05cb8150000000000000002 #38 history 7becb50ef3da384a0201a3185e588277649c53bb00e0d05cb8150000000000000003 #387:4e0a58cbc7129ea45f0f0ba7c35cda50 history 7becb50ef3da384a0201a3185e588277649c53bb00e0d05cb8150000000000000004 #145:b8915db22cfcc52bcfc0e726e491d254 history 7becb50ef3da384a0201a3185e588277649c53bb00e0d05cb816000000010000000000000001 =0801100122020801 +history 88235d7e7155ba860201a3185e7ca476349f597dc4a1062f54150000000000000001 #332:2273d27680f04b233c94aebb7c94258d +history 88235d7e7155ba860201a3185e7ca476349f597dc4a1062f54150000000000000002 #402:e0a526c499e8d469ef625db1b612b5da +history 88235d7e7155ba860201a3185e7ca476349f597dc4a1062f54150000000000000003 #399:8e6a17ebabe0f49de33f829fdf139732 +history 88235d7e7155ba860201a3185e7ca476349f597dc4a1062f54150000000000000004 #408:7871c124be6b0f18f9ad35530e0fe3cd +history 88235d7e7155ba860201a3185e7ca476349f597dc4a1062f54150000000000000005 #400:000fca4dc9b798b2943316131ef40ee7 +history 88235d7e7155ba860201a3185e7ca476349f597dc4a1062f54150000000000000006 #145:e77e3caac9db335fa5c7fd4aedb7f47b +history 88235d7e7155ba860201a3185e7ca476349f597dc4a1062f5416000000010000000000000001 =0801100122020801 +history 88235d7e7155ba860201a3185e7ca476349f597dc4a1062f5416000000010000000000000003 #134:5eba9fce85c59c832d3a33468dab512d +history 88235d7e7155ba860201a3185e7ca476349f597dc4a1062f5416000000010000000000000005 #213:53cf31c56d98e405246e344b1ba5add7 history 957748ea772a492d0201a3185ce1827fd69291cf63846a12f3150000000000000001 #393:a5706d57a699118e0b45412f75b8d67d history 957748ea772a492d0201a3185ce1827fd69291cf63846a12f3150000000000000002 #121:7dc2aefc1ddf60488d6abb2e757e1653 history 957748ea772a492d0201a3185ce1827fd69291cf63846a12f3150000000000000003 #297:94fea4a1ea446cb26713e1f5ce2644d2 @@ -387,6 +656,22 @@ history 9b749ea64ab099e80201a3185cea4c7d5982c3e7ad08b68172150000000000000002 #14 history 9b749ea64ab099e80201a3185cea4c7d5982c3e7ad08b68172150000000000000003 #419:4c7379ece2c41eefe68ec6fd0871147a history 9b749ea64ab099e80201a3185cea4c7d5982c3e7ad08b68172150000000000000004 #127:95e84b705457202aef3a5c57dc7a5188 history 9b749ea64ab099e80201a3185cea4c7d5982c3e7ad08b6817216000000010000000000000001 #224:4bf8ea34dcee365f7091af69238434c6 +history a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf16150000000000000001 #332:b54c06abee4a587427a10e85cb595d89 +history a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf16150000000000000017 #407:6c00e44e33d616b97ba1e50223f29d39 +history a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf16150000000000000018 #414:493a8d5749d79736cda515d999f7bf94 +history a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf16150000000000000019 #401:7fc749f3d5f7efe45d431ffb00339f3e +history a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1615000000000000001a #413:dafe4c73eb10c0837c652eb688a042c9 +history a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1615000000000000001b #407:d58e2c98aa656655edc6a8b1392abca8 +history a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1615000000000000001c #414:300b23ae65224a0a0fd83f31f1bdda8a +history a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1615000000000000001d #401:6b27deb39e1270783e18830521419e1b +history a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1615000000000000001e #413:e31ed3bbd774e1511b08184d5a006f58 +history a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1615000000000000001f #407:092eb9d35437d145aef72dde4e27d0ba +history a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1616000000010000000000000001 =0801100122020801 +history a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1616000000010000000000000017 #941:647c5f4d49968b4aa4b045801410b4a3 +history a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1616000000010000000000000019 #1022:a43966a70fb0dcb995791e2813f4e6f0 +history a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf161600000001000000000000001b #1103:2546c906426b1fe6cbdff2a973fbf5ec +history a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf161600000001000000000000001d #1184:08806de7c8a50cd56dd375c1d2dd684c +history a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf161600000001000000000000001f #1265:6ac437c2653e24fb5e19195dac14b9fa history bd8ac614675559d00201a3185e47e87c9ba5bc71cdf3a0f30a150000000000000001 #388:28d5bc7bd34f6d2e492284de0724246f history bd8ac614675559d00201a3185e47e87c9ba5bc71cdf3a0f30a16000000010000000000000001 #133:e918f86e9004d8053b88019b7fbed067 history cef57bddf594cff00201a3185d04aa7a8486e4fc3d7b333df3150000000000000001 #381:6405bffdc7fb8573dfa7192b4b8a3b9b @@ -411,6 +696,25 @@ history d879b97faa5ef0cf0201a3185c50007c4ab7515b188df65b7b15000000000000000e #44 history d879b97faa5ef0cf0201a3185c50007c4ab7515b188df65b7b15000000000000000f #115:8feedf101435a59fbbd603815c5bc84b history d879b97faa5ef0cf0201a3185c50007c4ab7515b188df65b7b150000000000000010 #496:cd39222ad67b4972f5b2e9b2f979f7c3 history d879b97faa5ef0cf0201a3185c50007c4ab7515b188df65b7b16000000010000000000000001 #251:95020fa0dca411c5b343b9eee3e9690a +history fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a150000000000000001 #349:5484531766b2e3975f4d13be826d86e8 +history fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a150000000000000017 #416:82f97b36a0f3efdebc459e3dbd98ee8f +history fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a150000000000000018 #417:a73b2c0d30d75e0a75f83e6e5a1dbe2e +history fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a150000000000000019 #415:03ad599ad1517b9a871eeea40435c7ad +history fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a15000000000000001a #423:ddff8d562f756dedc989e5b3139424ee +history fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a15000000000000001b #416:9392aca4f51e06f1c92df84601a01e1d +history fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a15000000000000001c #417:6dd491a80186c53412fdb7c34761c343 +history fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a15000000000000001d #415:61410b5fe3b555792d8a22a3a2dcac09 +history fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a15000000000000001e #423:8427a5b9d79f6c171eca4f2f35aedf07 +history fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a15000000000000001f #416:1c942c92646abbbc3d08acb92a0fe132 +history fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a150000000000000020 #422:297981a7358458bce464ecbc43624d70 +history fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a150000000000000021 #416:21a340a9f92836a640804cdb178689a4 +history fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a16000000010000000000000001 =0801100122020801 +history fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a16000000010000000000000017 #974:0e15e2ac63c42dc57bc33dc4e361d1df +history fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a16000000010000000000000019 #1058:48fcb9b079fc9b4934229f748f0561b5 +history fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a1600000001000000000000001b #1142:3d378f3c196aed474e3695160009f3fb +history fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a1600000001000000000000001d #1226:5408e2413bca538e1064bcf24512afa8 +history fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a1600000001000000000000001f #1310:cc6d57f0685f41b1b9966238abe95c5e +history fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a16000000010000000000000021 #1394:ceae986fbbce4cc241b0c84a0a49e5ec partition_index 0000018006652b48d23a000f0b8625e5f9724701a3185ce75e7c9ba774c48afeb866c032120a1001000000000000000300000000000000 = partition_index 000001800665a3fc0870300f0b8625e5f9724701a3185ce75e7c9ba774c48afeb866c02a00 = partition_index 000001800665a3fc2ac54030cdf0c23882674401a3185cf9ec7d97b2eddbc862722e0c2a00 = @@ -439,7 +743,9 @@ partition_index 00000a00000000000000000100094f72646572466c6f7701007ff99ae8d4b8da partition_index 00000a000000000000000001000a506172656e74466c6f7701007ff99ae8d4db2fcf7ff99ae8d514683f01a3185ce75e7c9ba774c48afeb866c00f0b8625e5f97247 = partition_index 00000a000000000000000001000b4163636f756e74466c6f77000100000000000000007ff99ae8d4fd84df01a3185ced3a76f5860d3203b7c6fd9f00c77b2df1ff0c81 = partition_index 00000a000000000000000001000b48616e646f7574466c6f7701007ff99ae8cfbebedf7ff99ae8d09fd00f01a3185e0b5c7f468a30ab8a1e4c9f300808577b47782b92 = +partition_index 00000f00c77b2df1ff0c8101a3185ced3a76f5860d3203b7c6fd9f = partition_index 00000f0808577b47782b9201a3185e0b5c7f468a30ab8a1e4c9f30 = +partition_index 00000f0f0b8625e5f9724701a3185ce75e7c9ba774c48afeb866c0 = partition_index 00000f30cdf0c23882674401a3185cf9ec7d97b2eddbc862722e0c = partition_index 000101800665a3fc6f6f604a90b11d2e1acfc701a3185d08927201ac6ac5c708d6402b2a00 = partition_index 000101800665a3fc8652c04174ea676bdbb0f301a3185d115c786fbaaa7a89714731b52a00 = @@ -488,46 +794,113 @@ partition_index 00010e =0801 partition_index 00010f4174ea676bdbb0f301a3185d115c786fbaaa7a89714731b5 = partition_index 00010f4a90b11d2e1acfc701a3185d08927201ac6ac5c708d6402b = partition_index 00010f55e03d097c903c0b01a3185e4dc47ee5962f06ca9e6d8ec6 = +partition_index 0002018006651734c14b0088235d7e7155ba8601a3185e7ca476349f597dc4a1062f542a00 = partition_index 0002018006652b48f48f109b749ea64ab099e801a3185cea4c7d5982c3e7ad08b6817232120a1002000000000000000300000000000000 = partition_index 000201800665a3fbf92df09b749ea64ab099e801a3185cea4c7d5982c3e7ad08b681722a00 = partition_index 000201800665a3fc326660957748ea772a492d01a3185ce1827fd69291cf63846a12f32a00 = partition_index 000202000000000000000000094368696c64466c6f77029b749ea64ab099e801a3185cea4c7d5982c3e7ad08b68172 = partition_index 000202000000000000000000095175657565466c6f7701bd8ac614675559d001a3185e47e87c9ba5bc71cdf3a0f30a = +partition_index 0002020000000000000000000a4c6567616379466c6f7701a8ecaec88da2f4f001a3185e5e5e7db7b414da763fa4bf16 = +partition_index 0002020000000000000000000a4c6567616379466c6f770688235d7e7155ba8601a3185e7ca476349f597dc4a1062f54 = partition_index 0002020000000000000000000b50726f66696c65466c6f7702957748ea772a492d01a3185ce1827fd69291cf63846a12f3 = +partition_index 000203000000000000000088235d7e7155ba8601a3185e7ca476349f597dc4a1062f54 =0000000000000006 partition_index 0002030000000000000000957748ea772a492d01a3185ce1827fd69291cf63846a12f3 =0000000000000009 partition_index 00020300000000000000009b749ea64ab099e801a3185cea4c7d5982c3e7ad08b68172 =0000000000000003 +partition_index 0002030000000000000000a8ecaec88da2f4f001a3185e5e5e7db7b414da763fa4bf16 =0000000000000001 partition_index 0002030000000000000000bd8ac614675559d001a3185e47e87c9ba5bc71cdf3a0f30a =0000000000000001 +partition_index 000204000000000000000088235d7e7155ba8601a3185e7ca476349f597dc4a1062f54 = partition_index 0002040000000000000000957748ea772a492d01a3185ce1827fd69291cf63846a12f3 = partition_index 00020400000000000000009b749ea64ab099e801a3185cea4c7d5982c3e7ad08b68172 = +partition_index 0002040000000000000000a8ecaec88da2f4f001a3185e5e5e7db7b414da763fa4bf16 = partition_index 0002040000000000000000bd8ac614675559d001a3185e47e87c9ba5bc71cdf3a0f30a = +partition_index 0002078006651734c14b0088235d7e7155ba8601a3185e7ca476349f597dc4a1062f54 = partition_index 000207800665a3fbf92df09b749ea64ab099e801a3185cea4c7d5982c3e7ad08b68172 = partition_index 000207800665a3fc326660957748ea772a492d01a3185ce1827fd69291cf63846a12f3 = +partition_index 000208a8ecaec88da2f4f001a3185e5e5e7db7b414da763fa4bf16 = partition_index 000208bd8ac614675559d001a3185e47e87c9ba5bc71cdf3a0f30a = +partition_index 00020a000000000000000000000100000000000000007ff99ae8cf5b903f01a3185e5e5e7db7b414da763fa4bf16a8ecaec88da2f4f0 = partition_index 00020a000000000000000000000100000000000000007ff99ae8cfb34d2f01a3185e47e87c9ba5bc71cdf3a0f30abd8ac614675559d0 = +partition_index 00020a00000000000000000001007ff99ae8ced23bff7ff99ae8cee54ecf01a3185e7ca476349f597dc4a1062f5488235d7e7155ba86 = partition_index 00020a00000000000000000001007ff99ae8d4b1399f7ff99ae8d52b4b9f01a3185ce1827fd69291cf63846a12f3957748ea772a492d = partition_index 00020a00000000000000000001007ff99ae8d4ea720f7ff99ae8d508f68f01a3185cea4c7d5982c3e7ad08b681729b749ea64ab099e8 = partition_index 00020a00000000000000000100094368696c64466c6f7701007ff99ae8d4ea720f7ff99ae8d508f68f01a3185cea4c7d5982c3e7ad08b681729b749ea64ab099e8 = partition_index 00020a00000000000000000100095175657565466c6f77000100000000000000007ff99ae8cfb34d2f01a3185e47e87c9ba5bc71cdf3a0f30abd8ac614675559d0 = +partition_index 00020a000000000000000001000a4c6567616379466c6f77000100000000000000007ff99ae8cf5b903f01a3185e5e5e7db7b414da763fa4bf16a8ecaec88da2f4f0 = +partition_index 00020a000000000000000001000a4c6567616379466c6f7701007ff99ae8ced23bff7ff99ae8cee54ecf01a3185e7ca476349f597dc4a1062f5488235d7e7155ba86 = partition_index 00020a000000000000000001000b50726f66696c65466c6f7701007ff99ae8d4b1399f7ff99ae8d52b4b9f01a3185ce1827fd69291cf63846a12f3957748ea772a492d = partition_index 00020d000000000000000000095175657565466c6f7700085061636b537465700180066517304cb2d0bd8ac614675559d001a3185e47e87c9ba5bc71cdf3a0f30a02000000015061636b537465702d31 =08011001 partition_index 000301800665a3fc13e1e0d879b97faa5ef0cf01a3185c50007c4ab7515b188df65b7b2a00 = partition_index 000301800665a3fc4d1a50cef57bddf594cff001a3185d04aa7a8486e4fc3d7b333df32a00 = partition_index 000302000000000000000000094f72646572466c6f7702d879b97faa5ef0cf01a3185c50007c4ab7515b188df65b7b = +partition_index 0003020000000000000000000a4c6567616379466c6f7701fe9d7958c1defd0d01a3185e82807eb09cd50af4d0a22d3a = partition_index 0003020000000000000000000d48656c64436c6f7365466c6f7702cef57bddf594cff001a3185d04aa7a8486e4fc3d7b333df3 = partition_index 0003030000000000000000cef57bddf594cff001a3185d04aa7a8486e4fc3d7b333df3 =0000000000000004 partition_index 0003030000000000000000d879b97faa5ef0cf01a3185c50007c4ab7515b188df65b7b =0000000000000010 +partition_index 0003030000000000000000fe9d7958c1defd0d01a3185e82807eb09cd50af4d0a22d3a =0000000000000001 partition_index 0003040000000000000000cef57bddf594cff001a3185d04aa7a8486e4fc3d7b333df3 = partition_index 0003040000000000000000d879b97faa5ef0cf01a3185c50007c4ab7515b188df65b7b = +partition_index 0003040000000000000000fe9d7958c1defd0d01a3185e82807eb09cd50af4d0a22d3a = partition_index 000307800665a3fc13e1e0d879b97faa5ef0cf01a3185c50007c4ab7515b188df65b7b = partition_index 000307800665a3fc4d1a50cef57bddf594cff001a3185d04aa7a8486e4fc3d7b333df3 = +partition_index 000308fe9d7958c1defd0d01a3185e82807eb09cd50af4d0a22d3a = +partition_index 00030a000000000000000000000100000000000000007ff99ae8cece6b6f01a3185e82807eb09cd50af4d0a22d3afe9d7958c1defd0d = partition_index 00030a00000000000000000001007ff99ae8d49685af7ff99ae8d4a1f75f01a3185d04aa7a8486e4fc3d7b333df3cef57bddf594cff0 = partition_index 00030a00000000000000000001007ff99ae8d4cfbe1f7ff99ae8d763af6f01a3185c50007c4ab7515b188df65b7bd879b97faa5ef0cf = partition_index 00030a00000000000000000100094f72646572466c6f7701007ff99ae8d4cfbe1f7ff99ae8d763af6f01a3185c50007c4ab7515b188df65b7bd879b97faa5ef0cf = +partition_index 00030a000000000000000001000a4c6567616379466c6f77000100000000000000007ff99ae8cece6b6f01a3185e82807eb09cd50af4d0a22d3afe9d7958c1defd0d = partition_index 00030a000000000000000001000d48656c64436c6f7365466c6f7701007ff99ae8d49685af7ff99ae8d4a1f75f01a3185d04aa7a8486e4fc3d7b333df3cef57bddf594cff0 = +partition_index 00030e =0801 +partition_index 00030ffe9d7958c1defd0d01a3185e82807eb09cd50af4d0a22d3a = partition_meta 0000 =18b0be8285f3a2990320013001 partition_meta 0001 =080118a09df384f3a2990320013001 -partition_meta 0002 =080218e086c282f3a2990320013001 -partition_meta 0003 =080318d0f4a5dbf2a2990320013001 +partition_meta 0002 =0802188088b789f3a2990320013001 +partition_meta 0003 =080318a0eebd8df3a2990320013001 value 7becb50ef3da384a0201a3185e588277649c53bb00e0d05cb810000000000000000200000000 #150:582bacc585ea0c8aa8ac6c9592ba64a3 value 7becb50ef3da384a0201a3185e588277649c53bb00e0d05cb810000000000000000300000000 #162:f5da54fe0d1772299e0c1fe67b606a81 +value 88235d7e7155ba860201a3185e7ca476349f597dc4a1062f5410000000000000000200000000 #159:722f48848ce256b7cc3b2e71e7e2b271 +value 88235d7e7155ba860201a3185e7ca476349f597dc4a1062f5410000000000000000300000000 #159:900ecdf80a33e023538507d0a64e1ed2 +value 88235d7e7155ba860201a3185e7ca476349f597dc4a1062f5410000000000000000400000000 #159:260047fd7371b2640a4838714625f03b +value 88235d7e7155ba860201a3185e7ca476349f597dc4a1062f5410000000000000000500000000 #159:87e10e25b69fbcbc91502f9278c29432 +value a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1610000000000000000300000000 #159:900ecdf80a33e023538507d0a64e1ed2 +value a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1610000000000000000500000000 #159:87e10e25b69fbcbc91502f9278c29432 +value a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1610000000000000000700000000 #159:2beee7176e619c399cbaefdb9adf415f +value a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1610000000000000000900000000 #159:8ac2c06050e829ea5046052d2091eec0 +value a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1610000000000000000b00000000 #159:dbdff555924c27afadb6cee6193e3a75 +value a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1610000000000000000d00000000 #160:110975bb3586c9c0587fe1959f41fd5a +value a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1610000000000000000f00000000 #160:c166fbd31e2ee7f4ce791121bdb0a0bc +value a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1610000000000000001100000000 #160:679237781d60ce2372f7bf9302da21b0 +value a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1610000000000000001300000000 #160:b10aa76b0bfdc063aecf81dfb6481805 +value a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1610000000000000001500000000 #160:6fcb29e8a0c9b6868396a928e87daa31 +value a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1610000000000000001600000000 #160:ab1a5c2a3772b2fc9373a25389609562 +value a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1610000000000000001700000000 #160:b7277bd9fa11ab1f5724e6d4b3cf2eaa +value a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1610000000000000001800000000 #160:82c8cfc71223e266d22e4f828c534d84 +value a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1610000000000000001900000000 #160:93cff5a299018f6e1c82a7cdab167974 +value a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1610000000000000001a00000000 #160:6db824991e996c80e7e82db91c519a0a +value a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1610000000000000001b00000000 #160:88aaa4a95b39187e720d617998bdb418 +value a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1610000000000000001c00000000 #160:d60f162ba4dfb149fdeab48f5434373f +value a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1610000000000000001d00000000 #160:da2fe20d16522a8c3071ed69a43c8d1a +value a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1610000000000000001e00000000 #160:18349e018f41cf8f4c41e005751e4695 +value a8ecaec88da2f4f00201a3185e5e5e7db7b414da763fa4bf1610000000000000001f00000000 #160:6823f3181b93574c60f72fb6819ce908 value d879b97faa5ef0cf0201a3185c50007c4ab7515b188df65b7b10000000000000000100000000 #150:582bacc585ea0c8aa8ac6c9592ba64a3 +value fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a10000000000000000300000000 #159:900ecdf80a33e023538507d0a64e1ed2 +value fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a10000000000000000500000000 #159:87e10e25b69fbcbc91502f9278c29432 +value fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a10000000000000000700000000 #159:2beee7176e619c399cbaefdb9adf415f +value fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a10000000000000000900000000 #159:8ac2c06050e829ea5046052d2091eec0 +value fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a10000000000000000b00000000 #159:dbdff555924c27afadb6cee6193e3a75 +value fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a10000000000000000d00000000 #160:110975bb3586c9c0587fe1959f41fd5a +value fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a10000000000000000f00000000 #160:c166fbd31e2ee7f4ce791121bdb0a0bc +value fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a10000000000000001100000000 #160:679237781d60ce2372f7bf9302da21b0 +value fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a10000000000000001300000000 #160:b10aa76b0bfdc063aecf81dfb6481805 +value fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a10000000000000001500000000 #160:6fcb29e8a0c9b6868396a928e87daa31 +value fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a10000000000000001600000000 #160:ab1a5c2a3772b2fc9373a25389609562 +value fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a10000000000000001700000000 #160:b7277bd9fa11ab1f5724e6d4b3cf2eaa +value fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a10000000000000001800000000 #160:82c8cfc71223e266d22e4f828c534d84 +value fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a10000000000000001900000000 #160:93cff5a299018f6e1c82a7cdab167974 +value fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a10000000000000001a00000000 #160:6db824991e996c80e7e82db91c519a0a +value fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a10000000000000001b00000000 #160:88aaa4a95b39187e720d617998bdb418 +value fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a10000000000000001c00000000 #160:d60f162ba4dfb149fdeab48f5434373f +value fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a10000000000000001d00000000 #160:da2fe20d16522a8c3071ed69a43c8d1a +value fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a10000000000000001e00000000 #160:18349e018f41cf8f4c41e005751e4695 +value fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a10000000000000001f00000000 #160:6823f3181b93574c60f72fb6819ce908 +value fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a10000000000000002000000000 #160:4ac4fa9a41bee38b5daa4795084e603d +value fe9d7958c1defd0d0201a3185e82807eb09cd50af4d0a22d3a10000000000000002100000000 #160:7fcf286b4655109e81722239eb8e51a8