diff --git a/sdk/cosmos/azure_data_cosmos/CHANGELOG.md b/sdk/cosmos/azure_data_cosmos/CHANGELOG.md index 84c1b4e2e55..6b7665e7573 100644 --- a/sdk/cosmos/azure_data_cosmos/CHANGELOG.md +++ b/sdk/cosmos/azure_data_cosmos/CHANGELOG.md @@ -25,6 +25,7 @@ ### Breaking Changes +- `ContainerClient::patch_item()` and `PatchItemOptions` are now gated behind the new, non-default `preview_patch` feature. PATCH is not exactly-once under transport failures: an interrupted patch may re-apply non-idempotent operations (`increment`, `add` on an array, `move`), so it is no longer part of the supported default surface. Enable `preview_patch` to keep using it, and prefer idempotent operations (`set` on a caller-computed value) until the guarantee is fixed. ([#5133](https://github.com/Azure/azure-sdk-for-rust/pull/5133)) - `CosmosClient::database_client` and `DatabaseClient::container_client` now take `impl Into` instead of `&str`; call sites passing a deref-able string (for example a `Cow` field) need `&*value` or `.as_ref()`. ([#4687](https://github.com/Azure/azure-sdk-for-rust/pull/4687)) - `DatabaseClient::id()` now returns `&ResourceIdentity` instead of `&str`. ([#4687](https://github.com/Azure/azure-sdk-for-rust/pull/4687)) - Control-plane APIs are now gated behind the new `control_plane` feature, which is **not** enabled by default. Code using database or container management (`CosmosClient::create_database`/`query_databases`, `DatabaseClient::read`/`create_container`/`query_containers`/`delete`, `ContainerClient::replace`/`delete`), throughput management (`read_throughput`/`begin_replace_throughput`, `ThroughputPoller`), or the associated model and options types (`DatabaseProperties`, `ThroughputProperties`, and the container create/replace/delete/query, database, and throughput option types) must now enable the `control_plane` feature. Reading container properties via `ContainerClient::read()` — along with `ContainerProperties`, `IndexingPolicy`, `ResourceResponse`, and `ReadContainerOptions` — remains available without the feature, since it works with Entra ID authentication and mirrors the metadata read the SDK already performs internally. ([#4854](https://github.com/Azure/azure-sdk-for-rust/pull/4854)) diff --git a/sdk/cosmos/azure_data_cosmos/Cargo.toml b/sdk/cosmos/azure_data_cosmos/Cargo.toml index 3e90e1aa5e6..146611d8b3d 100644 --- a/sdk/cosmos/azure_data_cosmos/Cargo.toml +++ b/sdk/cosmos/azure_data_cosmos/Cargo.toml @@ -106,6 +106,7 @@ metrics = [ preview_dtx = [ "azure_data_cosmos_driver/preview_dtx", ] # Enables preview Distributed Transaction APIs. Disabled by default and not production-ready. +preview_patch = [] # Enables the preview PATCH API. Disabled by default and not production-ready: an interrupted patch may re-apply non-idempotent operations. __internal_in_memory_emulator = [ "azure_data_cosmos_driver/__internal_in_memory_emulator", "key_auth", @@ -123,6 +124,7 @@ features = [ "key_auth", "metrics", "native_tls", + "preview_patch", "rustls", ] @@ -172,3 +174,10 @@ required-features = ["__internal_in_memory_emulator"] name = "cosmos" path = "examples/cosmos/main.rs" required-features = ["control_plane"] + +# The PATCH example calls the preview `patch_item` API, which only exists +# when `preview_patch` is enabled. +[[example]] +name = "cosmos_patch" +path = "examples/cosmos_patch.rs" +required-features = ["preview_patch"] diff --git a/sdk/cosmos/azure_data_cosmos/README.md b/sdk/cosmos/azure_data_cosmos/README.md index 794142038ca..c7abbd5a480 100644 --- a/sdk/cosmos/azure_data_cosmos/README.md +++ b/sdk/cosmos/azure_data_cosmos/README.md @@ -107,7 +107,6 @@ container metadata read the SDK already performs internally. ```rust use serde::{Serialize, Deserialize}; use azure_data_cosmos::CosmosClient; -use azure_data_cosmos::models::{PatchInstructions, PatchOperation}; #[derive(Serialize, Deserialize)] struct Item { @@ -137,21 +136,39 @@ async fn example(cosmos_client: CosmosClient) -> Result<(), Box`. The same struct type is reused at every explicit layer (runtime, account, operation) it participates in. Resolution walks from the highest-priority layer downward, returning the first `Some` value. @@ -85,10 +85,10 @@ incidents — flipping a feature off (or on) fleet-wide without a code change or redeploy — and should normally be left unset. Today the following options expose an override: -| Base env var | Kill switch | Effect when set | -| --- | --- | --- | -| `AZURE_COSMOS_HEDGING_ENABLED` | `AZURE_COSMOS_HEDGING_ENABLED_OVERRIDE` | Forces cross-region read hedging on/off regardless of any programmatic `AvailabilityStrategy` or per-request value. | -| `AZURE_COSMOS_PPCB_ENABLED` | `AZURE_COSMOS_PPCB_ENABLED_OVERRIDE` | Forces the per-partition circuit breaker (PPCB) on/off regardless of the `PartitionFailoverOptions` setting **and** the account property `enable_per_partition_failover_behavior`. | +| Base env var | Kill switch | Effect when set | +| -------------------------------------------------- | ----------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `AZURE_COSMOS_HEDGING_ENABLED` | `AZURE_COSMOS_HEDGING_ENABLED_OVERRIDE` | Forces cross-region read hedging on/off regardless of any programmatic `AvailabilityStrategy` or per-request value. | +| `AZURE_COSMOS_PPCB_ENABLED` | `AZURE_COSMOS_PPCB_ENABLED_OVERRIDE` | Forces the per-partition circuit breaker (PPCB) on/off regardless of the `PartitionFailoverOptions` setting **and** the account property `enable_per_partition_failover_behavior`. | | `AZURE_COSMOS_CONNECTION_POOL_GATEWAY_V2_DISABLED` | `AZURE_COSMOS_CONNECTION_POOL_GATEWAY_V2_DISABLED_OVERRIDE` | Forces the runtime to allow or suppress Gateway V2 regardless of the normal `ConnectionPoolOptions` value. It cannot bypass HTTP/2 availability, endpoint advertisement, or connectivity probing. | > **Note on the PPCB kill switch.** Unlike the hedging switch, PPCB enablement @@ -201,14 +201,14 @@ Cross-layer options that apply to individual service requests. This is the most pub struct OperationOptions { /* fields below */ } ``` -| Option | Type | Env Var | Notes | -| --- | --- | --- | --- | -| `read_consistency_strategy` | `Option` | `AZURE_COSMOS_READ_CONSISTENCY_STRATEGY` | Read consistency for the operation. Replaces the legacy `consistency_level` field. The SDK enforces weakening-only semantics relative to the account default. | -| `excluded_regions` | `Option>` | `AZURE_COSMOS_EXCLUDED_REGIONS` | Regions to exclude from routing. `None` inherits from a lower layer; `Some(vec![])` explicitly clears exclusions. Env var is comma-separated (e.g. `"West US,East US"`). | -| `content_response_on_write` | `Option` | `AZURE_COSMOS_CONTENT_RESPONSE_ON_WRITE` | Whether write operations return the resource body in the response. Only applicable to write operations; ignored by reads and queries. Cascades from runtime → account → operation, matching .NET/Java/Go behavior. | -| `hedging_enabled` | `Option` | `AZURE_COSMOS_HEDGING_ENABLED` | Master switch for cross-region read hedging. `None` (unset) means **no env override**: resolution defers to the programmatic `AvailabilityStrategy` (which may explicitly disable hedging). Only when neither the env switch nor a programmatic strategy is set does the driver default to hedging **on**, using the built-in default threshold of `min(1000ms, request_timeout / 2)`. When set, it is the **source of truth** and takes precedence over the programmatic `AvailabilityStrategy` in both directions: `false` disables hedging even when an explicit `AvailabilityStrategy::Hedging(..)` is configured, and `true` enables hedging even when an explicit `AvailabilityStrategy::Disabled` is configured (a programmatic `Hedging(..)` still supplies its custom threshold). Also recognizes the `AZURE_COSMOS_HEDGING_ENABLED_OVERRIDE` kill switch (see [`_OVERRIDE` kill switches](#_override-kill-switches)), which wins over every layer including a per-request value. | -| `throttling_retry_options.max_retry_count` | `Option` | `AZURE_COSMOS_MAX_THROTTLE_RETRY_COUNT` | Maximum number of retries when a request is throttled (HTTP 429, rate-limited) before the error surfaces to the caller. Field of the nested `ThrottlingRetryOptions` group. Analogous to .NET's `MaxRetryAttemptsOnRateLimitedRequests`. Defaults to `9` when unset; `0` disables throttle retries entirely. **Scope**: applies per transport-pipeline invocation, not per logical operation — an operation that fans out across regions (failover, hedging) gets a fresh budget per leg. Use `end_to_end_latency_policy` to bound total per-operation time. Settable client-wide via [`CosmosClientBuilder::with_default_operation_options`]. | -| `throttling_retry_options.max_retry_wait_time` | `Option` | — | Maximum cumulative time to spend waiting across throttle (HTTP 429) retries before the error surfaces. Field of the nested `ThrottlingRetryOptions` group. Analogous to .NET's `MaxRetryWaitTimeOnRateLimitedRequests`. Defaults to `30s` when unset. No env var because `Duration` is not parseable from a single string. **Scope**: same per-invocation scope as `max_retry_count` — not a per-operation cap. Settable client-wide via [`CosmosClientBuilder::with_default_operation_options`]. | +| Option | Type | Env Var | Notes | +| ---------------------------------------------- | --------------------------------- | ---------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| `read_consistency_strategy` | `Option` | `AZURE_COSMOS_READ_CONSISTENCY_STRATEGY` | Read consistency for the operation. Replaces the legacy `consistency_level` field. The SDK enforces weakening-only semantics relative to the account default. | +| `excluded_regions` | `Option>` | `AZURE_COSMOS_EXCLUDED_REGIONS` | Regions to exclude from routing. `None` inherits from a lower layer; `Some(vec![])` explicitly clears exclusions. Env var is comma-separated (e.g. `"West US,East US"`). | +| `content_response_on_write` | `Option` | `AZURE_COSMOS_CONTENT_RESPONSE_ON_WRITE` | Whether write operations return the resource body in the response. Only applicable to write operations; ignored by reads and queries. Cascades from runtime → account → operation, matching .NET/Java/Go behavior. | +| `hedging_enabled` | `Option` | `AZURE_COSMOS_HEDGING_ENABLED` | Master switch for cross-region read hedging. `None` (unset) means **no env override**: resolution defers to the programmatic `AvailabilityStrategy` (which may explicitly disable hedging). Only when neither the env switch nor a programmatic strategy is set does the driver default to hedging **on**, using the built-in default threshold of `min(1000ms, request_timeout / 2)`. When set, it is the **source of truth** and takes precedence over the programmatic `AvailabilityStrategy` in both directions: `false` disables hedging even when an explicit `AvailabilityStrategy::Hedging(..)` is configured, and `true` enables hedging even when an explicit `AvailabilityStrategy::Disabled` is configured (a programmatic `Hedging(..)` still supplies its custom threshold). Also recognizes the `AZURE_COSMOS_HEDGING_ENABLED_OVERRIDE` kill switch (see [`_OVERRIDE` kill switches](#_override-kill-switches)), which wins over every layer including a per-request value. | +| `throttling_retry_options.max_retry_count` | `Option` | `AZURE_COSMOS_MAX_THROTTLE_RETRY_COUNT` | Maximum number of retries when a request is throttled (HTTP 429, rate-limited) before the error surfaces to the caller. Field of the nested `ThrottlingRetryOptions` group. Analogous to .NET's `MaxRetryAttemptsOnRateLimitedRequests`. Defaults to `9` when unset; `0` disables throttle retries entirely. **Scope**: applies per transport-pipeline invocation, not per logical operation — an operation that fans out across regions (failover, hedging) gets a fresh budget per leg. Use `end_to_end_latency_policy` to bound total per-operation time. Settable client-wide via [`CosmosClientBuilder::with_default_operation_options`]. | +| `throttling_retry_options.max_retry_wait_time` | `Option` | — | Maximum cumulative time to spend waiting across throttle (HTTP 429) retries before the error surfaces. Field of the nested `ThrottlingRetryOptions` group. Analogous to .NET's `MaxRetryWaitTimeOnRateLimitedRequests`. Defaults to `30s` when unset. No env var because `Duration` is not parseable from a single string. **Scope**: same per-invocation scope as `max_retry_count` — not a per-operation cap. Settable client-wide via [`CosmosClientBuilder::with_default_operation_options`]. | ### 3.2 `ConnectionOptions` @@ -222,10 +222,10 @@ Options controlling network connection behavior. Not available at the operation pub struct ConnectionOptions { /* fields below */ } ``` -| Option | Type | Env Var | Notes | -| --- | --- | --- | --- | -| `request_timeout` | `Option` | `AZURE_COSMOS_REQUEST_TIMEOUT` | Per-request network timeout. | -| `connection_pool` | `Option` | — | Nested group for connection pool tuning. Marked `#[option(nested)]`. | +| Option | Type | Env Var | Notes | +| ----------------- | ------------------------------- | ------------------------------ | -------------------------------------------------------------------- | +| `request_timeout` | `Option` | `AZURE_COSMOS_REQUEST_TIMEOUT` | Per-request network timeout. | +| `connection_pool` | `Option` | — | Nested group for connection pool tuning. Marked `#[option(nested)]`. | ### 3.3 `ConnectionPoolOptions` @@ -239,11 +239,11 @@ Fine-grained connection pool tuning. Nested via `#[option(nested)]` on `Connecti pub struct ConnectionPoolOptions { /* fields below */ } ``` -| Option | Type | Env Var | Notes | -| --- | --- | --- | --- | -| `idle_timeout` | `Option` | `AZURE_COSMOS_POOL_IDLE_TIMEOUT` | How long idle connections are kept alive. | -| `max_connections` | `Option` | `AZURE_COSMOS_POOL_MAX_CONNECTIONS` | Maximum number of connections in the pool. | -| `gateway_v2_disabled` | `bool` | `AZURE_COSMOS_CONNECTION_POOL_GATEWAY_V2_DISABLED` | Runtime-scoped Gateway V2 opt-out configured with `ConnectionPoolOptionsBuilder::with_gateway_v2_disabled`; defaults to `false`. The generated `_OVERRIDE` variant is authoritative over the builder and base environment value, but cannot bypass HTTP/2 or server eligibility when set to `false`. | +| Option | Type | Env Var | Notes | +| --------------------- | ------------------ | -------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `idle_timeout` | `Option` | `AZURE_COSMOS_POOL_IDLE_TIMEOUT` | How long idle connections are kept alive. | +| `max_connections` | `Option` | `AZURE_COSMOS_POOL_MAX_CONNECTIONS` | Maximum number of connections in the pool. | +| `gateway_v2_disabled` | `bool` | `AZURE_COSMOS_CONNECTION_POOL_GATEWAY_V2_DISABLED` | Runtime-scoped Gateway V2 opt-out configured with `ConnectionPoolOptionsBuilder::with_gateway_v2_disabled`; defaults to `false`. The generated `_OVERRIDE` variant is authoritative over the builder and base environment value, but cannot bypass HTTP/2 or server eligibility when set to `false`. | ### 3.4 `RegionOptions` @@ -257,8 +257,8 @@ Options controlling region selection and routing. Not available at the operation pub struct RegionOptions { /* fields below */ } ``` -| Option | Type | Env Var | Notes | -| --- | --- | --- | --- | +| Option | Type | Env Var | Notes | +| -------------------- | -------------------- | --------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `application_region` | `Option` | `AZURE_COSMOS_APPLICATION_REGION` | The region where the application is running. The SDK and backend negotiate optimal region ordering from this location. Only one of `application_region` should be set (the old `preferred_regions` / `application_preferred_regions` list is removed). | ### 3.5 `RetryOptions` @@ -273,9 +273,9 @@ Options controlling retry behavior. Not available at the operation layer because pub struct RetryOptions { /* fields below */ } ``` -| Option | Type | Env Var | Notes | -| --- | --- | --- | --- | -| `session_retry` | `Option` | — | Nested group for session-consistency retry behavior on 404/1002 errors. Marked `#[option(nested)]`. | +| Option | Type | Env Var | Notes | +| --------------- | ----------------------------- | ------- | --------------------------------------------------------------------------------------------------- | +| `session_retry` | `Option` | — | Nested group for session-consistency retry behavior on 404/1002 errors. Marked `#[option(nested)]`. | ### 3.6 `SessionRetryOptions` @@ -289,10 +289,10 @@ Controls retry behavior for 404/1002 (session not found) errors. Nested via `#[o pub struct SessionRetryOptions { /* fields below */ } ``` -| Option | Type | Env Var | Notes | -| --- | --- | --- | --- | -| `min_in_region_retry_time` | `Option` | `AZURE_COSMOS_SESSION_RETRY_MIN_IN_REGION_TIME` | Minimum time spent retrying within the local region before considering a cross-region retry. | -| `max_in_region_retry_count` | `Option` | `AZURE_COSMOS_SESSION_RETRY_MAX_IN_REGION_COUNT` | Maximum number of retries within the local region. | +| Option | Type | Env Var | Notes | +| --------------------------- | ------------------ | ------------------------------------------------ | -------------------------------------------------------------------------------------------- | +| `min_in_region_retry_time` | `Option` | `AZURE_COSMOS_SESSION_RETRY_MIN_IN_REGION_TIME` | Minimum time spent retrying within the local region before considering a cross-region retry. | +| `max_in_region_retry_count` | `Option` | `AZURE_COSMOS_SESSION_RETRY_MAX_IN_REGION_COUNT` | Maximum number of retries within the local region. | > **Migration note:** The current `SessionRetryOptions` struct has non-`Option` fields with concrete defaults (`min_in_region_retry_time: Duration`, etc.). In the new model, all fields become `Option` to support layered resolution. The concrete defaults are applied at resolution time when all layers yield `None`. @@ -308,11 +308,11 @@ Per-account settings that don't fit other groups. Not available at the operation pub struct CosmosAccountOptions { /* fields below */ } ``` -| Option | Type | Env Var | Notes | -| --- | --- | --- | --- | -| `user_agent_suffix` | `Option` | `AZURE_COSMOS_USER_AGENT_SUFFIX` | Application identifier appended to the User-Agent header for telemetry. | -| `account_initialization_custom_endpoints` | `Option>` | `AZURE_COSMOS_CUSTOM_ENDPOINTS` | Custom endpoints for initial account discovery (private endpoints, etc.). Env var is comma-separated. | -| `custom_headers` | `Option>` | — | **Best-effort only.** Additional HTTP headers injected into outgoing requests. Intended for proxies, gateways, or external telemetry systems — **not** for setting Cosmos DB backend headers. The SDK may use non-standard transports (e.g., custom framing over TCP) where HTTP headers do not apply; in those cases custom headers are silently ignored. The SDK reserves the right to override any header that conflicts with its internal protocol. `None` inherits from a lower layer; `Some(map)` replaces (does not merge) the inherited value. No environment variable — headers are not representable as a single string. | +| Option | Type | Env Var | Notes | +| ----------------------------------------- | ------------------------------------------ | -------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `user_agent_suffix` | `Option` | `AZURE_COSMOS_USER_AGENT_SUFFIX` | Application identifier appended to the User-Agent header for telemetry. | +| `account_initialization_custom_endpoints` | `Option>` | `AZURE_COSMOS_CUSTOM_ENDPOINTS` | Custom endpoints for initial account discovery (private endpoints, etc.). Env var is comma-separated. | +| `custom_headers` | `Option>` | — | **Best-effort only.** Additional HTTP headers injected into outgoing requests. Intended for proxies, gateways, or external telemetry systems — **not** for setting Cosmos DB backend headers. The SDK may use non-standard transports (e.g., custom framing over TCP) where HTTP headers do not apply; in those cases custom headers are silently ignored. The SDK reserves the right to override any header that conflicts with its internal protocol. `None` inherits from a lower layer; `Some(map)` replaces (does not merge) the inherited value. No environment variable — headers are not representable as a single string. | --- @@ -381,15 +381,15 @@ pub struct ItemReadOptions { } ``` -| Option | Type | Notes | -| --- | --- | --- | -| `operation` | `OperationOptions` | Layered group; fields resolve through Operation → Account → Runtime → Env. | -| `session_token` | `Option` | Session token for session-consistent reads. Operation-only. | -| `precondition` | `Option` | Conditional ETag check. For reads, typically `IfNoneMatch` (returns 304 Not Modified if unchanged). Operation-only. | +| Option | Type | Notes | +| --------------- | ---------------------- | ------------------------------------------------------------------------------------------------------------------- | +| `operation` | `OperationOptions` | Layered group; fields resolve through Operation → Account → Runtime → Env. | +| `session_token` | `Option` | Session token for session-consistent reads. Operation-only. | +| `precondition` | `Option` | Conditional ETag check. For reads, typically `IfNoneMatch` (returns 304 Not Modified if unchanged). Operation-only. | ### 5.2 `ItemWriteOptions` -Options for item write operations (`create_item`, `replace_item`, `upsert_item`, `delete_item`, `patch_item`). +Options for item write operations (`create_item`, `replace_item`, `upsert_item`, `delete_item`). ```rust #[derive(Clone, Default)] @@ -404,11 +404,11 @@ pub struct ItemWriteOptions { } ``` -| Option | Type | Notes | -| --- | --- | --- | -| `operation` | `OperationOptions` | Layered group; `content_response_on_write` is resolved here and applied to write responses. | -| `session_token` | `Option` | Session token for session-consistent writes. Operation-only. | -| `precondition` | `Option` | Conditional ETag check. For writes, typically `IfMatch` (optimistic concurrency). Operation-only. | +| Option | Type | Notes | +| --------------- | ---------------------- | ------------------------------------------------------------------------------------------------- | +| `operation` | `OperationOptions` | Layered group; `content_response_on_write` is resolved here and applied to write responses. | +| `session_token` | `Option` | Session token for session-consistent writes. Operation-only. | +| `precondition` | `Option` | Conditional ETag check. For writes, typically `IfMatch` (optimistic concurrency). Operation-only. | ### 5.3 `QueryOptions` @@ -429,13 +429,13 @@ pub struct QueryOptions { } ``` -| Option | Type | Notes | -| --- | --- | --- | -| `operation` | `OperationOptions` | Layered group; `content_response_on_write` is ignored for queries. | -| `session_token` | `Option` | Session token for session-consistent queries. Operation-only. | -| `enable_scan_if_no_index` | `Option` | If the query can't be served by indexes because the relevant paths are not indexed, setting this permits the query engine to perform a full container scan. Operation-only. | -| `populate_index_metrics` | `Option` | If set to `true`, the response will contain metrics regarding indexes used. Operation-only. | -| `populate_query_advice` | `Option` | If set to `true`, the response will include query optimization suggestions from the query advisor. Operation-only. | +| Option | Type | Notes | +| ------------------------- | ---------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `operation` | `OperationOptions` | Layered group; `content_response_on_write` is ignored for queries. | +| `session_token` | `Option` | Session token for session-consistent queries. Operation-only. | +| `enable_scan_if_no_index` | `Option` | If the query can't be served by indexes because the relevant paths are not indexed, setting this permits the query engine to perform a full container scan. Operation-only. | +| `populate_index_metrics` | `Option` | If set to `true`, the response will contain metrics regarding indexes used. Operation-only. | +| `populate_query_advice` | `Option` | If set to `true`, the response will include query optimization suggestions from the query advisor. Operation-only. | ### 5.4 `TransactionalBatchOptions` @@ -453,10 +453,10 @@ pub struct TransactionalBatchOptions { } ``` -| Option | Type | Notes | -| --- | --- | --- | -| `operation` | `OperationOptions` | Layered group; `content_response_on_write` controls whether batch responses include resource bodies. `read_consistency_strategy` and `excluded_regions` cascade. | -| `session_token` | `Option` | Session token for the batch. Operation-only. | +| Option | Type | Notes | +| --------------- | ---------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `operation` | `OperationOptions` | Layered group; `content_response_on_write` controls whether batch responses include resource bodies. `read_consistency_strategy` and `excluded_regions` cascade. | +| `session_token` | `Option` | Session token for the batch. Operation-only. | ### 5.5 `TransactionalBatchItemOptions` @@ -471,10 +471,10 @@ pub struct TransactionalBatchItemOptions { } ``` -| Option | Type | Notes | -| --- | --- | --- | -| `precondition` | `Option` | Conditional ETag check on this batch item. Typically `IfMatch` for optimistic concurrency. | -| `filter_predicate` | `Option` | SQL-like filter predicate for conditional patch operations within the batch. Only applicable to patch operations; ignored for other operation types. | +| Option | Type | Notes | +| ------------------ | ---------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- | +| `precondition` | `Option` | Conditional ETag check on this batch item. Typically `IfMatch` for optimistic concurrency. | +| `filter_predicate` | `Option` | SQL-like filter predicate for conditional patch operations within the batch. Only applicable to patch operations; ignored for other operation types. | **Usage example:** @@ -506,18 +506,18 @@ container.execute_transactional_batch(batch, Some(batch_opts)).await?; Metadata operations (database and container CRUD, throughput management) remain simple structs with operation-specific fields. They do **not** currently include `OperationOptions` for cross-layer resolution, but all are `#[non_exhaustive]` so option groups can be added later without breaking changes. -| Type | Fields | Notes | -| --- | --- | --- | -| `CreateContainerOptions` | `throughput: Option` | Provision throughput on creation. | -| `ReplaceContainerOptions` | *(none)* | | -| `DeleteContainerOptions` | *(none)* | | -| `ReadContainerOptions` | *(none)* | | -| `CreateDatabaseOptions` | `throughput: Option` | Provision throughput on creation. | -| `DeleteDatabaseOptions` | *(none)* | | -| `ReadDatabaseOptions` | *(none)* | | -| `QueryContainersOptions` | *(none)* | | -| `QueryDatabasesOptions` | *(none)* | | -| `ThroughputOptions` | *(none)* | Read or replace throughput settings. | +| Type | Fields | Notes | +| ------------------------- | ------------------------------------------ | ------------------------------------ | +| `CreateContainerOptions` | `throughput: Option` | Provision throughput on creation. | +| `ReplaceContainerOptions` | *(none)* | | +| `DeleteContainerOptions` | *(none)* | | +| `ReadContainerOptions` | *(none)* | | +| `CreateDatabaseOptions` | `throughput: Option` | Provision throughput on creation. | +| `DeleteDatabaseOptions` | *(none)* | | +| `ReadDatabaseOptions` | *(none)* | | +| `QueryContainersOptions` | *(none)* | | +| `QueryDatabasesOptions` | *(none)* | | +| `ThroughputOptions` | *(none)* | Read or replace throughput settings. | --- @@ -571,28 +571,28 @@ The Cosmos SDK manages its own transport, retry, and telemetry pipeline internal ## 7. Migration from Current Types -| Current Field | Current Location | New Location | Change | -| --- | --- | --- | --- | -| `user_agent_suffix` | `CosmosClientOptions` | `CosmosAccountOptions.user_agent_suffix` | Moved to option group | -| `application_region` | `CosmosClientOptions` | `RegionOptions.application_region` | Moved to option group | -| `application_preferred_regions` | `CosmosClientOptions` | — | **Removed** | -| `excluded_regions` | `CosmosClientOptions` | `OperationOptions.excluded_regions` | Moved; now `Option>` for layered resolution | -| `account_initialization_custom_endpoints` | `CosmosClientOptions` | `CosmosAccountOptions.account_initialization_custom_endpoints` | Moved to option group | -| `consistency_level` | `CosmosClientOptions`, `ItemOptions`, `QueryOptions` | `OperationOptions.read_consistency_strategy` | **Replaced** with `ReadConsistencyStrategy` | -| `request_timeout` | `CosmosClientOptions` | `ConnectionOptions.request_timeout` | Moved to option group | -| `enable_remote_region_preferred_for_session_retry` | `CosmosClientOptions` | — | **Removed**; remote-region-preferred is now always-on behavior | -| `enable_partition_level_circuit_breaker` | `CosmosClientOptions` | — | **Removed**; partition-level circuit breaker is always enabled | -| `disable_partition_level_failover` | `CosmosClientOptions` | — | **Removed**; disabling PPAF degrades availability | -| `enable_upgrade_consistency_to_local_quorum` | `CosmosClientOptions` | — | **Removed**; use `ReadConsistencyStrategy::LatestCommitted` instead | -| `throughput_bucket` | `CosmosClientOptions`, `ItemOptions`, `QueryOptions` | — | **Deferred** to throughput control follow-up spec | -| `session_retry_options` | `CosmosClientOptions` | `RetryOptions.session_retry` | Nested; fields become `Option` | -| `priority` | `CosmosClientOptions`, `ItemOptions`, `QueryOptions` | — | **Deferred** to throughput control follow-up spec | -| `custom_headers` | `CosmosClientOptions`, `ItemOptions`, `QueryOptions` | `CosmosAccountOptions.custom_headers` | Moved to option group; best-effort only (see §6.3) | -| `pre_triggers` | `ItemOptions` | — | **Removed** (§6.5) | -| `post_triggers` | `ItemOptions` | — | **Removed** (§6.5) | -| `session_token` | `ItemOptions`, `QueryOptions` | Operation-only on each type | Duplicated across read/write/query/batch | -| `indexing_directive` | `ItemOptions` | — | **Removed** (§6.4) | -| `if_match_etag` | `ItemOptions` | `ItemWriteOptions.precondition` | Replaced by `Precondition::IfMatch(Etag)` | -| `content_response_on_write_enabled` | `ItemOptions` | `OperationOptions.content_response_on_write` | Moved to layered group; renamed; now `Option` | -| `excluded_regions` | `ItemOptions` | `OperationOptions.excluded_regions` | Consolidated into layered group | -| `ItemOptions` (unified) | — | `ItemReadOptions` / `ItemWriteOptions` | **Split** into separate read and write types | +| Current Field | Current Location | New Location | Change | +| -------------------------------------------------- | ---------------------------------------------------- | -------------------------------------------------------------- | ------------------------------------------------------------------- | +| `user_agent_suffix` | `CosmosClientOptions` | `CosmosAccountOptions.user_agent_suffix` | Moved to option group | +| `application_region` | `CosmosClientOptions` | `RegionOptions.application_region` | Moved to option group | +| `application_preferred_regions` | `CosmosClientOptions` | — | **Removed** | +| `excluded_regions` | `CosmosClientOptions` | `OperationOptions.excluded_regions` | Moved; now `Option>` for layered resolution | +| `account_initialization_custom_endpoints` | `CosmosClientOptions` | `CosmosAccountOptions.account_initialization_custom_endpoints` | Moved to option group | +| `consistency_level` | `CosmosClientOptions`, `ItemOptions`, `QueryOptions` | `OperationOptions.read_consistency_strategy` | **Replaced** with `ReadConsistencyStrategy` | +| `request_timeout` | `CosmosClientOptions` | `ConnectionOptions.request_timeout` | Moved to option group | +| `enable_remote_region_preferred_for_session_retry` | `CosmosClientOptions` | — | **Removed**; remote-region-preferred is now always-on behavior | +| `enable_partition_level_circuit_breaker` | `CosmosClientOptions` | — | **Removed**; partition-level circuit breaker is always enabled | +| `disable_partition_level_failover` | `CosmosClientOptions` | — | **Removed**; disabling PPAF degrades availability | +| `enable_upgrade_consistency_to_local_quorum` | `CosmosClientOptions` | — | **Removed**; use `ReadConsistencyStrategy::LatestCommitted` instead | +| `throughput_bucket` | `CosmosClientOptions`, `ItemOptions`, `QueryOptions` | — | **Deferred** to throughput control follow-up spec | +| `session_retry_options` | `CosmosClientOptions` | `RetryOptions.session_retry` | Nested; fields become `Option` | +| `priority` | `CosmosClientOptions`, `ItemOptions`, `QueryOptions` | — | **Deferred** to throughput control follow-up spec | +| `custom_headers` | `CosmosClientOptions`, `ItemOptions`, `QueryOptions` | `CosmosAccountOptions.custom_headers` | Moved to option group; best-effort only (see §6.3) | +| `pre_triggers` | `ItemOptions` | — | **Removed** (§6.5) | +| `post_triggers` | `ItemOptions` | — | **Removed** (§6.5) | +| `session_token` | `ItemOptions`, `QueryOptions` | Operation-only on each type | Duplicated across read/write/query/batch | +| `indexing_directive` | `ItemOptions` | — | **Removed** (§6.4) | +| `if_match_etag` | `ItemOptions` | `ItemWriteOptions.precondition` | Replaced by `Precondition::IfMatch(Etag)` | +| `content_response_on_write_enabled` | `ItemOptions` | `OperationOptions.content_response_on_write` | Moved to layered group; renamed; now `Option` | +| `excluded_regions` | `ItemOptions` | `OperationOptions.excluded_regions` | Consolidated into layered group | +| `ItemOptions` (unified) | — | `ItemReadOptions` / `ItemWriteOptions` | **Split** into separate read and write types | diff --git a/sdk/cosmos/azure_data_cosmos/examples/cosmos_patch.rs b/sdk/cosmos/azure_data_cosmos/examples/cosmos_patch.rs index cd2d8002bf5..d0ca43cb674 100644 --- a/sdk/cosmos/azure_data_cosmos/examples/cosmos_patch.rs +++ b/sdk/cosmos/azure_data_cosmos/examples/cosmos_patch.rs @@ -3,6 +3,10 @@ //! Partial updates with PATCH. //! +//! **Preview.** Requires the `preview_patch` feature. PATCH is not +//! production-ready: an interrupted patch may re-apply non-idempotent +//! operations. See `ContainerClient::patch_item` rustdoc. +//! //! Demonstrates `ContainerClient::patch_item` and the `PatchInstructions` //! builder, exercising every variant of `PatchOperation` (`set`, `add`, //! `replace`, `remove`, `increment`, `move_value`). @@ -15,8 +19,9 @@ //! 1. The SDK reads the current item (capturing its ETag). //! 2. The SDK applies the patch operations locally. //! 3. The SDK issues a conditional Replace gated on the ETag from step 1. -//! 4. On a 412 Precondition Failed (mid-air collision), the SDK retries -//! from step 1 up to `PatchItemOptions::with_max_attempts` times. +//! 4. On a 412 Precondition Failed (mid-air collision), the SDK starts another +//! attempt from step 1 until the total `PatchItemOptions::with_max_attempts` +//! budget, including the initial attempt, is exhausted. //! //! See `ContainerClient::patch_item` rustdoc for the idempotency caveats. //! @@ -27,7 +32,7 @@ //! ## Running //! //! ```text -//! cargo run --example cosmos_patch -- \ +//! cargo run --features preview_patch --example cosmos_patch -- \ //! https://.documents.azure.com:443/ --region "East US" --use-entra //! ``` @@ -110,9 +115,10 @@ async fn main() -> Result<(), Box> { .with_operation(PatchOperation::move_value("/tags/0", "/headline_tag")); // ----- Issue the patch. ------------------------------------------------- - // `with_max_attempts` bounds the SDK's internal RMW retry loop. The - // default (3) is fine for most workloads; very contended items may - // benefit from a higher cap, but consider re-shaping the workload first. + // `with_max_attempts` bounds the total number of RMW attempts, including + // the initial attempt. The default is 5; this example sets it explicitly. + // Very contended items may benefit from a higher cap, but consider + // re-shaping the workload first. let response = items .patch_item( "contoso", @@ -130,8 +136,9 @@ async fn main() -> Result<(), Box> { response.headers().request_charge(), ); - // The response is the standard `ItemResponse` shape, so we can read the - // patched item back via `into_model::()` when content-on-write is on. + // The response always contains the post-image: the handler uses the + // service body when present and otherwise synthesizes it from the locally + // merged document, so `into_model::()` does not require content-on-write. let patched: serde_json::Value = response.into_model()?; println!("after {patched:#}"); diff --git a/sdk/cosmos/azure_data_cosmos/src/clients/container_client.rs b/sdk/cosmos/azure_data_cosmos/src/clients/container_client.rs index 3f0b7ef847b..2a257ae4ec4 100644 --- a/sdk/cosmos/azure_data_cosmos/src/clients/container_client.rs +++ b/sdk/cosmos/azure_data_cosmos/src/clients/container_client.rs @@ -5,14 +5,16 @@ use crate::{ clients::ClientContext, diagnostics::CosmosOperationContext, feed::{ChangeFeedPageIterator, FeedRange, FeedScope, QueryItemIterator}, - models::{BatchResponse, ChangeFeedItem, ItemResponse, PatchInstructions, TransactionalBatch}, + models::{BatchResponse, ChangeFeedItem, ItemResponse, TransactionalBatch}, options::{ BatchOptions, BinaryEncodingOptions, ChangeFeedMode, ChangeFeedOptions, - ChangeFeedStartFrom, ItemReadOptions, ItemWriteOptions, OperationOptions, PatchItemOptions, - Precondition, QueryOptions, ReadContainerOptions, ReadFeedRangesOptions, SessionToken, + ChangeFeedStartFrom, ItemReadOptions, ItemWriteOptions, OperationOptions, Precondition, + QueryOptions, ReadContainerOptions, ReadFeedRangesOptions, SessionToken, }, PartitionKey, Query, ResourceIdentity, }; +#[cfg(feature = "preview_patch")] +use crate::{models::PatchInstructions, options::PatchItemOptions}; use azure_data_cosmos_driver::models::{ ContainerReference, CosmosOperation, ItemReference, PartitionKeyKind, @@ -524,6 +526,9 @@ impl ContainerClient { /// Applies a JSON-PATCH-style update to an item by reading it, applying /// the [`PatchInstructions`] locally, and issuing an ETag-guarded Replace. /// + /// **Preview.** Requires the `preview_patch` feature. This API is not + /// production-ready — see [Failure Semantics](#failure-semantics) below. + /// /// The handler refuses to PATCH paths that overlap the container's /// partition-key paths: rewriting the partition key would move the /// document to a different physical partition, so such requests are @@ -590,6 +595,7 @@ impl ContainerClient { /// appends should either build idempotent ops (`PatchOperation::set` on a /// caller-computed value) or detect duplicate-application via a /// monotonic application-level sequence number. + #[cfg(feature = "preview_patch")] pub async fn patch_item( &self, partition_key: impl Into, @@ -1500,9 +1506,13 @@ fn _assert_futures_are_send() { let client: &ContainerClient = todo!(); let partition_key: PartitionKey = todo!(); let item_id: &str = todo!(); - let patch: PatchInstructions = todo!(); - let options: Option = todo!(); - assert_send(client.patch_item(partition_key, item_id, patch, options)); + assert_send(client.read_item(partition_key.clone(), item_id, None)); + #[cfg(feature = "preview_patch")] + { + let patch: PatchInstructions = todo!(); + let options: Option = todo!(); + assert_send(client.patch_item(partition_key, item_id, patch, options)); + } } #[cfg(test)] diff --git a/sdk/cosmos/azure_data_cosmos/src/options/item.rs b/sdk/cosmos/azure_data_cosmos/src/options/item.rs index 8ee7c79a11b..93368f64f80 100644 --- a/sdk/cosmos/azure_data_cosmos/src/options/item.rs +++ b/sdk/cosmos/azure_data_cosmos/src/options/item.rs @@ -94,6 +94,10 @@ impl ItemWriteOptions { /// Options for [`ContainerClient::patch_item()`](crate::clients::ContainerClient::patch_item()). /// +/// **Preview.** Requires the `preview_patch` feature. PATCH is not +/// production-ready: an interrupted patch may re-apply non-idempotent +/// operations. See [Failure Semantics](crate::clients::ContainerClient::patch_item()). +/// /// PATCH is implemented driver-side as a Read-Modify-Write (RMW) loop: /// the driver reads the current item, applies your [`PatchInstructions`](crate::models::PatchInstructions) /// locally, and issues an ETag-guarded Replace. If the Replace returns @@ -143,6 +147,7 @@ impl ItemWriteOptions { /// 412 retries you want to tolerate. Setting the budget too low can /// cancel the RMW between the Read and the Replace, producing a /// timeout error even when the service is healthy. +#[cfg(feature = "preview_patch")] #[derive(Clone, Default)] #[non_exhaustive] pub struct PatchItemOptions { @@ -158,6 +163,7 @@ pub struct PatchItemOptions { pub max_attempts: Option, } +#[cfg(feature = "preview_patch")] impl PatchItemOptions { /// Sets the session token for this request. pub fn with_session_token(mut self, session_token: impl Into) -> Self { diff --git a/sdk/cosmos/azure_data_cosmos/src/options/mod.rs b/sdk/cosmos/azure_data_cosmos/src/options/mod.rs index e1f264cfb84..869e42a9808 100644 --- a/sdk/cosmos/azure_data_cosmos/src/options/mod.rs +++ b/sdk/cosmos/azure_data_cosmos/src/options/mod.rs @@ -40,7 +40,9 @@ pub use database::{ }; pub use feed::{FeedOptions, QueryOptions}; pub use feed_ranges::ReadFeedRangesOptions; -pub use item::{ItemReadOptions, ItemWriteOptions, PatchItemOptions}; +#[cfg(feature = "preview_patch")] +pub use item::PatchItemOptions; +pub use item::{ItemReadOptions, ItemWriteOptions}; pub use routing_strategy::RoutingStrategy; #[cfg(feature = "control_plane")] pub use throughput::ThroughputOptions; diff --git a/sdk/cosmos/azure_data_cosmos/tests/emulator_tests/cosmos_hpk.rs b/sdk/cosmos/azure_data_cosmos/tests/emulator_tests/cosmos_hpk.rs index 4766a4c1d9a..c63fccf4ff7 100644 --- a/sdk/cosmos/azure_data_cosmos/tests/emulator_tests/cosmos_hpk.rs +++ b/sdk/cosmos/azure_data_cosmos/tests/emulator_tests/cosmos_hpk.rs @@ -358,6 +358,7 @@ pub async fn hpk_item_delete_full_key() -> Result<(), Box> { } /// A5: patch a field on an item addressed by a full 2-level key. +#[cfg(feature = "preview_patch")] #[tokio::test] #[cfg_attr( not(any(test_category = "emulator", test_category = "emulator_vnext")), diff --git a/sdk/cosmos/azure_data_cosmos/tests/emulator_tests/mod.rs b/sdk/cosmos/azure_data_cosmos/tests/emulator_tests/mod.rs index bffc3271949..77d7c89042f 100644 --- a/sdk/cosmos/azure_data_cosmos/tests/emulator_tests/mod.rs +++ b/sdk/cosmos/azure_data_cosmos/tests/emulator_tests/mod.rs @@ -12,6 +12,7 @@ mod cosmos_hpk; mod cosmos_items; mod cosmos_offers; mod cosmos_partition_key_types; +#[cfg(feature = "preview_patch")] mod cosmos_patch; mod cosmos_proxy; mod cosmos_query; diff --git a/sdk/cosmos/azure_data_cosmos_driver/docs/ErrorCodesAndRetries.md b/sdk/cosmos/azure_data_cosmos_driver/docs/ErrorCodesAndRetries.md index 0fc6aad35c1..790702ccf45 100644 --- a/sdk/cosmos/azure_data_cosmos_driver/docs/ErrorCodesAndRetries.md +++ b/sdk/cosmos/azure_data_cosmos_driver/docs/ErrorCodesAndRetries.md @@ -16,17 +16,17 @@ Stored procedure execution is the only data-plane operation that is gated. The dividing line is whether the response proves the procedure did not run, not whether the operation is idempotent. -| Outcome | Stored procedure | Why | -|---------|------------------|-----| -| Transport error, request definitely **not sent** | Retry | Never reached the backend | -| Transport error, **sent** or unknown | **Abort** | May have run to completion | -| 408 Request Timeout | **Abort** | Outcome unknown | -| 500 / 502 / 504 | **Abort** | Outcome unknown | -| 503 Service Unavailable | Retry | Returned only for unprocessed requests | -| 410 Gone | Retry | Routing rejection, before execution | -| 429 / 429-3092 | Retry | Throttled, before execution | -| 449 Retry With | Retry (in-region) | Request never completed | -| 403/3, 403/1008 | Retry | Rejected on topology, before execution | +| Outcome | Stored procedure | Why | +| ------------------------------------------------ | ----------------- | -------------------------------------- | +| Transport error, request definitely **not sent** | Retry | Never reached the backend | +| Transport error, **sent** or unknown | **Abort** | May have run to completion | +| 408 Request Timeout | **Abort** | Outcome unknown | +| 500 / 502 / 504 | **Abort** | Outcome unknown | +| 503 Service Unavailable | Retry | Returned only for unprocessed requests | +| 410 Gone | Retry | Routing rejection, before execution | +| 429 / 429-3092 | Retry | Throttled, before execution | +| 449 Retry With | Retry (in-region) | Request never completed | +| 403/3, 403/1008 | Retry | Rejected on topology, before execution | Enforced by `is_unsafe_retry_after_possible_execution` in `src/driver/pipeline/retry_evaluation.rs`, which delegates the operation-type @@ -39,44 +39,44 @@ Write retries are not strictly idempotent — the initial attempt and a retry ma For replace and upsert operations, the driver **always retries** regardless of whether an ETag precondition is provided. If the application developer has concerns about idempotency or wants optimistic locking, ETag preconditions (`If-Match` headers) are the appropriate mitigation. Without ETags, there is no concurrency control — concurrent writers or retried writes can silently overwrite each other. -| Operation | Retried? | Initial attempt | On retry (duplicate) | App must handle | -|-----------|----------|-----------------|----------------------|-----------------| -| Create | Yes | 201 Created | 409 Conflict | 409 | -| Delete | Yes | 204 No Content | 404 Not Found | 404 | -| Replace / Upsert (with ETag) | Yes | 200 OK | 412 Precondition Failed (if concurrent update) | 412 | -| Replace / Upsert (without ETag) | Yes | 200 OK | 200 OK (silent overwrite — no concurrency control) | — | -| Patch | Yes | 200 OK | 200 OK (operation-level idempotency) | — | -| Stored Procedure | **Only when provably not executed** | Varies | N/A — see [Stored procedure retries](#stored-procedure-retries) | N/A | +| Operation | Retried? | Initial attempt | On retry (duplicate) | App must handle | +| ------------------------------- | ----------------------------------- | --------------- | --------------------------------------------------------------- | --------------- | +| Create | Yes | 201 Created | 409 Conflict | 409 | +| Delete | Yes | 204 No Content | 404 Not Found | 404 | +| Replace / Upsert (with ETag) | Yes | 200 OK | 412 Precondition Failed (if concurrent update) | 412 | +| Replace / Upsert (without ETag) | Yes | 200 OK | 200 OK (silent overwrite — no concurrency control) | — | +| Patch | Yes | 200 OK | 200 OK (operation-level idempotency) | — | +| Stored Procedure | **Only when provably not executed** | Varies | N/A — see [Stored procedure retries](#stored-procedure-retries) | N/A | ## Status Code Handling ### Non-Retryable (Abort Immediately) -| Status | Substatus | Meaning | Action | -|--------|-----------|---------|--------| -| 400 | — | Bad Request | Abort | -| 401 | — | Unauthorized | Abort | -| 404 | 0 | Not Found | Abort | -| 409 | — | Conflict | Abort | -| 412 | — | Precondition Failed | Abort | +| Status | Substatus | Meaning | Action | +| ------ | --------- | ------------------- | ------ | +| 400 | — | Bad Request | Abort | +| 401 | — | Unauthorized | Abort | +| 404 | 0 | Not Found | Abort | +| 409 | — | Conflict | Abort | +| 412 | — | Precondition Failed | Abort | These are deterministic client errors. No retry will change the outcome. ### 449 — Retry With -| Operation | Action | Budget | -|-----------|--------|--------| -| Any | SDK-owned retry | TBD | +| Operation | Action | Budget | +| --------- | --------------- | ------ | +| Any | SDK-owned retry | TBD | 449 indicates the request must be retried with a modified configuration (e.g., after a collection recreate or partition split). Gateway V1 can handle 449 retries internally, but the Rust SDK always disables Gateway-side 449 retries and owns them in the SDK. This is required for Gateway V2, where all 449 retries must be handled by the SDK. ### 403 — Forbidden -| Substatus | Meaning | Action | Budget (multi-write) | Budget (single-write) | -|-----------|---------|--------|----------------------|-----------------------| -| 3 | `WriteForbidden` — region is not currently a valid write region for this partition (writes only) | Refresh account topology + cross-region failover retry | **5s cumulative delay**, immediate first retry then exponential backoff with jitter (dedicated backend-failover state) | **5s cumulative delay**, immediate first retry then exponential backoff with jitter (dedicated backend-failover state) | -| 1008 | `DatabaseAccountNotFound` — region no longer owns this account (all op types, including reads, writes, queries, feed-range queries, metadata) | Refresh account topology + cross-region failover retry | **5s cumulative delay**, immediate first retry then exponential backoff with jitter (dedicated backend-failover state) | **5s cumulative delay**, immediate first retry then exponential backoff with jitter (dedicated backend-failover state) | -| Other | Permission denied | Abort | — | — | +| Substatus | Meaning | Action | Budget (multi-write) | Budget (single-write) | +| --------- | --------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------- | +| 3 | `WriteForbidden` — region is not currently a valid write region for this partition (writes only) | Refresh account topology + cross-region failover retry | **5s cumulative delay**, immediate first retry then exponential backoff with jitter (dedicated backend-failover state) | **5s cumulative delay**, immediate first retry then exponential backoff with jitter (dedicated backend-failover state) | +| 1008 | `DatabaseAccountNotFound` — region no longer owns this account (all op types, including reads, writes, queries, feed-range queries, metadata) | Refresh account topology + cross-region failover retry | **5s cumulative delay**, immediate first retry then exponential backoff with jitter (dedicated backend-failover state) | **5s cumulative delay**, immediate first retry then exponential backoff with jitter (dedicated backend-failover state) | +| Other | Permission denied | Abort | — | — | Both 403/3 and 403/1008 signal that the cached topology in the SDK has diverged from the backend's current routing — typically during a backend-initiated failover or a customer-initiated topology change. On each retry the driver requests `LocationEffect::RefreshAccountProperties` so the next attempt routes against the freshly learned region set. The metadata refresh itself is throttled by a lease on `refresh_interval` (default 5 s): an event-driven caller stamps the clock *before* fetching, which suppresses other event-driven callers for that interval. This is a throttle, not mutual exclusion — a fetch that outlives the interval (metadata requests are allowed up to 65 s) can be joined by a second refresh, and the background timer refresh bypasses the lease entirely. A failed or cancelled refresh also releases its claim so the next retry can fetch immediately. Metadata traffic is therefore not strictly bounded to one fetch per interval. The refresh is independent of the caller's `excluded_regions` — the GetDatabaseAccount probe iterates the global endpoint and the cached `readable_locations` regardless of the operation-level exclusion list, because excluding a region from data-plane routing should not blind the SDK to topology changes happening in that region. @@ -93,20 +93,20 @@ Outside these explicit exceptions, excluded regions remain a hard per-operation ### 404/1002 — Read Session Not Available -| Account Type | Action | Budget | -|--------------|--------|--------| -| Single-write | Session retry to write region (hub region) | 2 attempts | -| Multi-write | Session retry, advance through preferred endpoints | `preferred_endpoints.len()` attempts | +| Account Type | Action | Budget | +| ------------ | -------------------------------------------------- | ------------------------------------ | +| Single-write | Session retry to write region (hub region) | 2 attempts | +| Multi-write | Session retry, advance through preferred endpoints | `preferred_endpoints.len()` attempts | The session token is preserved on all retry attempts — it is never cleared to allow stale reads, as that would violate the customer's chosen consistency guarantees. When all session retries are exhausted, the 404/1002 error is surfaced to the caller. ### 408 — Request Timeout -| Operation | Action | Budget | -|-----------|--------|--------| -| Reads | Cross-region failover retry | 3 failover attempts | +| Operation | Action | Budget | +| --------------------------------- | ------------------------------- | ------------------- | +| Reads | Cross-region failover retry | 3 failover attempts | | Writes (except stored procedures) | **Cross-region failover retry** | 3 failover attempts | -| Stored Procedure writes | **Abort** | — | +| Stored Procedure writes | **Abort** | — | 408 indicates a server-side or client-side timeout. The Rust driver retries writes on 408 because: @@ -118,19 +118,19 @@ For single-write accounts, retry cycles through the available endpoint(s). For m ### 410 — Gone -| Operation | Action | Budget | -|-----------|--------|--------| -| Reads | Cross-region failover retry | 3 failover attempts | +| Operation | Action | Budget | +| ------------ | ------------------------------- | ------------------- | +| Reads | Cross-region failover retry | 3 failover attempts | | Writes (all) | **Cross-region failover retry** | 3 failover attempts | 410 indicates the partition has moved or is undergoing a split/merge. All operations retry, regardless of idempotency. ### 429 — Too Many Requests (Throttling) -| Substatus | Action | Budget | -|-----------|--------|--------| -| — (standard) | Local retry with backoff | 9 attempts / 30s total | -| 3092 (global throttle) | Cross-region failover retry | 3 failover attempts | +| Substatus | Action | Budget | +| ---------------------- | --------------------------- | ---------------------- | +| — (standard) | Local retry with backoff | 9 attempts / 30s total | +| 3092 (global throttle) | Cross-region failover retry | 3 failover attempts | Standard 429 is handled entirely within the transport pipeline — the operation pipeline never sees it. The transport layer respects `x-ms-retry-after-ms` headers and falls back to exponential backoff (5ms base, 5s cap per attempt). @@ -140,11 +140,11 @@ Standard 429 is handled entirely within the transport pipeline — the operation ### 5xx — Server Errors (500, 502, 503, 504) -| Operation | Action | Budget | -|-----------|--------|--------| -| Reads | Cross-region failover retry | 3 failover attempts | -| Writes (all) | **Cross-region failover retry** | 3 failover attempts | -| Stored Procedure execution | **Abort** (except 503) | — | +| Operation | Action | Budget | +| -------------------------- | ------------------------------- | ------------------- | +| Reads | Cross-region failover retry | 3 failover attempts | +| Writes (all) | **Cross-region failover retry** | 3 failover attempts | +| Stored Procedure execution | **Abort** (except 503) | — | All 5xx errors are retried uniformly. 503 is the canonical "safe to retry" signal from Cosmos DB — when the service intentionally returns 503, it guarantees the write was not processed. All other 5xx codes (500, 502, 504) are retried identically because CRUD write operations are idempotent when customers use ETag preconditions (see [Idempotency Requirements](#idempotency-requirements) above). 502/504 may be raised by intermediate proxies, but ETag preconditions (412 on stale ETag) prevent silent overwrites on retry. Stored procedure execution aborts on every 5xx except 503, which alone proves the procedure did not run — see `is_unsafe_retry_after_possible_execution` in `src/driver/pipeline/retry_evaluation.rs`. @@ -156,12 +156,12 @@ All 5xx errors are retried uniformly. 503 is the canonical "safe to retry" signa ### Transport Errors (Connection Failures) -| Sent Status | Operation | Action | Budget | -|-------------|-----------|--------|--------| -| **Not sent** (request never left client) | Any | Cross-region failover retry | 3 failover attempts | -| **Sent** or unknown | Reads | Cross-region failover retry | 3 failover attempts | -| **Sent** or unknown | Writes (all) | **Cross-region failover retry** | 3 failover attempts | -| **Sent** or unknown | Stored Procedure execution | **Abort** | — | +| Sent Status | Operation | Action | Budget | +| ---------------------------------------- | -------------------------- | ------------------------------- | ------------------- | +| **Not sent** (request never left client) | Any | Cross-region failover retry | 3 failover attempts | +| **Sent** or unknown | Reads | Cross-region failover retry | 3 failover attempts | +| **Sent** or unknown | Writes (all) | **Cross-region failover retry** | 3 failover attempts | +| **Sent** or unknown | Stored Procedure execution | **Abort** | — | When the request was definitely not sent (connection refused, DNS failure, TLS error), the endpoint itself is unreachable. The driver marks the endpoint as unavailable (affecting all partitions on it) and records a partition-level failure for PPCB tracking, then retries on the next preferred region. @@ -173,9 +173,9 @@ For connectivity errors (connection refused, I/O errors), the transport layer pe ### Deadline Exceeded (Client-Side Timeout) -| Operation | Action | Budget | -|-----------|--------|--------| -| Any | **Abort** — synthesize 408 / `CLIENT_OPERATION_TIMEOUT` | — | +| Operation | Action | Budget | +| --------- | ------------------------------------------------------- | ------ | +| Any | **Abort** — synthesize 408 / `CLIENT_OPERATION_TIMEOUT` | — | When the client's end-to-end deadline is exceeded, no retry is attempted. The operation has already consumed its time budget. @@ -221,10 +221,10 @@ PPAF is an **opt-in** feature for **single-master write accounts only**. When en PPCB is an **opt-out** feature (enabled by default) that provides partition-level health tracking and routing: -| Account Type | Reads | Writes | -|--------------|-------|--------| +| Account Type | Reads | Writes | +| ------------ | -------------- | ---------------------------------------- | | Single-write | ✅ PPCB-managed | ❌ Not PPCB-managed (PPAF handles writes) | -| Multi-write | ✅ PPCB-managed | ✅ PPCB-managed | +| Multi-write | ✅ PPCB-managed | ✅ PPCB-managed | ### Behavior @@ -244,13 +244,13 @@ Without PPCB, the driver marks entire endpoints as unavailable when errors occur ## Retry Budget Summary -| Layer | Budget | Scope | -|-------|--------|-------| -| Transport (429) | 9 attempts or 30s | Per-request, local only | -| Operation failover (generic — 5xx, 408, 410, transport) | 3 attempts | Per-operation, cross-region | +| Layer | Budget | Scope | +| ---------------------------------------------------------- | -------------------------------------------------------------------------------- | --------------------------- | +| Transport (429) | 9 attempts or 30s | Per-request, local only | +| Operation failover (generic — 5xx, 408, 410, transport) | 3 attempts | Per-operation, cross-region | | Backend-failover (403/1008) — single-write and multi-write | **5s cumulative delay**, immediate first retry then exponential backoff + jitter | Per-operation, cross-region | -| Backend-failover (403/3) — single-write and multi-write | **5s cumulative delay**, immediate first retry then exponential backoff + jitter | Per-operation, cross-region | -| Session retry (404/1002) | 2 (single-write) or `preferred_endpoints.len()` (multi-write) | Per-operation | +| Backend-failover (403/3) — single-write and multi-write | **5s cumulative delay**, immediate first retry then exponential backoff + jitter | Per-operation, cross-region | +| Session retry (404/1002) | 2 (single-write) or `preferred_endpoints.len()` (multi-write) | Per-operation | The 403/3 hub-region discovery branch is the one exception: a 403/3 on a read with the `hub_region_processing_only` latch rotates the cached hub endpoint and @@ -258,16 +258,16 @@ stays on the generic 3-attempt failover budget with no pacing. ## Comparison with Other SDKs -| Behavior | Python | Java | .NET | **Rust (Target)** | -|----------|--------|------|------|-------------------| -| 503 write retry | Always (no gate) | Multi-write only | Multi-write only | **Always** | -| 500 write retry | Only with `retry_write` | No | No | **Always** | -| 408 write retry | Only with `retry_write` | No | No | **Always** | -| 502/504 write retry | Only with `retry_write` | No | No | **Always** | -| Non-idempotent write retry | Gated by `retry_write` | Gated by multi-write | Gated by multi-write | **Always (no gate)** | -| Transport sent + write | Abort | Abort | Abort | **Retry** | -| Stored procedure retry | No | No | No | **Only when provably not executed** | -| PPAF | Yes (single-master) | Yes | Yes | **Yes** | -| PPCB | Yes | Yes | Yes | **Yes** | +| Behavior | Python | Java | .NET | **Rust (Target)** | +| -------------------------- | ----------------------- | -------------------- | -------------------- | ----------------------------------- | +| 503 write retry | Always (no gate) | Multi-write only | Multi-write only | **Always** | +| 500 write retry | Only with `retry_write` | No | No | **Always** | +| 408 write retry | Only with `retry_write` | No | No | **Always** | +| 502/504 write retry | Only with `retry_write` | No | No | **Always** | +| Non-idempotent write retry | Gated by `retry_write` | Gated by multi-write | Gated by multi-write | **Always (no gate)** | +| Transport sent + write | Abort | Abort | Abort | **Retry** | +| Stored procedure retry | No | No | No | **Only when provably not executed** | +| PPAF | Yes (single-master) | Yes | Yes | **Yes** | +| PPCB | Yes | Yes | Yes | **Yes** | The Rust driver is intentionally more aggressive about retrying writes. This is a deliberate design choice for maximum availability, leveraging Cosmos DB's conflict detection and the use of Etags as the safety net for duplicates and idempotency concerns. Stored procedure execution is the single carve-out, because the driver cannot reason about a procedure body it never sees. diff --git a/sdk/cosmos/azure_data_cosmos_driver/docs/PATCH_HANDLER_SPEC.md b/sdk/cosmos/azure_data_cosmos_driver/docs/PATCH_HANDLER_SPEC.md index 0742bead36f..aa9cb86e24b 100644 --- a/sdk/cosmos/azure_data_cosmos_driver/docs/PATCH_HANDLER_SPEC.md +++ b/sdk/cosmos/azure_data_cosmos_driver/docs/PATCH_HANDLER_SPEC.md @@ -3,6 +3,18 @@ This document describes the contract for `OperationType::Patch` in `azure_data_cosmos_driver`. +## Known limitation and SDK exposure + +The core driver always includes PATCH. Consuming SDKs decide whether and how +to expose it as preview using conventions appropriate to each language. The +Rust SDK, `azure_data_cosmos`, gates its public PATCH API behind the +**`preview_patch`** Cargo feature, which is off by default. + +The handler does not deliver exactly-once semantics under transport failures: +an interrupted patch may re-apply non-idempotent operations (`Increment`, +`Add` on an array, `Move`). See [Invariants](#invariants) for the exact +interleaving. The Rust SDK API will stay gated until that hole is closed. + ## Overview `Patch` is a *virtual* operation type: the Cosmos DB REST endpoint does not @@ -199,18 +211,21 @@ as a JSON number without precision loss. ## Errors -- All sub-operation errors are surfaced verbatim — including the - `DiagnosticsContext` and request-tracking info from the internal Read or - Replace. +- Sub-operation errors preserve the failing error's status, sub-status, raw + response, and source. The attached diagnostics are re-stamped with the + virtual PATCH operation's `patch_item` name before the error is surfaced. +- A non-412 failure after earlier sub-operations carries an aggregated + `DiagnosticsContext` containing those prior contexts plus the failing + sub-operation's context, in dispatch order. When the first sub-operation + fails, its context is retained with only the operation name rewritten. - The handler never retries beyond `max_attempts` and never converts a 412 into success; the final outcome is whichever of "internal sub-op error", "successful PATCH", or "exhausted RMW attempts (412)" terminated the loop. -- The aggregated `DiagnosticsContext` described in "Response Synthesis" - applies to the *successful* path. On error paths the surfaced - `DiagnosticsContext` is whatever the failing sub-op already carried — - the handler does not synthesize an aggregated context for partial - failures. +- When 412 retries exhaust `max_attempts`, the final error carries an + aggregated `DiagnosticsContext` containing every accumulated Read and + failed Replace context. Thus both successful and failed PATCH operations + follow the "one PATCH operation = one `DiagnosticsContext`" contract. ## Why Driver-Side? @@ -250,10 +265,12 @@ as a JSON number without precision loss. already replicated the original commit returns 412, which the RMW loop treats as a normal race-lost and recovers by re-Reading and re-applying. Non-idempotent ops (`Increment`, `Add` on an array, `Move`) may therefore - be applied **more than once** under this scenario. Lifting this caveat - requires marking the internal Replace as non-idempotent for retry - purposes (e.g. a per-op idempotency override on `CosmosOperation`); that - is tracked as a follow-up because it interacts with PPAF write-retry - semantics. Callers needing exactly-once should either use idempotent ops - (`Set` on a caller-computed value) or detect duplicate-application via a - monotonic application-level sequence number. + be applied **more than once** under this scenario. This is why the Rust SDK + treats PATCH as preview and gates it behind `preview_patch`; other consuming + SDKs choose their own exposure policy. Closing the hole requires the RMW loop + to be able to *recognize its own committed write* rather than mistaking it + for a concurrent writer — i.e. stamping each attempt with a marker the loop + can look for on the verification read. Until then, callers needing + exactly-once should either use idempotent ops (`Set` on a caller-computed + value) or detect duplicate-application via a monotonic application-level + sequence number. diff --git a/sdk/cosmos/azure_data_cosmos_driver/src/driver/pipeline/mod.rs b/sdk/cosmos/azure_data_cosmos_driver/src/driver/pipeline/mod.rs index 69f142f6c6f..5b41074c917 100644 --- a/sdk/cosmos/azure_data_cosmos_driver/src/driver/pipeline/mod.rs +++ b/sdk/cosmos/azure_data_cosmos_driver/src/driver/pipeline/mod.rs @@ -8,11 +8,13 @@ //! and pipeline stages are pure functions over those components. pub(crate) mod components; +// Exists only to synthesize the PATCH handler's response from a local body. pub(crate) mod from_local_body; pub(crate) mod hedge_budget; pub(crate) mod hedging_diagnostics; pub(crate) mod hedging_eligibility; pub(crate) mod operation_pipeline; +// Shared by the PATCH handler and the in-memory emulator's DTX patch handling. pub(crate) mod patch_eval; pub(crate) mod patch_handler; pub(crate) mod retry_evaluation; diff --git a/sdk/cosmos/azure_data_cosmos_driver/src/models/cosmos_operation.rs b/sdk/cosmos/azure_data_cosmos_driver/src/models/cosmos_operation.rs index 2b5cdf16444..b01c259eafc 100644 --- a/sdk/cosmos/azure_data_cosmos_driver/src/models/cosmos_operation.rs +++ b/sdk/cosmos/azure_data_cosmos_driver/src/models/cosmos_operation.rs @@ -886,6 +886,9 @@ impl CosmosOperation { /// the operation body (via [`with_body`](Self::with_body)) — the patch /// handler deserializes it before issuing the underlying transport /// operations. + /// + /// An interrupted patch may re-apply non-idempotent operations — see + /// `docs/PATCH_HANDLER_SPEC.md`. pub fn patch_item(item: ItemReference) -> Self { Self::for_item(OperationType::Patch, item) } diff --git a/sdk/cosmos/azure_data_cosmos_driver/tests/in_memory_emulator_tests/patch_retry_faults.rs b/sdk/cosmos/azure_data_cosmos_driver/tests/in_memory_emulator_tests/patch_retry_faults.rs new file mode 100644 index 00000000000..d4797e3295e --- /dev/null +++ b/sdk/cosmos/azure_data_cosmos_driver/tests/in_memory_emulator_tests/patch_retry_faults.rs @@ -0,0 +1,414 @@ +// Copyright (c) Microsoft Corporation. All rights reserved. +// Licensed under the MIT License. + +//! Retry behavior of the two patch execution paths under an ambiguous failure. +//! +//! The safety argument for running a patch server-side rests on one claim: when +//! a request fails in a way that leaves the outcome unknown, the driver sends it +//! again only if doing so cannot change the result. These tests exercise that +//! claim end to end rather than asserting it on a predicate. +//! +//! The fault is [`FaultInjectionErrorType::ResponseTimeout`], which the +//! framework injects with [`RequestSentStatus::Unknown`] — the request may +//! already have reached the backend. That is the exact condition +//! `CosmosOperation::allows_ambiguous_outcome_retry` governs. +//! +//! `rule.hit_count()` counts how many times a request reached the fault, so it +//! is the attempt count: `1` means the driver gave up immediately, `> 1` means +//! it resent. +//! +//! **Scope note.** Injection short-circuits above the emulator store, so the +//! mutation never lands and these tests cannot observe a literal double-apply +//! on the failing attempt. What they do cover is the decision the driver +//! actually owns — whether to resend — plus, on the recovery tests, that a +//! successful retry applies the operation exactly once. + +use std::sync::Arc; + +use azure_core::http::Url; + +use azure_data_cosmos_driver::fault_injection::{ + FaultInjectionConditionBuilder, FaultInjectionErrorType, FaultInjectionResultBuilder, + FaultInjectionRule, FaultInjectionRuleBuilder, FaultOperationType, +}; +use azure_data_cosmos_driver::in_memory_emulator::{ + ConsistencyLevel, InMemoryEmulatorHttpClient, VirtualAccountConfig, VirtualRegion, +}; +use azure_data_cosmos_driver::models::{ + AccountReference, ContainerReference, CosmosOperation, ItemReference, PartitionKey, + PatchInstructions, PatchOperation, +}; +use azure_data_cosmos_driver::options::{ + DriverOptions, OperationOptions, OperationOptionsBuilder, PatchStrategy, +}; +use azure_data_cosmos_driver::CosmosDriver; + +const GATEWAY_URL: &str = "https://eastus.emulator.local"; +const PK: &str = "pk1"; +const ITEM_ID: &str = "retry-item"; + +/// Builds a rule that fails every matching request with an ambiguous +/// post-send timeout, optionally only for the first `hit_limit` attempts. +fn ambiguous_failure_rule( + id: &str, + operation: FaultOperationType, + hit_limit: Option, +) -> Arc { + let condition = FaultInjectionConditionBuilder::new() + .with_operation_type(operation) + .build(); + let result = FaultInjectionResultBuilder::new() + .with_error(FaultInjectionErrorType::ResponseTimeout) + .with_probability(1.0) + .build(); + let mut builder = FaultInjectionRuleBuilder::new(id, result).with_condition(condition); + if let Some(limit) = hit_limit { + builder = builder.with_hit_limit(limit); + } + Arc::new(builder.build()) +} + +async fn build_driver(rules: Vec>) -> Arc { + let config = VirtualAccountConfig::new(vec![VirtualRegion::new( + "East US", + Url::parse(GATEWAY_URL).unwrap(), + )]) + .unwrap() + .with_consistency(ConsistencyLevel::Session); + + let emulator = Arc::new(InMemoryEmulatorHttpClient::new(config)); + let store = emulator.store(); + store.create_database("testdb"); + store.create_container( + "testdb", + "testcoll", + serde_json::from_value(serde_json::json!({ + "paths": ["/pk"], + "kind": "Hash", + "version": 2 + })) + .unwrap(), + ); + + let runtime = emulator + .runtime_builder_with_fault_rules(rules) + .build() + .await + .expect("runtime should build against the in-memory emulator"); + + let account = + AccountReference::with_master_key(Url::parse(GATEWAY_URL).unwrap(), "ZW11bGF0b3Ita2V5"); + runtime + .create_driver(DriverOptions::builder(account).build()) + .await + .expect("driver should initialize") +} + +async fn seed(driver: &CosmosDriver) -> ContainerReference { + let container = driver + .resolve_container("testdb", "testcoll") + .await + .expect("container should resolve"); + let item = ItemReference::from_name(&container, PartitionKey::from(PK), ITEM_ID.to_string()); + let body = + serde_json::json!({ "id": ITEM_ID, "pk": PK, "visits": 1, "name": "before", "tags": [] }); + driver + .execute_operation( + CosmosOperation::create_item(item).with_body(body.to_string().into_bytes()), + OperationOptions::default(), + ) + .await + .expect("seeding must succeed before faults are armed"); + container +} + +async fn patch( + driver: &CosmosDriver, + container: &ContainerReference, + ops: PatchInstructions, + strategy: PatchStrategy, +) -> Result<(), azure_data_cosmos_driver::error::CosmosError> { + let item = ItemReference::from_name(container, PartitionKey::from(PK), ITEM_ID.to_string()); + let operation = CosmosOperation::patch_item(item).with_body(serde_json::to_vec(&ops).unwrap()); + let options = OperationOptionsBuilder::new() + .with_patch_strategy(strategy) + .build(); + driver + .execute_operation(operation, options) + .await + .map(|_| ()) +} + +async fn stored_visits(driver: &CosmosDriver, container: &ContainerReference) -> i64 { + stored_item(driver, container).await["visits"] + .as_i64() + .expect("visits is an integer") +} + +async fn stored_tag_count(driver: &CosmosDriver, container: &ContainerReference) -> usize { + stored_item(driver, container).await["tags"] + .as_array() + .expect("tags is an array") + .len() +} + +async fn stored_item(driver: &CosmosDriver, container: &ContainerReference) -> serde_json::Value { + let item = ItemReference::from_name(container, PartitionKey::from(PK), ITEM_ID.to_string()); + let response = driver + .execute_operation( + CosmosOperation::read_item(item), + OperationOptions::default(), + ) + .await + .expect("read back must succeed") + .expect("read must return a response"); + let bytes = response.into_body().single().expect("point read body"); + serde_json::from_slice(&bytes).expect("body is JSON") +} + +fn increment() -> PatchInstructions { + PatchInstructions::from(vec![PatchOperation::increment("/visits", 1i64)]) +} + +fn set_name() -> PatchInstructions { + PatchInstructions::from(vec![PatchOperation::set( + "/name", + serde_json::json!("after"), + )]) +} + +/// `Set` at the RFC 6901 append token appends, exactly like `Add` — the mode +/// does not matter at `-`. +fn append_tag() -> PatchInstructions { + PatchInstructions::from(vec![PatchOperation::set( + "/tags/-", + serde_json::json!("new-tag"), + )]) +} + +// ── Server-side: the operation list decides whether a resend happens ── + +/// The core safety property. An `increment` sent to the service and lost to an +/// ambiguous failure must not be resent — the first attempt may already have +/// applied it, and a second would double it. +#[tokio::test] +async fn server_side_unsafe_patch_is_not_retried_after_ambiguous_failure() { + let rule = ambiguous_failure_rule("patch-timeout", FaultOperationType::PatchItem, None); + let driver = build_driver(vec![Arc::clone(&rule)]).await; + let container = seed(&driver).await; + + let outcome = patch(&driver, &container, increment(), PatchStrategy::ServerSide).await; + + assert!( + outcome.is_err(), + "the operation must surface the failure rather than silently retrying" + ); + assert_eq!( + rule.hit_count(), + 1, + "an unsafe server-side patch must be attempted exactly once; \ + {} attempts means the driver resent a mutation whose outcome was unknown", + rule.hit_count() + ); +} + +/// The counterpart: the block is a property of the operations, not of +/// server-side patch as such. A `set` is safe to resend, so the driver still +/// spends its failover budget on it. +#[tokio::test] +async fn server_side_safe_patch_is_retried_after_ambiguous_failure() { + let rule = ambiguous_failure_rule("patch-timeout", FaultOperationType::PatchItem, None); + let driver = build_driver(vec![Arc::clone(&rule)]).await; + let container = seed(&driver).await; + + let outcome = patch(&driver, &container, set_name(), PatchStrategy::ServerSide).await; + + assert!(outcome.is_err(), "every attempt was faulted, so it fails"); + assert!( + rule.hit_count() > 1, + "a retry-safe server-side patch must still be retried, got {} attempt(s)", + rule.hit_count() + ); +} + +/// A transient blip should not be fatal: when only the first attempt fails, the +/// retry succeeds and the value is applied once, not twice. +#[tokio::test] +async fn server_side_safe_patch_recovers_when_only_the_first_attempt_fails() { + let rule = ambiguous_failure_rule("patch-timeout", FaultOperationType::PatchItem, Some(1)); + let driver = build_driver(vec![Arc::clone(&rule)]).await; + let container = seed(&driver).await; + + patch(&driver, &container, set_name(), PatchStrategy::ServerSide) + .await + .expect("the retry must recover from a single transient failure"); + + assert_eq!(rule.hit_count(), 1, "only the first attempt was faulted"); + assert_eq!( + stored_visits(&driver, &container).await, + 1, + "a `set` retry must not disturb unrelated fields" + ); +} + +// ── Client-side and Auto keep retrying unsafe operations ───────────── + +/// The read-modify-write loop re-reads before each attempt, so an `increment` +/// is safe to retry there. This is the behavior that existed before +/// server-side patch and must be preserved. +#[tokio::test] +async fn client_side_unsafe_patch_is_still_retried_after_ambiguous_failure() { + // The loop's mutation is an inner Replace, so that is what to fault. + let rule = ambiguous_failure_rule("replace-timeout", FaultOperationType::ReplaceItem, None); + let driver = build_driver(vec![Arc::clone(&rule)]).await; + let container = seed(&driver).await; + + let outcome = patch(&driver, &container, increment(), PatchStrategy::ClientSide).await; + + assert!(outcome.is_err(), "every attempt was faulted, so it fails"); + assert!( + rule.hit_count() > 1, + "the client-side loop must keep retrying an increment, got {} attempt(s)", + rule.hit_count() + ); +} + +/// End-to-end proof that the client-side path applies an increment exactly once +/// when it recovers — the duplicate this whole design is guarding against. +#[tokio::test] +async fn client_side_unsafe_patch_applies_once_when_it_recovers() { + let rule = ambiguous_failure_rule("replace-timeout", FaultOperationType::ReplaceItem, Some(1)); + let driver = build_driver(vec![Arc::clone(&rule)]).await; + let container = seed(&driver).await; + + patch(&driver, &container, increment(), PatchStrategy::ClientSide) + .await + .expect("the loop must recover from a single transient failure"); + + assert_eq!(rule.hit_count(), 1, "only the first Replace was faulted"); + assert_eq!( + stored_visits(&driver, &container).await, + 2, + "the increment must land exactly once after the retry" + ); +} + +/// `Auto` routes an increment to the client-side loop, so a fault armed on the +/// service's patch endpoint never fires at all — the request is not sent there. +/// This is what makes `Auto` safe by default without disabling retries. +#[tokio::test] +async fn auto_keeps_unsafe_patches_off_the_service_patch_path() { + let rule = ambiguous_failure_rule("patch-timeout", FaultOperationType::PatchItem, None); + let driver = build_driver(vec![Arc::clone(&rule)]).await; + let container = seed(&driver).await; + + patch(&driver, &container, increment(), PatchStrategy::Auto) + .await + .expect("Auto must fall back to the loop and succeed"); + + assert_eq!( + rule.hit_count(), + 0, + "Auto must not send an unsafe patch to the service patch endpoint" + ); + assert_eq!( + stored_visits(&driver, &container).await, + 2, + "the increment must land exactly once" + ); +} + +/// `Auto` with safe operations does use the service, so the same fault fires +/// and — because the list is retry-safe — is retried. +#[tokio::test] +async fn auto_sends_safe_patches_to_the_service_and_retries_them() { + let rule = ambiguous_failure_rule("patch-timeout", FaultOperationType::PatchItem, None); + let driver = build_driver(vec![Arc::clone(&rule)]).await; + let container = seed(&driver).await; + + let outcome = patch(&driver, &container, set_name(), PatchStrategy::Auto).await; + + assert!(outcome.is_err(), "every attempt was faulted, so it fails"); + assert!( + rule.hit_count() > 1, + "a safe patch under Auto goes to the service and is retried, got {} attempt(s)", + rule.hit_count() + ); +} + +/// Regression guard: `Set` was once classified retry-safe unconditionally, so +/// `set("/tags/-", ..)` — which appends — would have gone server-side under the +/// default strategy and been resent after an ambiguous failure, appending twice. +#[tokio::test] +async fn auto_keeps_array_append_set_off_the_service_patch_path() { + let rule = ambiguous_failure_rule("patch-timeout", FaultOperationType::PatchItem, None); + let driver = build_driver(vec![Arc::clone(&rule)]).await; + let container = seed(&driver).await; + + patch(&driver, &container, append_tag(), PatchStrategy::Auto) + .await + .expect("Auto must fall back to the loop and succeed"); + + assert_eq!( + rule.hit_count(), + 0, + "an append is not retry-safe, so Auto must not send it to the service" + ); + assert_eq!( + stored_tag_count(&driver, &container).await, + 1, + "the append must land exactly once" + ); +} + +/// The same append sent server-side on purpose must not be resent — the first +/// attempt may already have appended. +#[tokio::test] +async fn server_side_array_append_set_is_not_retried_after_ambiguous_failure() { + let rule = ambiguous_failure_rule("patch-timeout", FaultOperationType::PatchItem, None); + let driver = build_driver(vec![Arc::clone(&rule)]).await; + let container = seed(&driver).await; + + let outcome = patch(&driver, &container, append_tag(), PatchStrategy::ServerSide).await; + + assert!(outcome.is_err(), "the failure must be surfaced"); + assert_eq!( + rule.hit_count(), + 1, + "an append must be attempted exactly once; {} attempts means the driver \ + resent a mutation whose outcome was unknown", + rule.hit_count() + ); +} + +/// A failure that definitively never left the client is safe for every +/// operation type, so even an unsafe server-side patch is retried. This +/// separates "unsafe to resend" from "never retry", which are different +/// claims — only the ambiguous case is blocked. +#[tokio::test] +async fn server_side_unsafe_patch_is_retried_when_the_request_was_never_sent() { + let condition = FaultInjectionConditionBuilder::new() + .with_operation_type(FaultOperationType::PatchItem) + .build(); + let result = FaultInjectionResultBuilder::new() + .with_error(FaultInjectionErrorType::ConnectionError) + .with_probability(1.0) + .build(); + let rule = Arc::new( + FaultInjectionRuleBuilder::new("patch-connect-fail", result) + .with_condition(condition) + .build(), + ); + + let driver = build_driver(vec![Arc::clone(&rule)]).await; + let container = seed(&driver).await; + + let outcome = patch(&driver, &container, increment(), PatchStrategy::ServerSide).await; + + assert!(outcome.is_err(), "every attempt was faulted, so it fails"); + assert!( + rule.hit_count() > 1, + "a definitively-unsent request is safe to retry for any operation, got {} attempt(s)", + rule.hit_count() + ); +} diff --git a/sdk/cosmos/azure_data_cosmos_driver_native/README.md b/sdk/cosmos/azure_data_cosmos_driver_native/README.md index 910624a9ef6..25e3f527044 100644 --- a/sdk/cosmos/azure_data_cosmos_driver_native/README.md +++ b/sdk/cosmos/azure_data_cosmos_driver_native/README.md @@ -31,7 +31,7 @@ for the full design. ### Capability matrix (current) | Capability | Status | -|---|---| +| --- | --- | | Master-key authentication | ✅ | | Token-credential / resource-token authentication | ⏳ follow-up (needs `TokenCredential` FFI bridge) | | Sync driver creation (`_blocking`) | ✅ | @@ -39,7 +39,8 @@ for the full design. | Cache-hit advisory (`5001 OPTIONS_IGNORED_ON_CACHE_HIT`) | ⏳ needs driver-side `was_cached` signal | | Sync + async `resolve_container` | ✅ | | Single + hierarchical partition keys | ✅ | -| Item-CRUD operations (read / create / upsert / replace / delete / patch) | ✅ | +| Item-CRUD operations (read / create / upsert / replace / delete) | ✅ | +| Item PATCH | ✅ (preview exposure is controlled by the consuming SDK) | | Container-CRUD operations (read / replace / delete) | ✅ | | Database + account-scope operations | ✅ | | `cosmos_submit_singleton_operation` (point ops) | ✅ | @@ -150,6 +151,10 @@ below for the production-shape guidance. > (queries, read-all, change feed); resumes from and surfaces a continuation > token. > +> Item PATCH and `patch_max_attempts` are always available through the native +> driver ABI. Consuming language SDKs decide whether and how to expose PATCH as +> preview using conventions appropriate to that language. +> > Both take `(driver, const cosmos_CosmosOperationRequest *request, queue, > user_data, out_pre_error)` and return a `cosmos_operation_handle_t *`. > The checked-in [header](https://github.com/Azure/azure-sdk-for-rust/blob/main/sdk/cosmos/azure_data_cosmos_driver_native/include/azurecosmosdriver.h) is the authoritative diff --git a/sdk/cosmos/azure_data_cosmos_driver_native/include/azurecosmosdriver.h b/sdk/cosmos/azure_data_cosmos_driver_native/include/azurecosmosdriver.h index 714a32dddab..43ec1b7cd19 100644 --- a/sdk/cosmos/azure_data_cosmos_driver_native/include/azurecosmosdriver.h +++ b/sdk/cosmos/azure_data_cosmos_driver_native/include/azurecosmosdriver.h @@ -741,8 +741,8 @@ enum cosmos_sub_status_t */ COSMOS_SUB_STATUS_CLIENT_FFI_PRECONDITION_ALREADY_SET = 20355, /** - * `CLIENT_FFI_UNSUPPORTED_OPERATION_FOR_MUTATOR` (20356). Reserved: mirrors - * the driver constant but no current wrapper path produces it. + * `CLIENT_FFI_UNSUPPORTED_OPERATION_FOR_MUTATOR` (20356). Returned when a + * request uses an operation that is unavailable in this wrapper build. */ COSMOS_SUB_STATUS_CLIENT_FFI_UNSUPPORTED_OPERATION_FOR_MUTATOR = 20356, /** diff --git a/sdk/cosmos/azure_data_cosmos_driver_native/src/error.rs b/sdk/cosmos/azure_data_cosmos_driver_native/src/error.rs index ad52ab557bd..259ccb14787 100644 --- a/sdk/cosmos/azure_data_cosmos_driver_native/src/error.rs +++ b/sdk/cosmos/azure_data_cosmos_driver_native/src/error.rs @@ -217,8 +217,8 @@ pub enum CosmosSubStatus { /// `CLIENT_FFI_PRECONDITION_ALREADY_SET` (20355). Reserved: mirrors the /// driver constant but no current wrapper path produces it. CosmosSubStatusClientFfiPreconditionAlreadySet = 20355, - /// `CLIENT_FFI_UNSUPPORTED_OPERATION_FOR_MUTATOR` (20356). Reserved: mirrors - /// the driver constant but no current wrapper path produces it. + /// `CLIENT_FFI_UNSUPPORTED_OPERATION_FOR_MUTATOR` (20356). Returned when a + /// request uses an operation that is unavailable in this wrapper build. CosmosSubStatusClientFfiUnsupportedOperationForMutator = 20356, /// `CLIENT_FFI_FEED_EXHAUSTED` (20357). CosmosSubStatusClientFfiFeedExhausted = 20357, diff --git a/sdk/cosmos/eng/scripts/Invoke-CosmosDefaultFeatureCheck.ps1 b/sdk/cosmos/eng/scripts/Invoke-CosmosDefaultFeatureCheck.ps1 index 5044da412de..143df61a022 100644 --- a/sdk/cosmos/eng/scripts/Invoke-CosmosDefaultFeatureCheck.ps1 +++ b/sdk/cosmos/eng/scripts/Invoke-CosmosDefaultFeatureCheck.ps1 @@ -20,6 +20,7 @@ if ($env:AZURE_COSMOS_DEFAULT_FEATURE_CHECK_COMPLETE -eq 'true') { $packages = @( 'azure_data_cosmos' 'azure_data_cosmos_driver' + 'azure_data_cosmos_driver_native' ) $packageArgs = '--package ' + ($packages -join ' --package ')