Skip to content

Commit 8090589

Browse files
ThorFuchsclaude
andauthored
Add JSONEnvelope, a versioned digest-checked envelope over JCS (#99)
Format version 1 is one RFC 8785 canonical object with digest, format, formatVersion, payload, type and version. The digest is the SHA-256 of the payload's canonical bytes; decoding verifies marker, fields and digest and reports JSONEnvelopeFailure for invalid or damaged envelopes. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
1 parent b3eef29 commit 8090589

7 files changed

Lines changed: 397 additions & 5 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -35,6 +35,14 @@
3535
default; `JSONTextSequenceOptions.MalformedTexts.skip` skips them and counts
3636
them on the iterator, but never skips resource failures. `JSONTextSequenceWriter`
3737
writes RS, strict JSON and LF per value, as one sequence or in awaited chunks.
38+
- Add `JSONEnvelope`, a versioned record of one payload in RFC 8785 canonical
39+
bytes. Format version 1 is a canonical object with `digest`, `format`
40+
(`swift-json-envelope`), `formatVersion` (1), `payload`, `type` and `version`.
41+
The digest is the SHA-256 of the payload's canonical bytes; decoding checks the
42+
marker, fields and digest and reports `JSONEnvelopeFailure` for invalid or
43+
damaged envelopes. Equal envelopes encode to equal bytes, and equal payloads
44+
have equal digests for content addressing. Tests pin bytes and digests computed
45+
independently.
3846

3947
## Unreleased — 1.9.1 hygiene
4048

‎Documentation/Qualification/TestDiscovery.json‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
{
22
"schema_version": 2,
33
"root_functions": {
4-
"JSONTests": 198,
4+
"JSONTests": 203,
55
"JSONSchemaTests": 79,
66
"JSONRegexRuntimeTests": 16,
77
"JSONStreamingTests": 42,
@@ -73,7 +73,7 @@
7373
"JSONSchemaDSL"
7474
],
7575
"portable_root_functions": {
76-
"JSONTests": 198,
76+
"JSONTests": 203,
7777
"JSONSchemaTests": 79,
7878
"JSONRegexRuntimeTests": 16,
7979
"JSONStreamingTests": 42,
@@ -89,7 +89,7 @@
8989
"JSONDSLTests": 47
9090
},
9191
"wasi_root_functions": {
92-
"JSONTests": 198,
92+
"JSONTests": 203,
9393
"JSONSchemaTests": 79,
9494
"JSONRegexRuntimeTests": 16,
9595
"JSONStreamingTests": 42,
Lines changed: 211 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,211 @@
1+
// SPDX-License-Identifier: Apache-2.0
2+
// Original versioned, digest-checked envelope over RFC 8785 canonical JSON.
3+
import Foundation
4+
5+
/// A versioned, self-checking record of one JSON payload, written as RFC 8785 canonical JSON.
6+
///
7+
/// An envelope names what its payload is (``type``) and which schema version of
8+
/// that type it follows (``version``), so a store can migrate old records. Its
9+
/// wire form is one canonical object with six members:
10+
///
11+
/// ```json
12+
/// {"digest":"sha256:…","format":"swift-json-envelope","formatVersion":1,"payload":{…},"type":"mcp.task","version":3}
13+
/// ```
14+
///
15+
/// `digest` is the SHA-256 of the payload's canonical bytes. Decoding recomputes
16+
/// it, so a damaged payload is detected, and equal payloads always have equal
17+
/// digests, so the digest can serve as a content address. Because the whole
18+
/// envelope is canonical, equal envelopes encode to equal bytes.
19+
///
20+
/// Format version 1 is fixed: the member names, the marker and the digest
21+
/// algorithm do not change. A later format would use a new `formatVersion`.
22+
public struct JSONEnvelope: Sendable, Equatable {
23+
/// The format marker, `swift-json-envelope`.
24+
public static let format = "swift-json-envelope"
25+
/// The format version this type reads and writes, `1`.
26+
public static let formatVersion = 1
27+
/// The largest accepted ``type`` length in UTF-8 bytes.
28+
public static let maximumTypeBytes = 256
29+
/// The accepted ``version`` range: the positive integers that binary64 represents exactly.
30+
public static let versions: ClosedRange<Int> = 1...9_007_199_254_740_991
31+
32+
/// The caller's name for the kind of payload, such as `mcp.task`.
33+
public let type: String
34+
/// The schema version of ``type`` that the payload follows.
35+
public let version: Int
36+
/// The enveloped value.
37+
public let payload: JSON
38+
39+
/// Creates an envelope after checking its type and version.
40+
///
41+
/// - Throws: ``JSONEnvelopeFailure/invalidType`` for an empty type or one longer
42+
/// than ``maximumTypeBytes``; ``JSONEnvelopeFailure/invalidVersion`` for a version
43+
/// outside ``versions``.
44+
///
45+
/// - Parameters:
46+
/// - type: A non-empty name for the kind of payload, at most 256 UTF-8 bytes.
47+
/// - version: The payload's schema version, from 1 through 2^53 − 1.
48+
/// - payload: The value to envelope.
49+
public init(type: String, version: Int, payload: JSON) throws {
50+
guard !type.isEmpty, type.utf8.count <= Self.maximumTypeBytes else { throw JSONEnvelopeFailure.invalidType }
51+
guard Self.versions.contains(version) else { throw JSONEnvelopeFailure.invalidVersion }
52+
self.type = type; self.version = version; self.payload = payload
53+
}
54+
55+
/// Writes the envelope's canonical bytes and returns them with the payload digest.
56+
///
57+
/// - Returns: The canonical envelope bytes and the `sha256:` digest of the payload.
58+
/// - Throws: Inexact conversion for a number the policy refuses, resource failure or cancellation.
59+
///
60+
/// - Parameters:
61+
/// - options: Limits and the canonical number policy.
62+
public func encode(options: JSONEnvelopeOptions = .init()) throws -> JSONEnvelopeEncoding {
63+
try encodeWithReport(options: options).result
64+
}
65+
/// Writes the envelope with one cumulative work ledger.
66+
///
67+
/// - Returns: The encoding and the ledger's charged counts.
68+
/// - Throws: Inexact conversion for a number the policy refuses, resource failure or cancellation.
69+
///
70+
/// - Parameters:
71+
/// - options: Limits and the canonical number policy.
72+
public func encodeWithReport(options: JSONEnvelopeOptions = .init()) throws -> JSONOperationReport<JSONEnvelopeEncoding> {
73+
var meter = try JSONWorkMeter(limits: options.limits, domain: .writing)
74+
let writer = JSONCanonicalWriter(limits: options.limits, numberPolicy: options.numberPolicy)
75+
let payloadBytes = try writer.write(payload, meter: &meter)
76+
let digest = try Self.digest(of: payloadBytes, meter: &meter)
77+
try meter.resetValueBounds(); let typeBytes = try writer.write(.string(type), meter: &meter)
78+
try meter.resetValueBounds(); let versionBytes = try writer.write(.number(JSONNumber(integer: version)), meter: &meter)
79+
// Members in RFC 8785 order; every name is ASCII, so UTF-16 order is byte order.
80+
var bytes: [UInt8] = []
81+
func append<C: Collection>(_ part: C, meter: inout JSONWorkMeter) throws where C.Element == UInt8 {
82+
for byte in part { try meter.work(); try meter.retain(1); bytes.append(byte) }
83+
}
84+
func literal(_ text: String, meter: inout JSONWorkMeter) throws {
85+
try meter.charge(.outputBytes, UInt64(text.utf8.count)); try append(text.utf8, meter: &meter)
86+
}
87+
try literal("{\"digest\":\"\(digest)\",\"format\":\"\(Self.format)\",\"formatVersion\":\(Self.formatVersion),\"payload\":", meter: &meter)
88+
try append(payloadBytes, meter: &meter)
89+
try literal(",\"type\":", meter: &meter); try append(typeBytes, meter: &meter)
90+
try literal(",\"version\":", meter: &meter); try append(versionBytes, meter: &meter)
91+
try literal("}", meter: &meter)
92+
try meter.checkpoint()
93+
return .init(result: .init(data: Data(bytes), digest: digest), statistics: meter.statistics())
94+
}
95+
96+
/// Reads an envelope and verifies its format, fields and payload digest.
97+
///
98+
/// The input may be any strict JSON spelling of the envelope object; canonical
99+
/// bytes are not required. Use the same number policy that encoded it.
100+
///
101+
/// - Returns: The checked envelope.
102+
/// - Throws: ``JSONEnvelopeFailure`` for a well-formed JSON value that is not a
103+
/// valid version 1 envelope or whose digest does not match; `JSONFailure` for
104+
/// syntax, a refused number, resource failure; or cancellation.
105+
///
106+
/// - Parameters:
107+
/// - data: The envelope bytes.
108+
/// - options: Limits and the canonical number policy used for the digest.
109+
public static func decode(_ data: Data, options: JSONEnvelopeOptions = .init()) throws -> JSONEnvelope {
110+
try decodeWithReport(data, options: options).result
111+
}
112+
/// Reads and verifies an envelope with one cumulative work ledger.
113+
///
114+
/// See ``decode(_:options:)``.
115+
///
116+
/// - Returns: The checked envelope and the ledger's charged counts.
117+
/// - Throws: ``JSONEnvelopeFailure``, `JSONFailure` or cancellation, as for ``decode(_:options:)``.
118+
///
119+
/// - Parameters:
120+
/// - data: The envelope bytes.
121+
/// - options: Limits and the canonical number policy used for the digest.
122+
public static func decodeWithReport(_ data: Data, options: JSONEnvelopeOptions = .init()) throws -> JSONOperationReport<JSONEnvelope> {
123+
var meter = try JSONWorkMeter(limits: options.limits, domain: .core)
124+
try meter.require(.inputBytes, UInt64(data.count))
125+
var input: [UInt8] = []
126+
for byte in data { try meter.work(); try meter.retain(1); input.append(byte) }
127+
let root = try JSONParser().parse(input, meter: &meter)
128+
try meter.release(UInt64(input.count)); input = []
129+
130+
guard case .object(let object) = root, object.count == 6 else { throw JSONEnvelopeFailure.invalidFormat }
131+
func member(_ name: String, meter: inout JSONWorkMeter) throws -> JSON {
132+
guard let value = try object.lookup(name, meter: &meter) else { throw JSONEnvelopeFailure.invalidFormat }
133+
return value
134+
}
135+
guard case .string(format) = try member("format", meter: &meter) else { throw JSONEnvelopeFailure.invalidFormat }
136+
guard case .number(let formatVersion) = try member("formatVersion", meter: &meter) else { throw JSONEnvelopeFailure.invalidFormat }
137+
guard try integer(formatVersion, meter: &meter) == Self.formatVersion else {
138+
throw JSONEnvelopeFailure.unsupportedFormatVersion
139+
}
140+
guard case .string(let type) = try member("type", meter: &meter) else { throw JSONEnvelopeFailure.invalidType }
141+
guard case .number(let versionNumber) = try member("version", meter: &meter),
142+
let version = try integer(versionNumber, meter: &meter) else { throw JSONEnvelopeFailure.invalidVersion }
143+
guard case .string(let digest) = try member("digest", meter: &meter), isDigest(digest) else { throw JSONEnvelopeFailure.invalidDigest }
144+
let payload = try member("payload", meter: &meter)
145+
let envelope = try JSONEnvelope(type: type, version: version, payload: payload)
146+
147+
try meter.resetValueBounds()
148+
let writer = JSONCanonicalWriter(limits: options.limits, numberPolicy: options.numberPolicy)
149+
let payloadBytes = try writer.write(payload, meter: &meter)
150+
guard try Self.digest(of: payloadBytes, meter: &meter) == digest else { throw JSONEnvelopeFailure.digestMismatch }
151+
try meter.checkpoint()
152+
return .init(result: envelope, statistics: meter.statistics())
153+
}
154+
155+
/// The integer value, or nil when the number is not an integer that fits `Int`.
156+
private static func integer(_ number: JSONNumber, meter: inout JSONWorkMeter) throws -> Int? {
157+
do { return try number.integer(as: Int.self, meter: &meter) }
158+
catch let failure as JSONFailure where failure.category == .conversion { return nil }
159+
}
160+
private static func digest(of bytes: [UInt8], meter: inout JSONWorkMeter) throws -> String {
161+
"sha256:" + (try JSONSHA256Core.hexadecimal(bytes, accounting: &meter))
162+
}
163+
private static func isDigest(_ text: String) -> Bool {
164+
let bytes = Array(text.utf8)
165+
guard bytes.count == 71, bytes.starts(with: Array("sha256:".utf8)) else { return false }
166+
return bytes.dropFirst(7).allSatisfy { (0x30...0x39).contains($0) || (0x61...0x66).contains($0) }
167+
}
168+
}
169+
170+
/// Limits and the number policy for encoding and decoding a ``JSONEnvelope``.
171+
public struct JSONEnvelopeOptions: Sendable {
172+
/// Finite capacities and work quotas applied to each call.
173+
public var limits: JSONLimits
174+
/// The canonical number policy for the payload; decoding must use the policy that encoded it.
175+
public var numberPolicy: JSONCanonicalWriter.NumberPolicy
176+
/// Creates the configuration; finite limits are checked when an operation begins.
177+
///
178+
/// - Parameters:
179+
/// - limits: Finite inclusive capacities and work quotas for each call.
180+
/// - numberPolicy: Refuse (the default) or round numbers that binary64 cannot represent exactly.
181+
public init(limits: JSONLimits = .balanced, numberPolicy: JSONCanonicalWriter.NumberPolicy = .preserving) {
182+
self.limits = limits; self.numberPolicy = numberPolicy
183+
}
184+
}
185+
186+
/// The canonical bytes of a ``JSONEnvelope`` and the digest of its payload.
187+
public struct JSONEnvelopeEncoding: Sendable, Equatable {
188+
/// The canonical envelope bytes.
189+
public let data: Data
190+
/// The payload digest, `sha256:` followed by 64 lowercase hexadecimal digits.
191+
public let digest: String
192+
}
193+
194+
/// Why a well-formed JSON value is not a valid version 1 ``JSONEnvelope``.
195+
///
196+
/// Syntax, resource and number-conversion failures are reported as `JSONFailure`.
197+
/// No case carries input text.
198+
public enum JSONEnvelopeFailure: Error, Sendable, Equatable {
199+
/// The root is not an object with exactly the six envelope members, or the format marker is wrong.
200+
case invalidFormat
201+
/// The format version is not 1.
202+
case unsupportedFormatVersion
203+
/// The type is not a non-empty string of at most 256 UTF-8 bytes.
204+
case invalidType
205+
/// The version is not an integer from 1 through 2^53 − 1.
206+
case invalidVersion
207+
/// The digest is not `sha256:` followed by 64 lowercase hexadecimal digits.
208+
case invalidDigest
209+
/// The payload's canonical digest differs from the recorded digest.
210+
case digestMismatch
211+
}

‎Sources/JSON/Documentation.docc/JSON.md‎

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,9 @@ preserved/normalized decimal spelling. Scalar sorting is deterministic JSON
1515
output, not a cryptographic canonicalization standard. For that, use
1616
``JSONCanonicalWriter``, which writes RFC 8785 (JCS) canonical bytes: UTF-16 key
1717
order, short string escapes and ECMAScript binary64 numbers. By default it refuses
18-
a number whose binary64 spelling would change its exact value.
18+
a number whose binary64 spelling would change its exact value. ``JSONEnvelope``
19+
builds on it: a typed, versioned payload in canonical bytes with a SHA-256
20+
digest that decoding verifies. See <doc:VersionedEnvelopes>.
1921

2022
``JSONValueEncoder`` and ``JSONValueDecoder`` implement native value-based
2123
Codable. Direct JSON/JSONObject round-trips preserve raw scalar keys and numbers.
@@ -60,6 +62,7 @@ belongs exclusively to each invocation.
6062
- <doc:QuickEntryPoints>
6163
- <doc:CommandLine>
6264
- <doc:ExactPersistence>
65+
- <doc:VersionedEnvelopes>
6366
- <doc:Composition>
6467

6568
### Exact model
@@ -75,6 +78,10 @@ belongs exclusively to each invocation.
7578
- ``JSONWriter``
7679
- ``JSONWritingOptions``
7780
- ``JSONCanonicalWriter``
81+
- ``JSONEnvelope``
82+
- ``JSONEnvelopeOptions``
83+
- ``JSONEnvelopeEncoding``
84+
- ``JSONEnvelopeFailure``
7885
- ``JSONIndentationUnit``
7986
- ``JSONValueEncoder``
8087
- ``JSONValueDecoder``
Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,61 @@
1+
# Versioned envelopes
2+
3+
Store a typed, versioned payload as canonical bytes with a content digest.
4+
5+
## Overview
6+
7+
A ``JSONEnvelope`` wraps one payload with the caller's name for its kind
8+
(``JSONEnvelope/type``) and the schema version of that kind
9+
(``JSONEnvelope/version``). It is meant for records that a store keeps for a
10+
long time, where later code must recognize what a record is, migrate older
11+
versions and notice damage.
12+
13+
```swift
14+
let envelope = try JSONEnvelope(type: "mcp.task", version: 3, payload: task)
15+
let encoding = try envelope.encode()
16+
store.save(encoding.data, key: encoding.digest)
17+
18+
let restored = try JSONEnvelope.decode(store.load(key))
19+
```
20+
21+
### Wire format
22+
23+
Format version 1 is one RFC 8785 canonical object with exactly six members:
24+
25+
```json
26+
{"digest":"sha256:87245ae1c062ec8fd5e2ab1ac60fbc44f6822d6176fe850381db1fd6e97c7810","format":"swift-json-envelope","formatVersion":1,"payload":{"id":7,"state":"done"},"type":"mcp.task","version":3}
27+
```
28+
29+
- `format` is always `swift-json-envelope`; `formatVersion` is always `1`.
30+
- `type` is a non-empty string of at most 256 UTF-8 bytes; `version` is an
31+
integer from 1 through 2^53 − 1.
32+
- `digest` is `sha256:` followed by the lowercase hexadecimal SHA-256 of the
33+
payload's canonical bytes.
34+
35+
Because the whole envelope is canonical, equal envelopes encode to equal bytes.
36+
Equal payloads, compared with the exact model's equality, have equal digests, so
37+
the digest works as a content address independent of the type and version.
38+
39+
### Decoding
40+
41+
``JSONEnvelope/decode(_:options:)`` accepts any strict JSON spelling of the
42+
object, recomputes the payload digest and compares it with the recorded one. A
43+
well-formed value that is not a valid envelope throws a ``JSONEnvelopeFailure``:
44+
a wrong marker or member set, an unsupported format version, an invalid type,
45+
version or digest field, or a digest mismatch. Syntax, resource and number
46+
failures stay `JSONFailure`.
47+
48+
### Numbers
49+
50+
The payload is canonicalized with ``JSONCanonicalWriter``. By default
51+
(``JSONCanonicalWriter/NumberPolicy/preserving``) a number that binary64 cannot
52+
spell exactly, such as `0.10000000000000001`, is refused rather than changed.
53+
With ``JSONCanonicalWriter/NumberPolicy/rounding`` such numbers are rounded
54+
before hashing, and decoding must use the same policy.
55+
56+
## Topics
57+
58+
- ``JSONEnvelope``
59+
- ``JSONEnvelopeOptions``
60+
- ``JSONEnvelopeEncoding``
61+
- ``JSONEnvelopeFailure``

‎Sources/JSONCLIEngine/CLIBuildIdentity.swift‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,6 @@
22
// Generated by Scripts/generate_cli_identity.py; excludes this file.
33
// SHA-256 of the ordered Package.swift + Sources content manifest (DocC catalogs excluded), not a Git commit.
44
package enum CLIBuildIdentity {
5-
package static let revision = "sha256:c0c4442af4a6e52b5e232fc41405acbf92616b366832ae0df0bfb843e18a3773"
5+
package static let revision = "sha256:fd026d72a5ef7b9f545d4ab4b08edd1460af5d0630551034a16f066824e1fa39"
66
package static let kind = "source-sha256"
77
}

0 commit comments

Comments
 (0)