diff --git a/README.md b/README.md index c49c9c6c..23fe387f 100644 --- a/README.md +++ b/README.md @@ -38,10 +38,12 @@ This package is the home for these APIs. Development and API design take place o #### Other useful asynchronous sequences - [`adjacentPairs()`](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncAlgorithms.docc/Guides/AdjacentPairs.md): Collects tuples of adjacent elements. +- [`buffer(policy:)`](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncAlgorithms.docc/Guides/Buffer.md): Store elements produced by an asynchronous sequence until the consumer is ready for them, according to a buffering policy. - [`chunks(...)` and `chunked(...)`](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncAlgorithms.docc/Guides/Chunked.md): Collect values into chunks. - [`compacted()`](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncAlgorithms.docc/Guides/Compacted.md): Remove nil values from an asynchronous sequence. - [`removeDuplicates()`](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncAlgorithms.docc/Guides/RemoveDuplicates.md): Remove sequentially adjacent duplicate values. - [`interspersed(with:)`](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncAlgorithms.docc/Guides/Intersperse.md): Place a value between every two elements of an asynchronous sequence. +- [`mapError(_:)`](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncAlgorithms.docc/Guides/MapError.md): Convert the failure of an asynchronous sequence into a new error. #### Asynchronous Sequences that transact in time diff --git a/Sources/AsyncAlgorithms/AsyncAlgorithms.docc/AsyncAlgorithms.md b/Sources/AsyncAlgorithms/AsyncAlgorithms.docc/AsyncAlgorithms.md index 8461d28a..35a2dfad 100644 --- a/Sources/AsyncAlgorithms/AsyncAlgorithms.docc/AsyncAlgorithms.md +++ b/Sources/AsyncAlgorithms/AsyncAlgorithms.docc/AsyncAlgorithms.md @@ -15,6 +15,7 @@ This package has three main goals: ### Getting Started - +- - - - @@ -24,9 +25,11 @@ This package has three main goals: - - - +- - - - +- - - - diff --git a/Sources/AsyncAlgorithms/AsyncAlgorithms.docc/Guides/Buffer.md b/Sources/AsyncAlgorithms/AsyncAlgorithms.docc/Guides/Buffer.md new file mode 100644 index 00000000..932762cf --- /dev/null +++ b/Sources/AsyncAlgorithms/AsyncAlgorithms.docc/Guides/Buffer.md @@ -0,0 +1,75 @@ +# Buffer + +* Author(s): [Thibault Wittemberg](https://github.com/twittemb) + +[ +[Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/Buffer/AsyncBufferSequence.swift) | +[Tests](https://github.com/apple/swift-async-algorithms/blob/main/Tests/AsyncAlgorithmsTests/TestBuffer.swift) +] + +## Introduction + +An `AsyncSequence` iterates in lock step with its consumer: the base sequence is only asked for the next element once the consumer has finished handling the previous one. That back pressure is usually desirable, but it means a slow consumer also slows down the production of elements. When the producer is time sensitive — a timer, a socket, a stream of notifications — the values that arrive while the consumer is busy should be kept for later rather than delaying the producer. + +## Proposed Solution + +A `buffer(policy:)` method is available on any `Sendable` `AsyncSequence`. It returns an asynchronous sequence that consumes the base sequence in a separate task, storing the elements it receives until the consumer is ready for them. + +```swift +extension AsyncSequence where Self: Sendable { + public func buffer( + policy: AsyncBufferSequencePolicy + ) -> AsyncBufferSequence +} +``` + +The policy determines what happens when elements are produced faster than they are consumed. + +```swift +public struct AsyncBufferSequencePolicy: Sendable { + public static func bounded(_ limit: Int) -> Self + public static var unbounded: Self { get } + public static func bufferingLatest(_ limit: Int) -> Self + public static func bufferingOldest(_ limit: Int) -> Self +} +``` + +- `bounded(_:)` buffers up to `limit` elements and then suspends the iteration of the base sequence until the consumer drains the buffer. No element is ever discarded, and back pressure is preserved beyond the limit. +- `unbounded` buffers every element produced by the base sequence. No element is discarded and the base sequence is never suspended, so memory usage is bounded only by how far the consumer falls behind. +- `bufferingLatest(_:)` keeps the `limit` most recent elements. Once the buffer is full, the oldest buffered element is discarded to make room for the newly produced one. +- `bufferingOldest(_:)` keeps the `limit` first elements. Once the buffer is full, newly produced elements are discarded. + +Passing a limit of `0` to any of the limited policies disables buffering entirely: the resulting sequence iterates the base sequence directly, exactly as if `buffer(policy:)` had not been applied. + +## Detailed Design + +```swift +public struct AsyncBufferSequence: AsyncSequence { + public typealias Element = Base.Element + + public struct Iterator: AsyncIteratorProtocol { + public mutating func next() async rethrows -> Element? + } + + public func makeAsyncIterator() -> Iterator +} + +extension AsyncBufferSequence: Sendable where Base: Sendable { } + +@available(*, unavailable) +extension AsyncBufferSequence.Iterator: Sendable { } +``` + +The base sequence is only iterated once a consumer asks for the first element; the task that drains it is created lazily by the first call to `next()`. When the base sequence finishes, the remaining buffered elements are still delivered to the consumer before the buffered sequence itself finishes. + +When the base sequence throws, the failure is delivered in the order it was produced: elements already buffered ahead of the failure are emitted first, and the error is thrown afterwards. As with any throwing base sequence, iteration terminates at that point. `AsyncBufferSequence` rethrows, so buffering a non-throwing sequence produces a non-throwing sequence. + +Cancelling the consuming task also terminates the iteration of the base sequence. + +## Effect on API resilience + +This is an additive API. + +## Credits/Inspiration + +The buffering policies are shaped after the `AsyncStream.Continuation.BufferingPolicy` type from the standard library, extended with the `bounded(_:)` case that preserves back pressure instead of discarding elements. diff --git a/Sources/AsyncAlgorithms/AsyncAlgorithms.docc/Guides/Chunked.md b/Sources/AsyncAlgorithms/AsyncAlgorithms.docc/Guides/Chunked.md index 94389220..41f1cd1b 100644 --- a/Sources/AsyncAlgorithms/AsyncAlgorithms.docc/Guides/Chunked.md +++ b/Sources/AsyncAlgorithms/AsyncAlgorithms.docc/Guides/Chunked.md @@ -5,8 +5,7 @@ [ [Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncChunkedByGroupSequence.swift), [Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncChunkedOnProjectionSequence.swift), -[Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncChunkedOnProjectionSequence.swift), -[Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncChunksOfCountAndSignalSequence.swift), +[Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncChunksOfCountOrSignalSequence.swift), [Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncChunksOfCountSequence.swift) | [Tests](https://github.com/apple/swift-async-algorithms/blob/main/Tests/AsyncAlgorithmsTests/TestChunk.swift) ] diff --git a/Sources/AsyncAlgorithms/AsyncAlgorithms.docc/Guides/MapError.md b/Sources/AsyncAlgorithms/AsyncAlgorithms.docc/Guides/MapError.md new file mode 100644 index 00000000..9c8e8787 --- /dev/null +++ b/Sources/AsyncAlgorithms/AsyncAlgorithms.docc/Guides/MapError.md @@ -0,0 +1,66 @@ +# MapError + +* Author(s): [Philippe Hausler](https://github.com/phausler) + +[ +[Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncMapErrorSequence.swift) | +[Tests](https://github.com/apple/swift-async-algorithms/blob/main/Tests/AsyncAlgorithmsTests/TestMapError.swift) +] + +## Introduction + +With [SE-0421](https://github.com/swiftlang/swift-evolution/blob/main/proposals/0421-generalize-async-sequence.md), an `AsyncSequence` carries the type of the failure it can throw as an associated `Failure` type. That makes the error type part of the interface an asynchronous sequence exposes, and it means an API that vends an asynchronous sequence has to decide which errors its callers are expected to handle. + +Composing sequences that come from different sources therefore runs into the same problem the synchronous world has: the failure type produced deep inside an implementation is rarely the failure type that should be visible at the boundary of a module. + +## Proposed Solution + +A `mapError(_:)` method transforms the failure of an asynchronous sequence, leaving its elements untouched. It is the error-side counterpart of `map(_:)`. + +```swift +extension AsyncSequence { + public func mapError( + _ transform: @Sendable @escaping (Self.Failure) -> MappedError + ) -> some AsyncSequence + + public func mapError( + _ transform: @Sendable @escaping (Self.Failure) -> MappedError + ) -> (some AsyncSequence & Sendable) + where Self: Sendable, Self.Element: Sendable +} +``` + +This lets a module wrap the failures of the sequences it composes into an error type of its own: + +```swift +struct ConnectionError: Error { + let underlying: any Error +} + +func messages() -> some AsyncSequence { + socket + .lines + .map(Message.init(parsing:)) + .mapError(ConnectionError.init(underlying:)) +} +``` + +Because the transform receives the concrete `Failure` type of the base sequence, it can also be used to narrow an existing `any Error` failure down to a specific type, which restores typed throws for callers of the resulting sequence. + +## Detailed Design + +Two overloads are provided. The second one is chosen when both the base sequence and its element are `Sendable`, and additionally guarantees that the resulting sequence is `Sendable` so that it can cross isolation boundaries. + +The returned sequence is opaque; only its `Element` and `Failure` types are part of the API. It forwards `next(isolation:)` to the base iterator and applies the transform to any error the base sequence throws. The transform is only invoked when the base sequence actually fails — it is never called for a sequence that finishes normally, and it is invoked at most once per iteration since a failure terminates the sequence. + +The iterator is not `Sendable`, matching the other sequences in this package. + +This algorithm requires a Swift 6.0 or later compiler. + +## Effect on API resilience + +This is an additive API. + +## Credits/Inspiration + +This is a direct analog of the `mapError(_:)` operator found in Combine, adapted to the typed failures of `AsyncSequence`.