Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,10 +38,12 @@ This package is the home for these APIs. Development and API design take place o

#### Other useful asynchronous sequences
- [`adjacentPairs()`](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncAlgorithms.docc/Guides/AdjacentPairs.md): Collects tuples of adjacent elements.
- [`buffer(policy:)`](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncAlgorithms.docc/Guides/Buffer.md): Store elements produced by an asynchronous sequence until the consumer is ready for them, according to a buffering policy.
- [`chunks(...)` and `chunked(...)`](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncAlgorithms.docc/Guides/Chunked.md): Collect values into chunks.
- [`compacted()`](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncAlgorithms.docc/Guides/Compacted.md): Remove nil values from an asynchronous sequence.
- [`removeDuplicates()`](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncAlgorithms.docc/Guides/RemoveDuplicates.md): Remove sequentially adjacent duplicate values.
- [`interspersed(with:)`](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncAlgorithms.docc/Guides/Intersperse.md): Place a value between every two elements of an asynchronous sequence.
- [`mapError(_:)`](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncAlgorithms.docc/Guides/MapError.md): Convert the failure of an asynchronous sequence into a new error.

#### Asynchronous Sequences that transact in time

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ This package has three main goals:
### Getting Started

- <doc:AdjacentPairs>
- <doc:Buffer>
- <doc:BufferedBytes>
- <doc:Chain>
- <doc:Channel>
Expand All @@ -24,9 +25,11 @@ This package has three main goals:
- <doc:Compacted>
- <doc:Debounce>
- <doc:Effects>
- <doc:FlatMapLatest>
- <doc:Intersperse>
- <doc:Joined>
- <doc:Lazy>
- <doc:MapError>
- <doc:Merge>
- <doc:Reductions>
- <doc:RemoveDuplicates>
Expand Down
75 changes: 75 additions & 0 deletions Sources/AsyncAlgorithms/AsyncAlgorithms.docc/Guides/Buffer.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
# Buffer

* Author(s): [Thibault Wittemberg](https://github.com/twittemb)

[
[Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/Buffer/AsyncBufferSequence.swift) |
[Tests](https://github.com/apple/swift-async-algorithms/blob/main/Tests/AsyncAlgorithmsTests/TestBuffer.swift)
]

## Introduction

An `AsyncSequence` iterates in lock step with its consumer: the base sequence is only asked for the next element once the consumer has finished handling the previous one. That back pressure is usually desirable, but it means a slow consumer also slows down the production of elements. When the producer is time sensitive — a timer, a socket, a stream of notifications — the values that arrive while the consumer is busy should be kept for later rather than delaying the producer.

## Proposed Solution

A `buffer(policy:)` method is available on any `Sendable` `AsyncSequence`. It returns an asynchronous sequence that consumes the base sequence in a separate task, storing the elements it receives until the consumer is ready for them.

```swift
extension AsyncSequence where Self: Sendable {
public func buffer(
policy: AsyncBufferSequencePolicy
) -> AsyncBufferSequence<Self>
}
```

The policy determines what happens when elements are produced faster than they are consumed.

```swift
public struct AsyncBufferSequencePolicy: Sendable {
public static func bounded(_ limit: Int) -> Self
public static var unbounded: Self { get }
public static func bufferingLatest(_ limit: Int) -> Self
public static func bufferingOldest(_ limit: Int) -> Self
}
```

- `bounded(_:)` buffers up to `limit` elements and then suspends the iteration of the base sequence until the consumer drains the buffer. No element is ever discarded, and back pressure is preserved beyond the limit.
- `unbounded` buffers every element produced by the base sequence. No element is discarded and the base sequence is never suspended, so memory usage is bounded only by how far the consumer falls behind.
- `bufferingLatest(_:)` keeps the `limit` most recent elements. Once the buffer is full, the oldest buffered element is discarded to make room for the newly produced one.
- `bufferingOldest(_:)` keeps the `limit` first elements. Once the buffer is full, newly produced elements are discarded.

Passing a limit of `0` to any of the limited policies disables buffering entirely: the resulting sequence iterates the base sequence directly, exactly as if `buffer(policy:)` had not been applied.

## Detailed Design

```swift
public struct AsyncBufferSequence<Base: AsyncSequence & Sendable>: AsyncSequence {
public typealias Element = Base.Element

public struct Iterator: AsyncIteratorProtocol {
public mutating func next() async rethrows -> Element?
}

public func makeAsyncIterator() -> Iterator
}

extension AsyncBufferSequence: Sendable where Base: Sendable { }

@available(*, unavailable)
extension AsyncBufferSequence.Iterator: Sendable { }
```

The base sequence is only iterated once a consumer asks for the first element; the task that drains it is created lazily by the first call to `next()`. When the base sequence finishes, the remaining buffered elements are still delivered to the consumer before the buffered sequence itself finishes.

When the base sequence throws, the failure is delivered in the order it was produced: elements already buffered ahead of the failure are emitted first, and the error is thrown afterwards. As with any throwing base sequence, iteration terminates at that point. `AsyncBufferSequence` rethrows, so buffering a non-throwing sequence produces a non-throwing sequence.

Cancelling the consuming task also terminates the iteration of the base sequence.

## Effect on API resilience

This is an additive API.

## Credits/Inspiration

The buffering policies are shaped after the `AsyncStream.Continuation.BufferingPolicy` type from the standard library, extended with the `bounded(_:)` case that preserves back pressure instead of discarding elements.
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,7 @@
[
[Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncChunkedByGroupSequence.swift),
[Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncChunkedOnProjectionSequence.swift),
[Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncChunkedOnProjectionSequence.swift),
[Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncChunksOfCountAndSignalSequence.swift),
[Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncChunksOfCountOrSignalSequence.swift),
[Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncChunksOfCountSequence.swift) |
[Tests](https://github.com/apple/swift-async-algorithms/blob/main/Tests/AsyncAlgorithmsTests/TestChunk.swift)
]
Expand Down
66 changes: 66 additions & 0 deletions Sources/AsyncAlgorithms/AsyncAlgorithms.docc/Guides/MapError.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
# MapError

* Author(s): [Philippe Hausler](https://github.com/phausler)

[
[Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncMapErrorSequence.swift) |
[Tests](https://github.com/apple/swift-async-algorithms/blob/main/Tests/AsyncAlgorithmsTests/TestMapError.swift)
]

## Introduction

With [SE-0421](https://github.com/swiftlang/swift-evolution/blob/main/proposals/0421-generalize-async-sequence.md), an `AsyncSequence` carries the type of the failure it can throw as an associated `Failure` type. That makes the error type part of the interface an asynchronous sequence exposes, and it means an API that vends an asynchronous sequence has to decide which errors its callers are expected to handle.

Composing sequences that come from different sources therefore runs into the same problem the synchronous world has: the failure type produced deep inside an implementation is rarely the failure type that should be visible at the boundary of a module.

## Proposed Solution

A `mapError(_:)` method transforms the failure of an asynchronous sequence, leaving its elements untouched. It is the error-side counterpart of `map(_:)`.

```swift
extension AsyncSequence {
public func mapError<MappedError: Error>(
_ transform: @Sendable @escaping (Self.Failure) -> MappedError
) -> some AsyncSequence<Self.Element, MappedError>

public func mapError<MappedError: Error>(
_ transform: @Sendable @escaping (Self.Failure) -> MappedError
) -> (some AsyncSequence<Self.Element, MappedError> & Sendable)
where Self: Sendable, Self.Element: Sendable
}
```

This lets a module wrap the failures of the sequences it composes into an error type of its own:

```swift
struct ConnectionError: Error {
let underlying: any Error
}

func messages() -> some AsyncSequence<Message, ConnectionError> {
socket
.lines
.map(Message.init(parsing:))
.mapError(ConnectionError.init(underlying:))
}
```

Because the transform receives the concrete `Failure` type of the base sequence, it can also be used to narrow an existing `any Error` failure down to a specific type, which restores typed throws for callers of the resulting sequence.

## Detailed Design

Two overloads are provided. The second one is chosen when both the base sequence and its element are `Sendable`, and additionally guarantees that the resulting sequence is `Sendable` so that it can cross isolation boundaries.

The returned sequence is opaque; only its `Element` and `Failure` types are part of the API. It forwards `next(isolation:)` to the base iterator and applies the transform to any error the base sequence throws. The transform is only invoked when the base sequence actually fails — it is never called for a sequence that finishes normally, and it is invoked at most once per iteration since a failure terminates the sequence.

The iterator is not `Sendable`, matching the other sequences in this package.

This algorithm requires a Swift 6.0 or later compiler.

## Effect on API resilience

This is an additive API.

## Credits/Inspiration

This is a direct analog of the `mapError(_:)` operator found in Combine, adapted to the typed failures of `AsyncSequence`.