Skip to content

Improve SMTP protocol support, streaming paths, benchmarks, and conformance - #284

Open
vhafdal wants to merge 45 commits into
cosullivan:masterfrom
vhafdal:nexer-prod
Open

Improve SMTP protocol support, streaming paths, benchmarks, and conformance#284
vhafdal wants to merge 45 commits into
cosullivan:masterfrom
vhafdal:nexer-prod

Conversation

@vhafdal

@vhafdal vhafdal commented Jul 15, 2026

Copy link
Copy Markdown

Summary

This draft PR aggregates the SMTP Server Jira work completed on nexer-prod from the fork. It is intended as a single review path for the protocol, streaming, benchmark, parser, limit, and EHLO conformance improvements instead of opening more dependent PRs.

Included Work

  • Tighten SMTP parser syntax handling for malformed known commands.
  • Fix PROXY TCP4/TCP6 version validation.
  • Add streaming DATA ingestion through IStreamingMessageStore while preserving existing IMessageStore behavior.
  • Add verb-based parser dispatch and parser benchmarks.
  • Wire NetworkBufferSize into connection pipe setup.
  • Reduce AUTH/EHLO allocations and add focused benchmarks.
  • Add DSN envelope parameter parsing and parameter-aware recipient/filter surfaces.
  • Add HELP, VRFY, EXPN command handling and policy extension points.
  • Add CHUNKING/BDAT support, including multi-chunk transactions and streaming store support.
  • Add enhanced SMTP status codes for common replies.
  • Add benchmark coverage for parser, AUTH/EHLO, DATA, BDAT, and throughput paths.
  • Reduce buffered BDAT fallback copying by avoiding a final whole-message ToArray() where possible.
  • Harden parser support for multi-segment command buffers.
  • Add MaxCommandLineLength separate from MaxMessageSize.
  • Document and test EHLO extension advertisement behavior.

Notes For Reviewers

  • The existing IMessageStore.SaveAsync(... ReadOnlySequence<byte> ...) contract is preserved.
  • Streaming storage is opt-in via IStreamingMessageStore.
  • MaxMessageSize remains focused on DATA/BDAT message content; command and AUTH continuation lines now use MaxCommandLineLength.
  • EHLO extension documentation now states behavior and limitations for PIPELINING, 8BITMIME, SMTPUTF8, SIZE, DSN, CHUNKING, STARTTLS, and AUTH.
  • This branch currently includes work that also exists in earlier smaller PRs from the fork; this consolidated PR can replace those if preferred.

Validation

Most recent verification on the final nexer-prod merge state:

  • DOTNET_ROLL_FORWARD=Major dotnet test src/SmtpServer.Tests/SmtpServer.Tests.csproj
    • Passed: 196
    • Skipped: 1
  • Benchmark slices were run for the benchmark/BDAT allocation work, including before/after BDAT allocation checks.

Existing warnings observed during local validation:

  • NU1900 package vulnerability lookup warnings when package sources were unreachable from the environment.
  • Existing package readme warning.
  • Existing xUnit analyzer warnings in pre-existing tests.

Jira

Covers SS-2 through SS-16, including the final consolidated work for SS-12, SS-13, SS-14, SS-15, and SS-16.

vhafdal and others added 30 commits July 15, 2026 10:15
# Conflicts:
#	src/SmtpServer/Protocol/SmtpParser.cs
# Conflicts:
#	src/SmtpServer.Tests/SmtpParserTests.cs
#	src/SmtpServer/Protocol/SmtpParser.cs
# Conflicts:
#	CHANGELOG.md
#	src/SmtpServer.Tests/SmtpParserTests.cs
#	src/SmtpServer.Tests/SmtpServerTests.cs
#	src/SmtpServer/Protocol/RcptCommand.cs
#	src/SmtpServer/Protocol/SmtpParser.cs
@vhafdal

vhafdal commented Jul 15, 2026

Copy link
Copy Markdown
Author

So I would not have to wait a long time for PR approval I decided to just do one clean PR that includes all the changes.
Not sure how long you take in approving, but I will just create a custom nuget package and keep going from there and try to track what we can add later on.

vhafdal and others added 8 commits July 15, 2026 23:09
Add builder-configurable SMTPUTF8, DSN, and CHUNKING advertisement/enforcement with compatibility defaults.

Add session policy callbacks for connection and HELO/EHLO decisions, plus safe command snapshots for AUTH-safe diagnostics.

Build/test: dotnet build src/SmtpServer.sln; env DOTNET_ROLL_FORWARD=Major dotnet test src/SmtpServer.Tests/SmtpServer.Tests.csproj --no-build; dotnet pack src/SmtpServer/SmtpServer.csproj -c Release --no-restore.

Co-Authored-By: Codex <codex@openai.com>

Session: local Codex CLI
The initial 11.2.0 feed package was published from stale Release output and did not expose the SS-17 API. Bump the package version so consumers can restore an immutable corrected artifact.

Build/test: dotnet build src/SmtpServer/SmtpServer.csproj -c Release -m:1; verified Release XML contains SmtpServerOptionsBuilder.Extensions and SmtpCommandEventArgs.SafeCommand.

Co-Authored-By: Codex <codex@openai.com>

Session: local Codex CLI
Mirrors the DevOp IMAP server's bearer-token support so SMTP submission can
authenticate an external-IdP access token the same way. The mechanisms decode
the SASL initial-response blob and surface the bearer token to IUserAuthenticator
as the password and the SASL identity as the user, so a host validates the token
with its existing credential path — no identity-provider call happens in the
session tier.

- AuthenticationMethod gains XOAuth2 and OAuthBearer.
- SmtpParser recognizes the two mechanisms (XOAUTH2's trailing digit tokenizes
  separately, so its matcher consumes the "XOAUTH" text and "2" number tokens).
- AuthCommand decodes the blob via OAuthSaslDecoder (base64 -> UTF8, split on
  U+0001; user=/auth=Bearer for XOAUTH2, the GS2 a= authzid for OAUTHBEARER).
- Advertisement is opt-in: SmtpServerExtensionOptions.OAuth(true) adds
  "XOAUTH2 OAUTHBEARER" to the EHLO AUTH line (off by default so a host never
  advertises a mechanism it cannot honour); the mechanisms are always parsed.

Package bumped 11.2.1 -> 11.3.0. Tests: decoder unit tests (both mechanisms,
host/port fields, invalid base64, missing token/user/authzid) + parser
recognition tests. Full suite green (218 passed, 1 skipped).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012Fu2ZfXS5UNPbcHjfmHMyB
Switch the package version from an explicit <Version> to MinVer driven by a
git tag (prefix smtp-server-v), matching the sibling DevOp.IMAPServer library so
releases are cut by tagging rather than editing the csproj. Add a Jenkinsfile
(mirrored from DevOp.IMAPServer, minus the private-feed restore step this library
does not need) that resolves the version from the tag, builds, tests, packs, and
pushes to nuget.devop.is when PUSH_NUGET is set. Retarget the test project to
net10.0 so it runs on the dotnet build agent.

The first tag under the new scheme is smtp-server-v11.3.0.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012Fu2ZfXS5UNPbcHjfmHMyB
The nuget.devop.is feed requires basic auth even to read its service index, so
`dotnet nuget push` failed with 401 before it could push. Add a committed
nuget.config (nuget.org for the library's own dependencies; devop-marketplace as
the push target) and inject the basic-auth credential into that source in the
build stage of the Jenkinsfile, exactly as the sibling DevOp.IMAPServer pipeline
does. The push stage reuses the same nuget.config, so it can now reach the feed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012Fu2ZfXS5UNPbcHjfmHMyB
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant