Skip to content

feat: Add contrib/aws/secretsmanager envconfig extension - #2772

Open
johanngan wants to merge 6 commits into
mainfrom
johanngan/aws-secrets-manager
Open

johanngan wants to merge 6 commits into
mainfrom
johanngan/aws-secrets-manager

Conversation

@johanngan

Copy link
Copy Markdown
Contributor

What was changed

This module defines a new envconfig extension to support Temporal client credentials stored in AWS Secrets Manager. The user specifies a secret ARN or friendly name via envconfig (TOML or environment variables), and the client will automatically fetch the secret material from Secrets Manager, with periodic refreshes to support secret rotation.

The AWS Parameters and Secrets Lambda Extension is used if detected in the environment. Otherwise, secrets are retrieved directly from Secrets Manager.

See the README or package documentation for usage details.

Why?

This is intended to be used with the Lambda worker contrib package (and in the future other serverless providers), to make it more convenient to store worker secrets in a secrets manager when using serverless.

Checklist

  1. How was this tested:
    Added new unit tests. Also did manual E2E verification with the Lambda worker integration with AWS Secrets Manager with a test setup, which exercised this code code.

  2. Any docs updates needed?
    The associated in-repo docs are updated. This specific module is primarily an implementation detail for the Lambda worker integration with Secrets Manager. We'll probably want to update our public docs for Lambda workers, but that can be done later.

@johanngan
johanngan added this pull request to stack #2773 October 12, 2026 19:53
@johanngan
johanngan requested a review from a team as a code owner October 12, 2026 19:53
@johanngan

Copy link
Copy Markdown
Contributor Author

Note: This can't be merged until the secretsmanager go.mod is updated with the appropriate envconfig version to be cut after #2771 is merged.

@@ -0,0 +1,12 @@
<!--

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems like the CI is complaining about this, but I think it can be ignored since we're adding a totally new changelog?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, we may have to just apply the skip-changelog label

@johanngan
johanngan requested a review from eamsden October 12, 2026 20:20
@johanngan
johanngan force-pushed the johanngan/aws-secrets-manager branch 2 times, most recently from 279b0d4 to f6ffafb Compare October 15, 2026 15:34
@johanngan
johanngan force-pushed the johanngan/aws-secrets-manager branch from f6ffafb to af62486 Compare October 30, 2026 15:57
Base automatically changed from johanngan/envconfig-extensions to main October 30, 2026 20:55
@johanngan
johanngan force-pushed the johanngan/aws-secrets-manager branch from af62486 to 21ba1c5 Compare October 30, 2026 20:55
@johanngan
johanngan requested a review from Sushisource November 3, 2026 17:19
This module defines a new envconfig extension to support Temporal client
credentials stored in AWS Secrets Manager. The user specifies a secret
ARN or friendly name via envconfig (TOML or environment variables), and
the client will automatically fetch the secret material from Secrets
Manager, with periodic refreshes to support secret rotation.

The AWS Parameters and Secrets Lambda Extension is used if detected in
the environment. Otherwise, secrets are retrieved directly from Secrets
Manager.

See the README or package documentation for usage details.
@johanngan
johanngan force-pushed the johanngan/aws-secrets-manager branch from 21ba1c5 to b20e2fe Compare November 3, 2026 18:25
@johanngan

Copy link
Copy Markdown
Contributor Author

(Latest push just rebases + go mod tidy)

Comment thread contrib/aws/secretsmanager/cache.go Resolved
Comment thread contrib/aws/secretsmanager/cache.go Resolved
Comment thread contrib/aws/secretsmanager/fetch.go Resolved
Comment thread contrib/aws/secretsmanager/extension.go Resolved
This prevents us from seeing some random local service listening on port
2773 and confusing it for the Lambda extension.
Makes various assorted improvements to caching behavior:
- Always couple cert/key cache entries together so they're always
  refreshed/invalidated together and can't be retrieved in a torn state.
  In service of this change, the cache is specialized more from a
  generic map, and instead exposes per-secret-type get APIs. This
  unifies the cert bundle/separate cert-key secret paths into a single
  client cert retrieval path.
- Improve concurrency behavior during a cache refresh; don't lock around
  the fetch and instead allow stale values to be served while another
  thread has a cache refresh in-flight. In the case that no usable value
  is in the cache, still respect the caller's context by waiting on a
  channel instead of the entry's mutex.
- Use a separate context and timeout for fetches, so that a caller's
  deadline or cancellation doesn't affect the cache's failure state.
  This is accomplished by using a background thread to perform fetches.
Comment thread contrib/aws/secretsmanager/extension.go Resolved
Comment thread contrib/aws/secretsmanager/cache.go Resolved
Comment thread contrib/aws/secretsmanager/extension.go Resolved
This allows each cache entry to directly store its value type instead of
a raw byte slice, which allows us to avoid cert parsing on every call to
getClientCert().
The SDK does this automatically with opts.Credentials set.
@johanngan
johanngan force-pushed the johanngan/aws-secrets-manager branch from 10f08e4 to 81231e8 Compare November 6, 2026 01:22
@johanngan
johanngan requested a review from Sushisource November 6, 2026 01:37
Comment thread contrib/aws/secretsmanager/doc.go Resolved
Comment thread contrib/aws/secretsmanager/README.md Resolved
For plugin fetch operations, try to fetch the telemetry providers in the
plugin's NewClient() rather than immediately during its
ConfigureClient(), so that logging and metrics can happen through the
final configured providers, after other plugins and the SDK defaults
have had a chance to take effect. Also move the eager cache fetches to
NewClient() where possible, since they log and emit metrics.

The one exception is the server CA cert, which needs to be configured on
the options.
@johanngan
johanngan force-pushed the johanngan/aws-secrets-manager branch from 81231e8 to 3cfb9ea Compare November 7, 2026 00:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants