-
Notifications
You must be signed in to change notification settings - Fork 5k
AWS default credential chain, SigV4-signed fetch, Bun.aws / Bun.gcp #39210
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from 10 commits
1a5e2e8
1d4bc5b
c270e59
f345a6c
3601115
581a1d8
f5e26f8
b968023
fdf3c15
22476dd
4a89065
0a14a14
c327de3
bf7aa4e
dc236bd
421968f
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,187 @@ | ||
| --- | ||
| title: AWS & Google Cloud auth | ||
| description: Call AWS and Google Cloud APIs from Bun with the machine's own credentials — no SDK required | ||
| --- | ||
|
|
||
| Bun can authenticate requests to AWS and Google Cloud the same way their CLIs and SDKs do: from environment variables, config files, SSO sessions, or the instance / container / pod the code is running on. Bun fetches credentials without blocking the JavaScript thread on the network, caches them, and refreshes ones that are in use in the background shortly before they expire. | ||
|
|
||
| ```ts | ||
| // SigV4-signed request with whatever credentials this machine has | ||
| const queues = await Bun.aws.fetch("https://sqs.us-east-1.amazonaws.com/?Action=ListQueues"); | ||
|
|
||
| // Bearer token from Application Default Credentials | ||
| const buckets = await Bun.gcp.fetch("https://storage.googleapis.com/storage/v1/b?project=my-project"); | ||
| ``` | ||
|
|
||
| `Bun.aws` and `Bun.gcp` are ready-made clients using ambient credentials. For several accounts, regions or key files at once, construct more: | ||
|
|
||
| ```ts | ||
| const prod = new Bun.AWSClient({ profile: "prod", region: "eu-west-1" }); | ||
| const deployer = new Bun.GCPClient({ keyFile: "/secrets/deployer.json" }); | ||
| ``` | ||
|
|
||
| `Bun.s3`, `new S3Client()` and `fetch("s3://…")` use the AWS chain automatically when no `accessKeyId`/`secretAccessKey` is configured — see [S3 › Credentials](/runtime/s3#credentials). | ||
|
|
||
| ## AWS | ||
|
|
||
| ### Where credentials come from | ||
|
|
||
| When a request needs AWS credentials and none were passed explicitly, Bun tries, in order: | ||
|
|
||
| 1. `AWS_ACCESS_KEY_ID` + `AWS_SECRET_ACCESS_KEY` (+ `AWS_SESSION_TOKEN`) | ||
| 2. The profile named by the `profile` option, `AWS_PROFILE`, or `default`, from `~/.aws/credentials` and `~/.aws/config` (or `AWS_SHARED_CREDENTIALS_FILE` / `AWS_CONFIG_FILE`). A profile may hold static keys, `role_arn` + `source_profile` / `credential_source` (STS `AssumeRole`), `web_identity_token_file`, `credential_process`, or IAM Identity Center settings (`sso_session` / `sso_*`, using the token that `aws sso login` cached). | ||
| 3. `AWS_WEB_IDENTITY_TOKEN_FILE` + `AWS_ROLE_ARN` — EKS service-account roles (IRSA) | ||
| 4. The container credentials endpoint — `AWS_CONTAINER_CREDENTIALS_RELATIVE_URI` or `AWS_CONTAINER_CREDENTIALS_FULL_URI` (+ `AWS_CONTAINER_AUTHORIZATION_TOKEN[_FILE]`), set by ECS, EKS Pod Identity, App Runner, … | ||
| 5. EC2 instance metadata (IMDSv2). Honours `AWS_EC2_METADATA_DISABLED`, `AWS_EC2_METADATA_SERVICE_ENDPOINT[_MODE]`, `AWS_EC2_METADATA_V1_DISABLED`, `AWS_METADATA_SERVICE_TIMEOUT` and `AWS_METADATA_SERVICE_NUM_ATTEMPTS`. | ||
|
|
||
| A source that isn't configured is skipped. A source that _is_ configured but fails (an expired SSO session, an STS `AccessDenied`, a `credential_process` that exits non-zero) is an error — Bun does not silently fall through to the next one. | ||
|
|
||
| As in the AWS SDKs, selecting a profile (`AWS_PROFILE` or the `profile` option) takes precedence over exported `AWS_ACCESS_KEY_ID` / `AWS_SECRET_ACCESS_KEY`. Environment variables are read from `process.env` when credentials are resolved, so setting `process.env.AWS_PROFILE` at startup works. | ||
|
|
||
| ### `new Bun.AWSClient(options?)` / `Bun.aws` | ||
|
|
||
| An `AWSClient` bundles signing defaults: `region`, `service`, `profile` or `accessKeyId`/`secretAccessKey`/`sessionToken`, and `endpoint` (a base URL for path-only requests, e.g. LocalStack). Every method accepts the same options to override them per call. `Bun.aws` is the instance with no overrides. | ||
|
|
||
| ```ts | ||
| const prod = new Bun.AWSClient({ profile: "prod", region: "eu-west-1" }); | ||
| prod.region; // "eu-west-1" | ||
| prod.profile; // "prod" | ||
| ``` | ||
|
|
||
| ### `client.credentials()` | ||
|
|
||
| Resolve (or return the cached) credentials. Useful for handing them to another SDK, or just to see what Bun found. | ||
|
|
||
| ```ts | ||
| const creds = await Bun.aws.credentials(); | ||
| // { | ||
| // accessKeyId: "ASIA…", | ||
| // secretAccessKey: "…", | ||
| // sessionToken: "…", // temporary credentials only | ||
| // expiration: 2026-01-01T…, // temporary credentials only | ||
| // region: "us-east-1", // if AWS_REGION / the profile set one | ||
| // accountId: "123456789012", // if the source reported it | ||
| // source: "sso", // "env" | "profile" | "assume-role" | "web-identity" | "process" | "sso" | "container" | "imds" | ||
| // } | ||
|
|
||
| await Bun.aws.credentials({ refresh: true }); // ignore the cache | ||
| await new Bun.AWSClient({ profile: "prod" }).credentials(); // a specific profile | ||
| ``` | ||
|
|
||
| When nothing is found, the promise rejects with `code: "ERR_AWS_MISSING_CREDENTIALS"` and a message listing each source and why it was skipped. A configured source that failed rejects with `code: "ERR_AWS_CREDENTIALS"`. | ||
|
|
||
| ### `client.fetch(input, init?)` | ||
|
|
||
| `fetch()`, with the request signed using [Signature Version 4](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_sigv.html). `init` accepts everything `fetch` does plus the signing options below; with none, the service and region are inferred from `*.amazonaws.com` / `*.on.aws` hostnames and ambient credentials are used. | ||
|
|
||
| ```ts | ||
| const res = await Bun.aws.fetch("https://dynamodb.us-east-1.amazonaws.com/", { | ||
| method: "POST", | ||
| headers: { | ||
| "content-type": "application/x-amz-json-1.0", | ||
| "x-amz-target": "DynamoDB_20120810.GetItem", | ||
| }, | ||
| body: JSON.stringify({ TableName: "users", Key: { id: { S: "42" } } }), | ||
| }); | ||
|
|
||
| // A path-only URL goes to the service's standard endpoint in the resolved | ||
| // region: https://sqs.<region>.amazonaws.com/?Action=ListQueues | ||
| await Bun.aws.fetch("/?Action=ListQueues", { service: "sqs" }); | ||
|
|
||
| // API Gateway / Lambda function URL behind IAM auth on a custom domain | ||
| await Bun.aws.fetch("https://api.example.com/orders", { service: "execute-api", region: "eu-west-1", profile: "prod" }); | ||
| ``` | ||
|
|
||
| | Option | Default | | ||
| | ------------------------------------------------ | ---------------------------------------------------------------------------------------- | | ||
| | `service`, `region` | inferred from the hostname; region falls back to `AWS_REGION`, then the profile's region | | ||
| | `accessKeyId`, `secretAccessKey`, `sessionToken` | ambient credentials | | ||
| | `profile` | `AWS_PROFILE` / `default` | | ||
| | `signQuery`, `expiresIn` | `false`, `900` — put the signature in the query string | | ||
| | `unsignedPayload` | `false` — send `UNSIGNED-PAYLOAD` (S3-style services only) | | ||
|
|
||
| Bun adds `Authorization`, `x-amz-date`, `x-amz-security-token` (temporary credentials) and, for S3, `x-amz-content-sha256`; setting those yourself is an error. `ReadableStream` bodies can only be signed for S3-style services (as `UNSIGNED-PAYLOAD`), since other services need the body's SHA-256 up front. | ||
|
|
||
| A signature is only valid for the exact host and path it was computed for, so `redirect` defaults to `"manual"` — a `307` from S3 pointing at another region comes back as a `307` rather than being followed unsigned. Pass `redirect: "follow"` explicitly if you really want that. | ||
|
|
||
| For `s3://` URLs keep using `fetch("s3://…", { s3: {...} })` or `Bun.s3`. | ||
|
|
||
| ### `client.presign(url, options?)` | ||
|
|
||
| Returns a query-string-signed URL — typically an S3 object URL to hand to a browser. Takes the same signing options as `fetch`, plus `method` (default `"GET"`) and `expiresIn` (seconds, default `900`, max `604800`). | ||
|
|
||
| ```ts | ||
| const download = await Bun.aws.presign("https://my-bucket.s3.eu-west-1.amazonaws.com/report.pdf", { expiresIn: 3600 }); | ||
| const upload = await Bun.aws.presign("https://my-bucket.s3.eu-west-1.amazonaws.com/upload.bin", { method: "PUT" }); | ||
| const queue = await Bun.aws.presign("/123456789012/jobs?Action=ReceiveMessage", { service: "sqs" }); | ||
| ``` | ||
|
|
||
| It returns a promise because the credentials may still have to be fetched; the signing itself is local. For `s3://`-style paths and ACLs, [`S3File.presign()`](/runtime/s3#presigning-urls) is usually more convenient. | ||
|
|
||
| ### `client.eventStream(response)` | ||
|
|
||
| AWS streaming APIs (Bedrock `InvokeModelWithResponseStream` / `ConverseStream`, Lambda response streaming, S3 Select, …) answer with `application/vnd.amazon.eventstream` frames rather than SSE. `eventStream()` turns such a response (or any `ReadableStream` / async iterable of bytes) into an async iterator of messages, verifying checksums as it goes. | ||
|
|
||
| ```ts | ||
| const res = await Bun.aws.fetch(`https://bedrock-runtime.us-east-1.amazonaws.com/model/${modelId}/converse-stream`, { | ||
| method: "POST", | ||
| headers: { "content-type": "application/json" }, | ||
| body: JSON.stringify({ messages: [{ role: "user", content: [{ text: "Write a haiku about Bun" }] }] }), | ||
| }); | ||
| for await (const message of Bun.aws.eventStream(res)) { | ||
| // message.event: "messageStart" | "contentBlockDelta" | "messageStop" | … | ||
| if (message.event === "contentBlockDelta") process.stdout.write(message.json().delta.text); | ||
| } | ||
| ``` | ||
|
|
||
| Each message has `headers`, `payload` (`Uint8Array`), `type` / `event` / `contentType` shortcuts for the `:message-type` / `:event-type` / `:content-type` headers, and `text()` / `json()`. An `exception` or `error` frame ends the iteration by throwing an `Error` whose `name` is the exception type exactly as the service sends it (Bedrock uses `throttlingException`, `validationException`, …); a non-2xx response is thrown before any message, with its `status`, `headers` and `body`. | ||
|
|
||
| ## Google Cloud | ||
|
|
||
| ### Where tokens come from | ||
|
|
||
| [Application Default Credentials](https://cloud.google.com/docs/authentication/application-default-credentials), in order: | ||
|
|
||
| 1. `GOOGLE_APPLICATION_CREDENTIALS` — a service-account key file, or an `authorized_user` file | ||
| 2. The file `gcloud auth application-default login` writes: `~/.config/gcloud/application_default_credentials.json` (`%APPDATA%\gcloud\…` on Windows, or under `CLOUDSDK_CONFIG`) | ||
| 3. The metadata server on Compute Engine, GKE, Cloud Run, Cloud Functions, App Engine, Cloud Build … (honours `GCE_METADATA_HOST` and `NO_GCE_CHECK`) | ||
|
|
||
| Service-account keys are exchanged for tokens with a self-signed RS256 JWT; user credentials use their refresh token. `external_account` (workload identity federation) files are not supported yet. | ||
|
|
||
| ### `new Bun.GCPClient(options?)` / `Bun.gcp` | ||
|
|
||
| `Bun.gcp` uses Application Default Credentials. A `GCPClient` can instead be given a `keyFile` path or the key's `credentials` (object or JSON string), plus default `scopes` or an `audience` (making `fetch()` send ID tokens by default). | ||
|
|
||
| ```ts | ||
| const deployer = new Bun.GCPClient({ keyFile: "/secrets/deployer.json", scopes: ["cloud-platform"] }); | ||
| const invoker = new Bun.GCPClient({ | ||
| credentials: JSON.parse(process.env.SA_KEY!), | ||
| audience: "https://my-service-abc123.a.run.app", | ||
| }); | ||
| ``` | ||
|
|
||
| ### `client.accessToken()` / `client.idToken()` | ||
|
|
||
| ```ts | ||
| const { token, expiration, source } = await Bun.gcp.accessToken(); | ||
| // source: "service-account" | "authorized-user" | "metadata" | ||
| // also: email, projectId, quotaProjectId when known | ||
|
|
||
| await Bun.gcp.accessToken({ scopes: ["bigquery", "https://www.googleapis.com/auth/pubsub"] }); | ||
| // bare scope names expand to https://www.googleapis.com/auth/<name>; the default is cloud-platform | ||
|
|
||
| // OIDC identity token for calling Cloud Run / Cloud Functions / IAP | ||
| const { token: idToken } = await Bun.gcp.idToken("https://my-service-abc123.a.run.app"); | ||
| ``` | ||
|
|
||
| Tokens are cached per scope set / audience; pass `refresh: true` to force a new one. Failures reject with `code: "ERR_GCP_MISSING_CREDENTIALS"` (nothing configured) or `"ERR_GCP_CREDENTIALS"` (a configured source failed). | ||
|
|
||
| ### `client.fetch(input, init?)` (GCP) | ||
|
|
||
| ```ts | ||
| await Bun.gcp.fetch("https://pubsub.googleapis.com/v1/projects/my-project/topics"); | ||
| await Bun.gcp.fetch("https://storage.googleapis.com/storage/v1/b/my-bucket/o", { scopes: "devstorage.read_only" }); | ||
| await Bun.gcp.fetch("https://my-service-abc123.a.run.app/", { audience: "https://my-service-abc123.a.run.app" }); | ||
| ``` | ||
|
|
||
| `init` accepts everything `fetch` does plus `scopes` (access token, the default) or `audience` (ID token). Bun sets `Authorization: Bearer <token>`, and `x-goog-user-project` when the credentials carry a `quota_project_id` (or `GOOGLE_CLOUD_QUOTA_PROJECT` is set) unless you set it yourself. | ||
| Original file line number | Diff line number | Diff line change | ||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
|
@@ -291,6 +291,17 @@ Only PUT and POST methods support request bodies when using S3. For uploads, Bun | |||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||
| See the [S3](/runtime/s3) documentation. | ||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||
| ### AWS- and Google-authenticated requests | ||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||
| `Bun.aws.fetch` and `Bun.gcp.fetch` are `fetch` with the request SigV4-signed for AWS, or carrying a Google Cloud bearer token, using the machine's ambient credentials: | ||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||
| ```ts | ||||||||||||||||||||||||||||||
| await Bun.aws.fetch("https://sqs.us-east-1.amazonaws.com/?Action=ListQueues"); | ||||||||||||||||||||||||||||||
| await Bun.gcp.fetch("https://storage.googleapis.com/storage/v1/b?project=my-project"); | ||||||||||||||||||||||||||||||
| ``` | ||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||
| See [AWS & Google Cloud auth](/runtime/networking/cloud-auth) for the options and where credentials come from. | ||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||
|
Comment on lines
+294
to
+304
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win Fix the heading level so the following protocol sections stay under "Protocol support". Line 294 uses The new section also documents a client API, not a URL protocol. Move it after the 📝 Proposed fix (demote in place)-### AWS- and Google-authenticated requests
+#### AWS- and Google-authenticated requests📝 Committable suggestion
Suggested change
🤖 Prompt for AI Agents |
||||||||||||||||||||||||||||||
| #### File URLs - `file://` | ||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||
| You can fetch local files using the `file:` protocol: | ||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||
Uh oh!
There was an error while loading. Please reload this page.