-
Notifications
You must be signed in to change notification settings - Fork 14
feat: OIDC device-flow authentication #133
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
glasstiger
wants to merge
104
commits into
main
Choose a base branch
from
ia_oidc_device_flow
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from 1 commit
Commits
Show all changes
104 commits
Select commit
Hold shift + click to select a range
4559587
feat: OIDC device flow
glasstiger c7c269f
fix: pandas 3 string dtype in test_parquet_roundtrip
glasstiger e394fbd
ci: keep 32-bit wheel tests on the pandas 2 / numpy 1 path
glasstiger ae92178
test: silence mock server tracebacks on Windows client disconnect
glasstiger 88308d5
ci: skip readonly AZP_ENHANCED agent var in Windows wheel build
glasstiger d5df186
fix: harden OIDC device-flow auth; drop on-disk FileCache backend
glasstiger 5ef892b
ci: build questdb master's -SNAPSHOT java client via local-client pro…
glasstiger ae8baa7
ci: build/run questdb master on JDK 25
glasstiger a4b41c3
ci: invoke Maven directly to build questdb master on JDK 25
glasstiger 61abd4f
ci: pass JDK 25 module access flags to questdb master server
glasstiger 23fb823
do not follow redirects
glasstiger c13cf69
fix: require IdP pin for plaintext /settings
glasstiger bb9147c
fix: clamp device-flow poll timing fields
glasstiger 0edf9dc
fix: map malformed inputs to typed OidcError
glasstiger 35c3fbb
fix: sanitize device-flow terminal output
glasstiger f6321c3
docs: correct questdb.auth changelog and API reference
glasstiger e043561
fix: make TokenSet immutable and keep tokens out of repr
glasstiger 4e62938
style: sort questdb.auth __all__ to satisfy Ruff RUF022
glasstiger 4a67cc5
docs: make review-pr level-0/1 Step 2.5 rules consistent
glasstiger 124d4c2
docs: exclude Agent 10 from review-pr Step 2.5 input contract
glasstiger a062a0a
fix: ignore user-writable /settings preferences
glasstiger fe34ffa
fix: reject conf metachars in QuestDB host
glasstiger ae4c5ba
fix: harden questdb.auth untrusted-input handling
glasstiger afbd808
fix: bound IdP timeout; broaden auth edge tests
glasstiger 63edfce
fix: address 3 minor questdb.auth review nits
glasstiger c869e31
fix: address 4 follow-up auth review findings
glasstiger bcfebd9
fix: address 4 moderate questdb.auth review findings
glasstiger 8f61ff2
fix: make deeply-nested-JSON auth test robust on Python 3.14
glasstiger a9c58c8
fix: make questdb.auth clear() reliable across shared-cache instances
glasstiger 7ed6f05
fix: tolerate non-string acl.oidc.* from QuestDB /settings and IdP di…
glasstiger 63a3f7a
fix: keep device-flow poll alive through transient IdP errors
glasstiger 954f8a8
fix: map malformed IdP discovery / /exec payloads to OidcError
glasstiger 6e2970b
fix: sanitize untrusted device fields on the Jupyter prompt path
glasstiger c31eb0f
fix: bind discovery_url pin to the IdP origin; close auth test gaps
glasstiger 20bf2cd
fix: classify IdP token-endpoint errors transient vs terminal
glasstiger dcd44bc
fix: keep the device-flow prompt visible on a non-UTF-8 terminal
glasstiger 86cf83f
fix: close issuer-path pin bypass via encoded/backslash/;params trave…
glasstiger 0001e4d
fix: tighten device-auth, host/port handling, and refresh audience
glasstiger 99c05ac
docs: condense the questdb.auth comments and docstrings
glasstiger 73e3c5f
fix: default groups_in_token to False to match the QuestDB server
glasstiger b23e5db
refactor(auth): replace the QuestDB session with PG-wire adapter func…
glasstiger c189141
refactor(auth): drop the unused `flow` parameter from from_questdb
glasstiger 806be9c
refactor(auth): only trust absolute /settings endpoint URLs (match Java)
glasstiger 19f46d0
feat(auth): open the verification URL in a browser by default
glasstiger f567e26
fix(auth): floor the device-flow poll interval at 5s (RFC 8628 default)
glasstiger 9f5035f
fix(auth): bound the HTTP response read (size cap + wall-clock deadline)
glasstiger 2d75e01
fix(test): drop stale REST-adapter test imports from test.py
glasstiger b32207f
fix(auth): clamp token lifetime to match the Java client
glasstiger 48aae3b
refactor(auth): drop the pluggable cache; memory cache is always on
glasstiger 0a70fa5
fix(auth): close path-pin bypass via inner-segment ;param
glasstiger 2fed74a
fix(auth): coerce non-string IdP token fields to None
glasstiger deb111d
fix(auth): map truncated error body to OidcNetworkError
glasstiger 5adb3f4
test(auth): cover proactive refresh in the skew window
glasstiger 0c0f499
fix(auth): validate constructor arg types; redact sub from repr
glasstiger c7cd7f5
fix(auth): validate default_interval and timeout args
glasstiger 02a29f2
fix(auth): propagate HTTP status on non-dict JSON body
glasstiger 117f06a
fix(auth): honor read deadline on slow-dribble responses
glasstiger 9e62239
fix(auth): strip terminal control chars by Unicode category
glasstiger bf7ea91
fix(auth): harden parsing of untrusted IdP response fields
glasstiger 0b62d09
fix(auth): normalize empty audience; drop dead field; review cleanup
glasstiger c1223f2
fix(auth): evict unusable token; sanitize error output
glasstiger b569014
fix(auth): enforce the discovery_url pin on /settings endpoints
glasstiger baa4c4d
fix(auth): fail fast on a 3xx poll; normalize issuer in cache key
glasstiger 075a01c
fix(auth): scope the issuer-origin pin to /settings; drop discovery_url
glasstiger 5f92dcb
fix(auth): keep the Jupyter QR code visible across re-renders
glasstiger af721f6
fix(auth): apply the issuer-path pin after discovery
glasstiger 899304d
fix(auth): harden the prompt renderer and error sanitizer
glasstiger a91c1f2
fix(auth): bound the token-cache maps; fix the /settings URL
glasstiger 6c41d2c
fix(auth): tighten origin/path normalization edge cases
glasstiger 4482b94
fix(auth): harden error fields and verification-link safety
glasstiger a053233
fix(auth): trim link whitespace and reject IPv6 zone-id host
glasstiger 5d1421b
fix(auth): show real link host and unify the open/QR target
glasstiger c0d8a6e
fix(auth): validate pg_port and attach status to get_json errors
glasstiger 6806bdf
fix(auth): guard verification_uri, real-lifetime message, Retry-After
glasstiger ff84625
fix(auth): reject confusable endpoint authority and inf timeout
glasstiger 6581ad5
fix(auth): apply minor review fixes (m1-m7)
glasstiger 623b983
fix(auth): detect non-interactive notebook executors; review nits
glasstiger e78de24
fix(auth): bound chunked reads and keep errors typed
glasstiger 81f72a5
fix(auth): refuse pool-thread sign-in; floor slow_down backoff
glasstiger 1e5a5f8
fix(auth): adopt fresh cached token; honor non-JSON Retry-After
glasstiger e7c208a
fix(auth): escape confusable host on display fail-open
glasstiger 28dd619
fix(auth): apply minor review fixes (5xx backoff, cache key, re-entry)
glasstiger 1ce9599
fix(auth): reject tab/newline/CR in endpoint host
glasstiger 99c2d68
fix(auth): minor review fixes (link, host, return type)
glasstiger b37fe54
fix(auth): public doc xrefs; non-object JWT test
glasstiger 1431df3
fix(auth): isolate renderer errors; reject % host
glasstiger 57e522d
opt-in token persistence
glasstiger 0b63719
fix(auth): bound the response head read by the deadline
glasstiger 7790c4a
fix(auth): unify token identity keys; screen network tokens
glasstiger 2a55c50
fix(auth): harden numeric, render, and issuer edge cases
glasstiger b0d2ce8
fix(auth): align in-memory and on-disk token keys
glasstiger e4744e4
fix(auth): isolate persisted tokens by issuer
glasstiger 69b9235
fix(auth): harden retry-after, blank tokens, IPv6 keys, and store locks
glasstiger 9912a51
fix(auth): sync persisted refresh token when adopting a cached token
glasstiger 80cf3b7
docs(auth): export Renderer so its docstring xref resolves
glasstiger d4710c7
fix(auth): harden token persistence and discovery
glasstiger ccc400c
fix(auth): reject malformed verification-URL port so the shown link c…
glasstiger 710843a
fix(auth): reject blank-after-strip device prompts; close test-covera…
glasstiger 75f27bb
fix(auth): keep bad-arg errors typed and reuse a peer token before re…
glasstiger 5aac5e8
fix(auth): sweep orphaned token temps, harden lock, tighten docs and …
glasstiger c393dd9
fix(auth): reject multi-host PG hosts and truncated HTTP bodies
glasstiger 5a70301
fix(auth): close credential-routing bypass and two token-store gaps
glasstiger 797ceae
fix(auth): keep non-string issuer typed; canonicalize endpoint confir…
glasstiger e50f490
fix(auth): honor slow_down on a 429, coerce groups flag, strip hidden…
glasstiger File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,233 @@ | ||
| .. _oidc_auth: | ||
|
|
||
| =================== | ||
| OIDC Authentication | ||
| =================== | ||
|
|
||
| QuestDB Enterprise can be secured with `OpenID Connect (OIDC) | ||
| <https://questdb.com/docs/operations/rbac/>`_. The :mod:`questdb.auth` module | ||
| lets you sign in interactively from Python — including from a **remote** kernel | ||
| (JupyterHub, SageMaker, Colab, VS Code-remote, containers) where there is no | ||
| local browser. | ||
|
|
||
| It runs the `OAuth 2.0 Device Authorization Grant (RFC 8628) | ||
| <https://datatracker.ietf.org/doc/html/rfc8628>`_ entirely client-side: you | ||
| authorize in **any** browser (your laptop or your phone), while the kernel only | ||
| makes outbound calls to your identity provider (IdP). The resulting token is | ||
| then presented to QuestDB over the auth paths it already supports — HTTP | ||
| ``Authorization: Bearer`` or PG-wire ``_sso`` — so **no server change is | ||
| required**. | ||
|
|
||
| .. note:: | ||
|
|
||
| This feature targets **QuestDB Enterprise with OIDC enabled**. The IdP | ||
| client referenced by ``acl.oidc.client.id`` must have the device grant | ||
| (``urn:ietf:params:oauth:grant-type:device_code``) enabled and be a public | ||
| client. See :ref:`oidc_idp_requirements`. | ||
|
|
||
| Two ways to use it | ||
| ================== | ||
|
|
||
| You can let the helper drive everything, or you can just take the token and use | ||
| it with your own tooling. | ||
|
|
||
| Just the token (PG-wire / HTTP / anything) | ||
| ------------------------------------------ | ||
|
|
||
| If you connect to QuestDB yourself — over PG-wire, raw HTTP, or any other | ||
| client — you only need a valid token. This path has **no extra dependencies**. | ||
|
|
||
| .. code-block:: python | ||
|
|
||
| from questdb.auth import OidcDeviceAuth | ||
|
|
||
| # Discover the OIDC configuration from the QuestDB server: | ||
| auth = OidcDeviceAuth.from_questdb("https://questdb.example.com:9000") | ||
|
|
||
| token = auth.token() # runs the device flow on first use, else cached | ||
| headers = auth.headers() # {"Authorization": "Bearer <token>"} | ||
|
|
||
| # Use the token however you like, e.g. PG-wire via psycopg: | ||
| import psycopg | ||
| conn = psycopg.connect( | ||
| host="questdb.example.com", port=8812, dbname="qdb", | ||
| user="_sso", password=token) | ||
|
|
||
| The integrated session | ||
| ---------------------- | ||
|
|
||
| The high-level :func:`questdb.auth.connect` returns a :class:`~questdb.auth.QuestDB` | ||
| session that signs you in and adapts the token into the common Python access | ||
| paths. | ||
|
|
||
| .. code-block:: python | ||
|
|
||
| from questdb.auth import connect | ||
|
|
||
| qdb = connect("https://questdb.example.com:9000") # interactive sign-in | ||
| df = qdb.sql("SELECT * FROM trades WHERE ts > dateadd('h', -1, now())") | ||
|
|
||
| # Bring-your-own client, same auto-refreshed token: | ||
| engine = qdb.sqlalchemy_engine() # PG-wire, token as _sso | ||
| with qdb.psycopg() as conn: # raw psycopg | ||
| ... | ||
| with qdb.sender() as sender: # ingestion (ILP/HTTP) | ||
| sender.row("trades", columns={"price": 101.5}, | ||
| at=TimestampNanos.now()) | ||
|
|
||
| On first use you will see a sign-in prompt (rendered as a clickable link in | ||
| Jupyter, plain text on a terminal):: | ||
|
|
||
| 🔐 Sign in to QuestDB | ||
| Open https://idp.example.com/device and enter code: WDJB-MJHT | ||
| (or open directly: https://idp.example.com/device?user_code=WDJB-MJHT) | ||
| ⏳ waiting for authorization… (4:51 left) | ||
| ✅ Signed in as alice@example.com — token cached, expires in 60 min | ||
|
|
||
| Re-running any cell is silent — the token is cached and refreshed silently on | ||
| the next use once it nears expiry. | ||
|
|
||
| How it works | ||
| ============ | ||
|
|
||
| Configuration discovery | ||
| ------------------------ | ||
|
|
||
| :meth:`OidcDeviceAuth.from_questdb <questdb.auth.OidcDeviceAuth.from_questdb>` | ||
| (and :func:`~questdb.auth.connect`) resolve the OIDC configuration in this | ||
| order: | ||
|
|
||
| 1. ``GET {url}/settings`` (public, no auth) for the QuestDB-authoritative | ||
| values: ``acl.oidc.client.id``, ``acl.oidc.scope``, ``acl.oidc.token.endpoint``, | ||
| ``acl.oidc.groups.encoded.in.token`` and (on newer servers) | ||
| ``acl.oidc.device.authorization.endpoint``. | ||
| 2. If the device-authorization endpoint is not advertised, the helper falls | ||
| back to the IdP discovery document | ||
| (``{issuer}/.well-known/openid-configuration``). The issuer is taken from an | ||
| explicit ``issuer=`` / ``discovery_url=`` argument, or derived from the token | ||
| endpoint's origin. | ||
|
|
||
| Anything you pass explicitly overrides discovery. You can also skip discovery | ||
| entirely: | ||
|
|
||
| .. code-block:: python | ||
|
|
||
| auth = OidcDeviceAuth( | ||
| client_id="questdb", | ||
| device_authorization_endpoint="https://idp/.../device", | ||
| token_endpoint="https://idp/.../token", | ||
| scope="openid groups", | ||
| groups_in_token=True, # send id_token (True) vs access_token (False) | ||
| audience="questdb", # optional; some IdPs need it to set `aud` | ||
| cache="memory") | ||
|
|
||
| Which token is sent | ||
| ------------------- | ||
|
|
||
| The helper mirrors QuestDB's own selection logic | ||
| (``groupsEncodedInToken ? idToken : accessToken``): | ||
|
|
||
| ============================================ ================= | ||
| ``acl.oidc.groups.encoded.in.token`` Helper sends | ||
| ============================================ ================= | ||
| ``true`` ``id_token`` | ||
| ``false`` ``access_token`` | ||
| ============================================ ================= | ||
|
|
||
| When sending the ``id_token`` the ``openid`` scope is requested automatically. | ||
|
|
||
| Token lifecycle (cache + refresh) | ||
| --------------------------------- | ||
|
|
||
| ``token()`` returns the cached token while it is valid (with a small clock-skew | ||
| margin). When it nears expiry the helper silently refreshes it using the | ||
| ``refresh_token`` if one was issued. If the refresh token is missing or rejected | ||
| (expired/revoked), it re-runs the interactive sign-in; a transient network error | ||
| is raised instead, so you can retry without being needlessly re-prompted. A lock | ||
| serializes refresh so parallel cells/threads don't double-prompt. | ||
|
|
||
| Cache backends (``cache=`` argument): | ||
|
|
||
| * ``"memory"`` *(default)* — process-global, nothing written to disk. | ||
| Re-running cells is silent; a kernel restart re-prompts once. | ||
| * ``"file"`` — ``~/.questdb/oidc-cache.json`` (mode ``600``). Survives kernel | ||
| restarts and is shared across kernels on the same host. **Security | ||
| trade-off:** the refresh token is stored at rest. | ||
| * ``None`` — never persist; prompt every time. | ||
|
|
||
| Non-interactive contexts | ||
| ------------------------- | ||
|
|
||
| Scheduled / non-interactive notebooks (papermill, cron, CI) have no human to | ||
| authorize the device. The helper detects this and raises | ||
| :class:`~questdb.auth.OidcInteractionRequired` instead of hanging. Use a QuestDB | ||
| **service-account REST token** or the **client-credentials** grant there. | ||
|
|
||
| Connection adapters | ||
| =================== | ||
|
|
||
| * :meth:`QuestDB.sql <questdb.auth.QuestDB.sql>` — query over REST ``/exec`` to a | ||
| pandas DataFrame using ``Authorization: Bearer``. Recommended: there is no | ||
| token-length limit (a groups-encoded JWT can be several KB). | ||
| * :meth:`QuestDB.sqlalchemy_engine <questdb.auth.QuestDB.sqlalchemy_engine>` — | ||
| PG-wire engine that injects a fresh token as the ``_sso`` password for every | ||
| new connection. Requires ``acl.oidc.pg.token.as.password.enabled=true``. | ||
| * :meth:`QuestDB.psycopg <questdb.auth.QuestDB.psycopg>` — a raw psycopg / | ||
| psycopg2 connection. | ||
| * :meth:`QuestDB.sender <questdb.auth.QuestDB.sender>` — a | ||
| :class:`~questdb.ingress.Sender` for ingestion (ILP over HTTP). | ||
|
|
||
| .. note:: | ||
|
|
||
| QuestDB validates the token at **authentication** time, not per query. An | ||
| already-open PG connection survives token expiry; only **new** connections | ||
| need a fresh token — which is why the PG-wire adapter supplies the token | ||
| per-connect. | ||
|
|
||
| .. _oidc_idp_requirements: | ||
|
|
||
| IdP requirements | ||
| =============== | ||
|
|
||
| The OIDC client referenced by ``acl.oidc.client.id`` must: | ||
|
|
||
| * have the **Device Authorization grant** enabled; | ||
| * be a **public client** (no secret in a notebook); | ||
| * optionally issue **refresh tokens** for the device grant (for silent refresh); | ||
| * issue tokens whose ``aud`` matches ``acl.oidc.audience`` (some IdPs need an | ||
| ``audience``/``resource`` request parameter); | ||
| * include the **groups** claim in the token (``groups.encoded.in.token=true``) | ||
| or expose it via the **userinfo** endpoint (``false``), matching the server. | ||
|
|
||
| Security notes | ||
| ============= | ||
|
|
||
| * No IdP passwords are ever entered in the notebook; MFA/SSO happen at the IdP. | ||
| * ``https`` is required. Plaintext ``http`` to a **loopback** address | ||
| (``localhost`` / ``127.0.0.1`` / ``::1``) is always allowed — it never leaves | ||
| the host. ``insecure=True`` additionally permits plaintext to a non-loopback | ||
| **QuestDB** host (local development only); it does **not** downgrade the | ||
| **IdP**, so the device code and refresh token are never sent in cleartext | ||
| over the network. Certificate verification is never disabled. | ||
| * **Endpoint trust.** The device code and the long-lived refresh token are sent | ||
| to the device-authorization and token endpoints, which are discovered from | ||
| QuestDB ``/settings``. The helper requires both endpoints to share a single | ||
| origin and rejects the configuration otherwise. Because ``/settings`` is | ||
| authoritative-by-QuestDB, a compromised server could in principle point them | ||
| elsewhere; pass ``issuer=`` (or ``discovery_url=``) to **pin** the IdP so the | ||
| endpoints are verified to belong to it and credentials can't be redirected to | ||
| another host. | ||
| * Adapters avoid logging the token / PG DSN. Avoid logging them yourself. | ||
| * Standard proxy / CA settings (``HTTPS_PROXY``, ``REQUESTS_CA_BUNDLE``, | ||
| ``SSL_CERT_FILE``) are honoured; you can also pass ``ca_bundle=``. | ||
|
|
||
| Dependencies | ||
| =========== | ||
|
|
||
| ``token()`` / ``headers()`` need nothing beyond the standard library. The | ||
| following are imported lazily, only when used: | ||
|
|
||
| * ``pandas`` — for :meth:`QuestDB.sql`; | ||
| * ``sqlalchemy`` and ``psycopg`` / ``psycopg2`` — for the PG-wire adapters; | ||
| * ``qrcode`` — to render a QR code for phone-based authorization (``qr=True``); | ||
| * ``IPython`` — for the rich Jupyter prompt (falls back to plain text). | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -14,6 +14,7 @@ Contents | |
| installation | ||
| sender | ||
| conf | ||
| auth | ||
| examples | ||
| api | ||
| troubleshooting | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.