feat: OIDC device-flow authentication - #133
Conversation
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughThis PR introduces the Changesquestdb.auth OIDC Device Authorization Grant
Claude PR Review Skill
Sequence Diagram(s)sequenceDiagram
participant User
participant connect as questdb.auth.connect()
participant OidcDeviceAuth
participant resolve_config
participant QuestDB_Settings as QuestDB /settings
participant IdP
participant Renderer
participant TokenCache
participant QuestDB_Session as QuestDB session
User->>connect: connect(url, eager=True)
connect->>OidcDeviceAuth: from_questdb(url)
OidcDeviceAuth->>resolve_config: url, overrides
resolve_config->>QuestDB_Settings: GET /settings
QuestDB_Settings-->>resolve_config: acl.oidc.* config
resolve_config->>IdP: GET /.well-known/openid-configuration (if needed)
IdP-->>resolve_config: device_authorization_endpoint
resolve_config-->>OidcDeviceAuth: OidcConfig
connect->>QuestDB_Session: QuestDB(url, auth)
connect->>OidcDeviceAuth: token() [eager]
OidcDeviceAuth->>TokenCache: load(cache_key)
TokenCache-->>OidcDeviceAuth: miss
OidcDeviceAuth->>IdP: POST device_authorization_endpoint
IdP-->>OidcDeviceAuth: device_code, user_code
OidcDeviceAuth->>Renderer: on_prompt(resp)
Renderer-->>User: display verification URI + user code
loop Poll IdP
OidcDeviceAuth->>IdP: POST token_endpoint (device_code grant)
IdP-->>OidcDeviceAuth: authorization_pending / tokens
end
OidcDeviceAuth->>TokenCache: store(cache_key, TokenSet)
OidcDeviceAuth->>Renderer: on_success(identity, expires_in)
connect-->>User: QuestDB session
User->>QuestDB_Session: sql("SELECT ...") / sender() / sqlalchemy_engine()
🎯 4 (Complex) | ⏱️ ~75 minutes
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Actionable comments posted: 6
🧹 Nitpick comments (1)
src/questdb/auth/_questdb.py (1)
82-94: 💤 Low valueChain the import exception for better diagnostics.
When both
psycopgandpsycopg2fail to import, the raisedImportErrorloses the underlying cause. Chaining the exception preserves the diagnostic context.Suggested fix
def _pg_module(): try: import psycopg # type: ignore # psycopg v3 return psycopg except ImportError: pass try: import psycopg2 # type: ignore return psycopg2 - except ImportError: + except ImportError as e: raise ImportError( 'A PostgreSQL driver is required: install `psycopg` (v3) or ' - '`psycopg2-binary`.') + '`psycopg2-binary`.') from e🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/questdb/auth/_questdb.py` around lines 82 - 94, The second except ImportError block in the _pg_module() function should preserve the exception chain for better diagnostics. Capture the ImportError exception in the second except block using `as e` syntax, then use `raise ... from e` syntax when raising the new ImportError to chain the exception, which preserves the underlying cause and provides better debugging context.Source: Linters/SAST tools
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.claude/skills/review-pr/SKILL.md:
- Around line 157-162: The Step 3 preamble at lines 157-162 states that every
agent receives the full change-surface map, but Agent 10's input contract at
lines 231-240 explicitly limits it to only the PR diff and changed file names.
Add an explicit exception clause in the Step 3 preamble section to clarify that
Agent 10 does not receive the full change-surface map and instead receives only
the diff and changed file names, ensuring consistency between the general rule
and the specific Agent 10 behavior documented at lines 231-240.
- Around line 39-42: The table at lines 39-42 states that level 0 (default)
skips Step 2.5, but there is a later reference (around lines 144-155) that marks
Step 2.5e as mandatory at every level, creating a contradiction. Fix this
inconsistency by clarifying the level 0 description in the table: either modify
the "Skip Step 2.5" statement to specify that Step 2.5e is an exception and is
still performed, or restructure it to list exactly which substeps (2.5a, 2.5b,
2.5c, 2.5d, 2.5e) are executed or skipped. Ensure the description at level 0 and
the mandatory requirement stated at lines 144-155 are aligned so the default
path is no longer self-contradictory.
In `@docs/auth.rst`:
- Around line 65-76: The code example in the documentation is missing the import
statement for TimestampNanos, which is used in the sender.row() method call. Add
an import statement at the top of the code snippet to include TimestampNanos
from the questdb.ingress module (or appropriate module) so that the example can
be executed successfully when copied verbatim.
In `@src/questdb/auth/__init__.py`:
- Around line 75-92: The `__all__` list in the module needs to be sorted
alphabetically to comply with Ruff's RUF022 rule and maintain consistency in the
public API exports. Reorder all the items in the `__all__` list (which includes
'connect', 'QuestDB', 'OidcDeviceAuth', 'OidcConfig', 'TokenCache', 'TokenSet',
'MemoryCache', 'FileCache', 'NullCache', and the various OidcError types) in
alphabetical order while ensuring all items remain in the list.
In `@src/questdb/auth/_http.py`:
- Around line 123-129: The `_opener()` function doesn't receive the `insecure`
parameter, so it cannot enforce HTTPS-only redirect policies. To fix this, add
an `insecure` parameter to the `_opener()` function signature. When
`insecure=False`, create a custom redirect handler that validates redirects do
not downgrade from HTTPS to HTTP for non-loopback hosts (similar to the
validation in `_require_secure()`), and pass this handler to
`urllib.request.build_opener()` instead of using the default. When
`insecure=True`, use urllib's default redirect handling. Update all callers of
`_opener()` to pass the `insecure` parameter value.
In `@test/test_auth.py`:
- Around line 600-605: The timed joins with a 10-second timeout do not verify
that threads actually completed, which can cause non-daemon threads to remain
running and hang the test process if a regression causes a deadlock. After
calling t.join(10) on each thread in both the loop at lines 600-605 and at lines
828-835, add a verification check that confirms each thread has terminated by
asserting that t.is_alive() returns False, or raise an exception if any thread
is still alive after the timeout. This ensures the test fails cleanly rather
than leaving hanging threads.
---
Nitpick comments:
In `@src/questdb/auth/_questdb.py`:
- Around line 82-94: The second except ImportError block in the _pg_module()
function should preserve the exception chain for better diagnostics. Capture the
ImportError exception in the second except block using `as e` syntax, then use
`raise ... from e` syntax when raising the new ImportError to chain the
exception, which preserves the underlying cause and provides better debugging
context.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: bf4af383-9022-4ae5-b189-73691f4ccd00
📒 Files selected for processing (18)
.claude/skills/review-pr/SKILL.mdCHANGELOG.rstdocs/api.rstdocs/auth.rstdocs/index.rstdocs/installation.rstexamples/oidc_device_auth.pysetup.pysrc/questdb/auth/__init__.pysrc/questdb/auth/_cache.pysrc/questdb/auth/_device.pysrc/questdb/auth/_discovery.pysrc/questdb/auth/_errors.pysrc/questdb/auth/_http.pysrc/questdb/auth/_questdb.pysrc/questdb/auth/_render.pytest/test.pytest/test_auth.py
The fastparquet -> pyarrow parquet roundtrip decays the categorical
column to a plain string column. On pandas >= 3 that reads back as the
new default string dtype (StringDtype(na_value=nan)) rather than object,
so the hardcoded np.dtype('O') in fallback_exp_dtypes no longer matched
and the assertion failed.
Derive the expected dtype from pd.Series(['x']).dtype instead of
hardcoding it: this is object on pandas < 3 and the new string dtype on
pandas >= 3, matching exactly what pyarrow's read_parquet produces, so
the test is version-agnostic.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Pandas 3 ships no 32-bit wheels. On win32 Python 3.11+ the pandas>=3 install was silently swallowed, fastparquet then pulled in a numpy-1-built pandas 2.0.3 alongside numpy 2, and importing pandas crashed at runtime. test.py tolerated the failed import and silently skipped every pandas test (skip count 39 vs 32), while the 64-bit-only import sanity check never fired to catch it. Gate should_use_pandas3() on a 64-bit interpreter so 32-bit targets stay on the consistent pandas 2 / numpy 1 stack and actually exercise the dataframe tests again. 64-bit targets keep testing pandas 3 unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The mock HTTP server only caught BrokenPipeError, but on Windows an abrupt client disconnect raises ConnectionAbortedError/ConnectionResetError -- siblings of BrokenPipeError under the common base ConnectionError, not subclasses of it. The timeout, min-throughput, and retry tests disconnect mid-request on purpose, so these slipped past the handler and the stdlib dumped tracebacks to stderr. The tests still passed, but the CI logs looked broken. Broaden the handler except clauses to ConnectionError, and override HTTPServer.handle_error to swallow connection errors that surface in the stdlib keep-alive readline of the next request line -- outside any request handler's try/except. Real errors are still reported. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The windows-2025 image injects a new readonly agent variable, AZP_ENHANCED_WORKER_CRASH_HANDLING. The Windows "Build wheels" step re-exports the vcvars environment via ##vso[task.setvariable ...], and attempting to set this readonly var made the agent emit an ##[error], marking the task failed even though all wheels built and tests passed. Add AZP_ENHANCED to the exclusion regex (prefix match also covers any future AZP_ENHANCED_* vars) in both windows_i686 and windows_x86_64. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Review fixes on the OIDC device-flow auth module, plus a simplification that removes the file cache entirely. Hardening: - device-flow poll gates success on _has_required_token (id_token in groups mode, else access_token) instead of always access_token: a completed grant missing the required kind now fails once with a clear error instead of caching an unusable token or discarding a usable id_token. - QuestDB.sql() guards the 2xx path against non-JSON / non-dict bodies, raising OidcError instead of a raw JSONDecodeError / AttributeError. - discovery requires an explicit issuer= (or discovery_url=) before the IdP .well-known fallback; the discovery origin is never derived from a server-supplied token endpoint, so a tampered /settings can't redirect the device-code / refresh-token POSTs. - PG-wire / ILP adapters bracket IPv6 literals in the ILP addr= and raise a clear error on a host-less URL instead of passing None to the driver. - validate_endpoint_origins and cache-key normalization raise OidcConfigError (not a bare ValueError) on a malformed port, via a shared safe_urlparse helper. - the example imports questdb.ingress lazily, so it loads on the pure-Python path with no compiled extension. Simplification: - drop FileCache and its cross-process locking + at-rest refresh token; MemoryCache (process-global, survives notebook cell re-runs) is the only persistent backend, with NullCache for cache=None. This also removes the Windows msvcrt-lock no-op and corrupt-file edge cases entirely. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…file The linux-qdb-master job's "Compile QuestDB master" step failed because questdb master depends on a -SNAPSHOT java-questdb-client that is not published to Maven Central. Add a detect-local-client step template (adapted from questdb/questdb's ci/templates/detect-local-client.yml) that reads questdb.client.version from the cloned core/pom.xml: for a -SNAPSHOT client it inits the java-questdb-client submodule and builds it via the `local-client` Maven profile; for a released client it resolves from Maven Central. The "Compile QuestDB master" step appends the resulting $(CLIENT_PROFILE). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
QuestDB master bumped its build to Java 25 (javac.target=25). Its maven-enforcer requireJavaVersion reads ${java.enforce.version}, which is only set by the JDK-activated 'java25+' profile (<jdk>(24,)</jdk>). Building the linux-qdb-master leg with JDK 17 left that property empty, so the enforcer failed with 'JDK version can't be empty' before compilation.
Point the 'Compile QuestDB master' Maven task at $(JAVA_HOME_25_X64) via jdkVersionOption: path, and run 'Test vs master' on the same JDK 25 so the freshly compiled Java 25 bytecode can run. 'Test vs released' stays on JDK 17. Both ubuntu-latest and windows-2025 images preinstall Temurin 25 (JAVA_HOME_25_X64).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The Maven@3 task crashes while parsing JDK 25 ('Cannot read properties of null (reading 'major')') — its JDK selection tops out at 21 and its Node-side version detector returns null for 25. Replace the task with a bash step that exports JAVA_HOME=$(JAVA_HOME_25_X64) and runs mvn directly, mirroring the task defaults (questdb/pom.xml, goal 'package', same -DskipTests -Pbuild-web-console $(CLIENT_PROFILE) options).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
QuestDB master runs as the io.questdb JPMS module and now uses jdk.internal.vm.ContinuationScope, so on JDK 25 the server dies at startup with IllegalAccessError (java.base does not export jdk.internal.vm to io.questdb), plus Unsafe/native-access warnings. The test fixture launches questdb.jar directly rather than via questdb.sh, so the access flags questdb.sh normally supplies are absent. Set JDK_JAVA_OPTIONS on the 'Test vs master' step with the exact module access flags from questdb.sh (all targeting io.questdb). Scoped to that step, so the JDK 17 'Test vs released' run is unaffected. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
When QuestDB is reached over plaintext http to a non-loopback host (only possible with insecure=True), its /settings response is MITM-able. The issuer-pin requirement previously fired only when an IdP endpoint was missing (the discovery path). A tampered /settings advertising BOTH the token and device-authorization endpoints at one attacker origin skipped that path: the co-location check passed trivially (same origin) and the issuer-pin check was vacuous (no issuer), so the device code and the long-lived refresh token were POSTed to the attacker. Require the same out-of-band pin (issuer= / discovery_url=) before trusting /settings-supplied credential endpoints fetched over such an untrusted channel. Endpoints the caller passed explicitly, and endpoints from an authenticated (https / loopback) /settings, are unaffected, so the https happy path and local-dev loopback are unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The device-authorization response's expires_in / interval were trusted
verbatim, so a hostile or buggy IdP could break or stall the poll loop —
which runs under the acquisition lock, so a stall freezes every other
thread needing a token on that instance:
* expires_in <= 0 set the deadline to "now", timing the flow out before
its first poll even though the user could still authorize;
* an unbounded interval (or repeated slow_down) produced a single
enormous sleep() holding the lock.
Clamp both: expires_in <= 0 -> default, capped at a max lifetime; interval
to [1s, 60s] (including after slow_down); and never sleep past the
deadline. RFC-typical values (interval=5, expires_in=600) are unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Several malformed-input paths escaped the package's typed-error contract
(callers catch OidcError) with a bare ValueError / AttributeError /
http.client.InvalidURL:
* a non-string OIDC endpoint in /settings -> AttributeError from
.startswith(); now treated as absent so resolution raises a clear
OidcConfigError;
* a /exec "columns" entry that isn't an object -> AttributeError from
.get(); now raises OidcError;
* a malformed port in the QuestDB URL -> bare ValueError when an adapter
read .port; QuestDB now validates it at construction via safe_urlparse;
* the same malformed port reaching the /settings or discovery fetch ->
http.client.InvalidURL; request() now wraps InvalidURL as
OidcConfigError and any other HTTPException as OidcNetworkError, so the
single HTTP choke point never leaks a raw http.client exception.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The plain-text terminal prompt wrote untrusted device-authorization response fields — verification_uri, user_code, the IdP error_description and the JWT-derived identity — verbatim to the TTY. A hostile or MITM'd response could embed ANSI escape sequences (cursor moves, screen clears) to spoof the sign-in prompt or hide the real verification URL. Strip C0/C1 control characters (incl. ESC) from those untrusted strings in format_prompt() and TerminalRenderer.on_success/on_failure before they reach the stream. The Jupyter renderer already html-escapes its output, and the QR path encodes the URL as image data rather than terminal text, so neither needed changes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* CHANGELOG: drop the stale "optional on-disk cache" claim (the FileCache backend was removed; tokens are never written to disk) and note the python_requires bump to 3.10. * docs/auth.rst: import TimestampNanos in the integrated-session snippet so it runs as written. * docs/api.rst: document TokenCache, TokenSet, MemoryCache and NullCache, which are exported in __all__ but were missing from the reference. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The lock-free fast path in OidcDeviceAuth reads a published TokenSet without holding a lock, which is only safe because its fields never change after construction — an invariant previously kept by convention alone. Mark TokenSet frozen so any future in-place mutation fails loudly instead of introducing a torn read, and convert the one such mutation (the refresh carry-forward in _refresh) to dataclasses.replace(). Also keep the access/id/refresh tokens out of repr() so a TokenSet that lands in a log line or traceback cannot leak credentials. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Reorder the export list using Ruff's isort-style ordering (CamelCase names first, natural-sorted, then lowercase 'connect' last). The public API is unchanged; this only resolves the RUF022 lint warning. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The level table said level 0 skips all of Step 2.5, but Step 2.5e is documented as running at every level — a self-contradictory default path. Clarify that levels 0 and 1 skip Steps 2.5a-d while still running Step 2.5e (build & binding profile), aligning the table with the 'runs at every level' rule. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The Step 3 preamble and Step 2.5 stated that every Step 3 agent receives the change-surface map and build/binding profile facts, but Agent 10 (the fresh-context adversarial agent) is documented as receiving only the diff and changed file names. Carve Agent 10 out of all three general-rule statements (Steps 2.5, 2.5e, and the Step 3 input contract) so they no longer contradict Agent 10's own section. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
QuestDB /settings nests server-authoritative values under a top-level "config" object alongside a user-writable "preferences" sibling (the web console persists UI prefs there via PUT /settings). Discovery now reads only "config" and refuses to fall back to the top level of a structured response, so a user who can write a preference cannot smuggle an acl.oidc.* key (e.g. a redirected token endpoint that points the device code / refresh token at an attacker) into the resolved OIDC config. Genuinely flat legacy responses are still tolerated at the top level. Ports the trust model from the Java client (java-questdb-client#52). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
QuestDB._require_host() passed the URL hostname unsanitized into the ILP
conf string sender() builds (addr=host:port;). urlparse keeps ';' and '='
in .hostname, so a crafted or tampered URL such as
"https://host;tls_verify=unsafe_off;x=" injected extra conf params —
silently disabling the sender's TLS certificate verification (and exposing
the bearer token to a MITM), or e.g. auto_flush=off for data loss.
Reject ';', '=', whitespace and control characters in the resolved host
(':' stays allowed for IPv6 literals) at the single chokepoint shared by
sender()/psycopg()/sqlalchemy_engine(). Add a regression test.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
M1 — keep four untrusted-input failures within the OidcError contract
instead of leaking a raw stdlib exception:
* int(float('inf')) from a JSON Infinity (json.loads accepts it) raised
OverflowError — not a ValueError — in the expires_in/interval parses
(_device.py); add OverflowError to those handlers.
* urllib.parse.urlparse() itself raises ValueError on a malformed IPv6
literal (e.g. "https://[::1") before .port is read. safe_urlparse now
wraps the urlparse() call, and _require_secure routes through it, so a
bad endpoint from /settings or discovery raises OidcConfigError.
* build_ssl_context() leaked FileNotFoundError/ssl.SSLError for a
mistyped ca_bundle path (or env var); map to OidcConfigError.
* deeply-nested JSON makes json.loads raise RecursionError — not a
ValueError; catch it in get_json/post_form and QuestDB.sql.
M2 — _strip_control now also strips Unicode bidi-override / zero-width /
line-separator ranges (U+200B-200F, U+2028-202E, U+2066-2069, U+FEFF),
not just C0/C1. U+202E (RIGHT-TO-LEFT OVERRIDE) in an untrusted device
response could otherwise reverse displayed text in the terminal prompt to
disguise the real sign-in host.
Add 8 regression tests (overflow expires_in/interval, malformed-IPv6 at
safe_urlparse/_require_secure/_normalize_url/validate_endpoint_origins/
constructor, bad CA bundle, deeply-nested JSON, bidi/zero-width stripping).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
M3 — thread a per-request `timeout` (default 30s, unchanged) through every IdP call (device-code request, each poll, refresh) via _idp_post, and through the /settings + discovery fetches in from_questdb. It bounds how long a single network leg can hold the token-acquisition lock when the IdP stalls, so an IdP outage no longer freezes other acquiring threads (e.g. SQLAlchemy pool connections) for the urllib default per leg. Exposed on OidcDeviceAuth.__init__ / from_questdb and forwarded by connect(**opts): lower it for tighter lock-hold, raise it for a slow IdP. Total interactive poll duration stays capped by _MAX_DEVICE_CODE_LIFETIME. M4 — add the remaining edge / error-path tests: * device-code / poll / refresh use the configured timeout (M3) * connect(eager=False) defers sign-in until the first token use * IdP .well-known 404 -> OidcError (discovery non-2xx path) * token/device non-JSON 2xx and non-dict JSON -> OidcError; /settings and discovery non-2xx / non-JSON -> OidcError (only /exec had this before) * make_cache resolves 'memory' / None / 'none' / a TokenCache and rejects an unknown spec with OidcConfigError * QR helpers degrade to None when `qrcode` is absent or data is empty Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* cache_key now includes the token-kind mode (groups_in_token): two
sessions differing ONLY in that mode no longer collide on one in-memory
cache entry and repeatedly evict each other's token. _select gates the
served kind so no wrong token was ever returned, but the collision caused
avoidable refreshes / re-prompts. (_device.py)
* _resolve_endpoint returns None for a path-only endpoint ("/as/token")
when acl.oidc.host is absent, so resolution fails with a clear config
error (pin the IdP / pass the endpoint explicitly) — or recovers via IdP
discovery when issuer= is pinned — instead of passing a scheme-less
"/path" downstream that surfaced as a confusing "insecure/malformed
URL". (_discovery.py)
* _pg_module chains the underlying ImportError (raise ... from e) so the
traceback preserves the real cause. (_questdb.py)
Add 3 regression tests.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
M1 (issuer pin path-scope) — the issuer pin now scopes /settings-advertised
credential endpoints to the issuer PATH (segment-aware), not just its
origin, so a tampered /settings can't steer the device code / refresh
token to a different realm on a path-based IdP (Keycloak issuers are
https://host/realms/{realm}). The origin check (validate_endpoint_origins)
stays universal; the path check lives in resolve_config and applies ONLY
to /settings-supplied endpoints — caller-explicit and IdP-discovered
endpoints are authoritative and not path-restricted, so IdPs that place
endpoints outside the issuer path (e.g. Azure AD) still work.
M2 (forward CA to Sender) — QuestDB.sender() now forwards the private CA
(explicit ca_bundle=, else REQUESTS_CA_BUNDLE / SSL_CERT_FILE, same
precedence as build_ssl_context) to the ILP Sender as tls_roots for an
https QuestDB, so REST queries and ILP ingestion trust the same roots. PEM
file only; caller can override via tls_roots=/tls_ca=. ca_bundle is now
stored on OidcDeviceAuth and read by QuestDB.
M4 (token-field race) — the lock-free fast path is now strictly READ-ONLY:
the cache->field promotion of self._tokens moved into the locked slow-path
re-check (and an expired token is still promoted there so _acquire can
refresh it). Every write to self._tokens is now serialized; the lock-free
read of the frozen TokenSet is kept.
M5 (default_interval) — from_questdb (and thus connect(**opts)) now accepts
and forwards default_interval; connect(default_interval=...) previously
raised TypeError.
(M3, malformed-IPv6 -> OidcConfigError, was already fixed in ae4c5ba.)
docs/auth.rst: issuer pin documented as origin+path with the Azure caveat;
sender documented to inherit the CA bundle. Add 6 regression tests.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- _decode_jwt_claims: catch RecursionError so a deeply-nested JWT payload from a hostile/buggy IdP degrades to no-identity instead of crashing token()/refresh with a raw exception (mirrors _http/_questdb guards). - _endpoint_path_under_issuer: reject '.'/'..' path segments (decoding %2e first) so a tampered /settings can't steer the device code and refresh token to a different realm via a path the IdP normalizes after the prefix check passes. - _render: coerce non-string verification_uri/_complete to str/None and harden _safe_link_url, so an untrusted device response can't crash the prompt renderer with a bare TypeError/AttributeError. - OidcDeviceAuth docstring: document that a missing/expired-token caller blocks behind an in-progress interactive sign-in, and that connect( eager=True) (the default) is the mitigation. Adds regression tests for each (full auth suite: 127 pass, 11 skipped). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Three more review fixes to questdb.auth: - M5: sqlalchemy_engine's do_connect listener ran auth.token() on pool threads, so a first-time interactive sign-in could block the whole pool (or storm OidcInteractionRequired in a headless pool). Thread an internal allow_interactive flag: the pool callback now reuses / silently refreshes the cached token but refuses to START the device flow, raising a clear "sign in up front" error instead. Public token() is unchanged, and the silent refresh still runs on pool threads. - M6: a slow_down poll response carrying a Retry-After lower than the current interval reset the interval to the floor -- polling FASTER right after the IdP told it to slow down (RFC 8628 section 3.5 requires an increase). slow_down now never drops below current + 5; a plain 429/5xx still honors Retry-After verbatim. - M4: the concurrency stress test used a single instance, so the cross-instance generation/inflight CAS (the pool case) was never raced. Add a cross-instance stress test driving several instances that share one cache_key under real contention. Full questdb.auth suite green (217 tests). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Remaining minor review fixes (the no-GIL CI gap is intentionally left out): - Cross-instance cache: when an instance held a stale (non-None, expired) self._tokens while another instance sharing the process-global cache had stored a fresh valid token, the locked slow path reloaded the shared store only when self._tokens was None -- so the stale local token shadowed the fresher cached one and forced a redundant refresh. Reload and adopt a valid cached token whenever the local one is stale. - Retry-After on non-JSON errors: post_form parsed Retry-After off the response headers but dropped it when the body was non-JSON, so a non-JSON 429/503 from a proxy/WAF fell back to the fixed +5s step. Carry it on OidcError.retry_after and honor it in the poll loop's exception arm. - _display_url: parts.port raises ValueError on a junk port, which aborted host normalization and showed a homoglyph host raw. Read the port defensively so the host is still IDNA/punycode-normalized (junk port dropped) -- the spoof is still revealed. - Docs/infra: drop stale Python 3.8 references (RELEASING.rst, a dead proj.py comment); narrow the mock server's handle_error suppression to the specific client-disconnect errors (not all ConnectionError); group the exception autodoc by hierarchy and document OidcConfig's fields. Adds regression tests for the first three (all teeth-checked by mutation). Full questdb.auth suite green (221 tests). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
_display_url rebuilds an untrusted verification host in IDNA/punycode to defeat homoglyph spoofing, but a confusable that NFKC-folds to a URL delimiter (fullwidth solidus U+FF0F -> '/', U+FF20 -> '@', U+FF03 -> '#', U+FF1F -> '?') makes urllib.parse.urlparse raise ValueError, and the except branch returned the raw string. So a value like "https://login.questdb.io<U+FF0F>@evil.example/device" was shown verbatim in the terminal and Jupyter prompts -- reading as host login.questdb.io while a browser resolves evil.example after the fold. Confined to inert text (never clickable/opened/QR-encoded -- _safe_target already rejected these), but that is exactly the manual copy/retype path the IDNA-display defense exists to stop. Route both fail-open branches (and the existing IDNA-failure branch) through a shared _ascii_visible() that escapes non-ASCII to a visible \uXXXX, so a confusable can't masquerade as a trusted host even when the host can't be normalized. Add a TestRendererSecurity regression covering U+FF0F/FF20/FF03/FF1F. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Three minor findings from the device-flow review: - Poll loop: a non-JSON 5xx from a proxy/WAF carrying a Retry-After header was ignored by the exception arm (only a 429 backed off), while the JSON-body path honored it. Mirror the JSON arm so a transient 5xx with a Retry-After backs off by that value; a 429 still backs off without a header, and a 5xx without one keeps its cadence. - cache_key: the issuer was trailing-slash-normalized but the token endpoint was not, so ".../token" and ".../token/" produced different keys and forced an avoidable re-prompt. rstrip the normalized token endpoint too; the realm path still distinguishes multi-tenant keys. - Renderer re-entrancy: the acquisition lock is held across the whole sign-in, including renderer callbacks, so a custom renderer calling back into the same instance's token()/clear() silently deadlocked for up to the device-code lifetime. Track the lock-owning thread and raise a typed OidcError on same-thread re-entry instead of hanging. Adds a regression test for each. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
urllib.parse.urlparse() silently removes tab/newline/CR from the URL before producing .netloc, but the transport (http.client via Request.host) keeps them. So _reject_confusable_authority validated a host that could diverge from the one urllib connects to — e.g. "https://idp\t.good/token" parses with hostname "idp.good" while the transport targets the raw "idp\t.good". Its _UNSAFE_AUTHORITY_RE lists \s/\x00-\x1f but never saw these bytes because urlparse stripped them first, leaving the guard's stated invariant to two incidental backstops (the issuer/origin pin and http.client._validate_host). Check the raw URL for these bytes up front so the dedicated guard enforces its own contract. Add a regression test covering both the label-merge and host-split cases across every construction path. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
m1: _safe_link_url rejected only surrounding whitespace, but urlparse silently removes tab/newline/CR from anywhere in the URL before parsing — so it would vet the stripped form yet return (→ href / browser / QR) the original with the bytes intact. Reject a URL carrying them so the value vetted equals the value returned. Production already strips via _safe_target; this closes the standalone footgun. m2: an explicit host="[::1]" override reached the PG driver bracketed (→ a confusing connection failure on a copy-pasted IPv6 literal), while the URL-derived path is unbracketed by urlparse. Strip a surrounding [...] so both paths hand the driver a bare address, matching the docstring. m3: post_form's return annotation said tuple[int, Dict] but it returns _PostResult (a tuple subclass carrying .retry_after); correct it. Add regression tests for m1 and m2. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Point docstring :class: refs at the public questdb.auth.* targets that api.rst documents, instead of the private _device/_errors module paths (unresolved under the -nW nitpicky doc build). - Add a test that a well-formed JWT whose payload decodes to non-object JSON (list/str/number) reads as no-claims and doesn't crash the success-path identity decode, guarding _decode_jwt_claims. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Isolate custom-renderer exceptions in the device flow: route on_prompt/on_waiting/on_failure through a best-effort _render_safe so a buggy renderer can't mask the authoritative OidcDeviceFlowError / OidcTimeoutError. An OidcError is re-raised, not swallowed, so the reentrancy guard's deadlock signal still reaches the caller; on_success stays inline-wrapped (it must never discard an already-authorized token). - Reject '%' (IPv6 zone-id / percent-encoding) in a credential-endpoint authority, matching the host hygiene in _adapters._ILLEGAL_HOST_CHARS and _render._SAFE_HOST_RE that the comment already claimed to mirror; update the comment and error message to match. - Correct the OidcConfig.scope docstring: the openid scope is added by OidcDeviceAuth in groups mode, not by the dataclass. - Add regression tests for the renderer-masking and %-authority cases. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The response HEAD read (status line + headers) inside urllib's open() had no wall-clock bound: urllib's timeout is per-socket-read and http.client's begin() loops readline() over reads, so a peer dribbling the status/header bytes one per timeout window kept open() blocked for up to ~_MAXLINE * timeout per line. This pinned the thread holding the acquisition lock and defeated the existing body-read watchdog, which is armed only after open() returns. Capture the connection socket as soon as it connects (a do_open-only mixin that swaps in a socket-capturing connection class, leaving the stdlib's version-specific TLS handling untouched) and arm a watchdog that shuts it down at the deadline, released once the head read finishes so it can't disturb the body read. A stalled head now surfaces as a typed OidcNetworkError instead of hanging. Add a real-socket regression test that dribbles the status line. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
M2 — the in-memory cache_key and the on-disk TokenStoreKey derived identity by different rules, so they could disagree: the token-endpoint query string was kept in memory but dropped on disk (two distinct tenants could collide onto one store file and be served each other's token), while scope order and a trailing slash were normalised in memory but not on disk (splitting one identity across two files). Give both a single normalisation: _canonical_endpoint now keeps the query and strips a trailing slash, and the store key's scope is order-normalised via the shared _normalize_scope, exactly as cache_key already does. For the common case (no query, no trailing slash) the on-disk hash is unchanged, so the cross-language file-sharing contract is preserved; edge cases now agree and fail safe. M3 — a token taken straight from the (untrusted) IdP token endpoint was not screened for control/non-ASCII characters, unlike a token loaded from the persistence file. A decoded CR/LF in the served token is an Authorization-header / PG-wire _sso-password injection vector. Route the wire-bound access/id tokens through the same _has_only_token_chars gate the file path uses (new _safe_token_or_none); a screened-out token reads as absent, so the grant fails terminally instead of routing a tampered credential onto the wire. M4 — add the two missing persistence tests: a coordinated refresh using the newer in-memory token when a prior save left disk stale, and clear() staying non-fatal when the store's lock backend (not clear() itself) raises. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Four minor hardening fixes: - _seconds_to_millis checked finiteness BEFORE the *1000 scale, so a finite-but-huge value (e.g. 1e306) passed, overflowed to inf when scaled, and int(round(inf)) raised a raw OverflowError — escaping the store's OidcError contract via the public PersistedToken/save. Check finiteness after scaling. - FileTokenStore accepted a non-finite lock_stale / lock_acquire_budget: inf passes the bare `> 0` / `> _MIN_LOCK_STALE` comparisons, and an infinite staleness window means a crashed holder's lock is never reclaimed. Reject non-finite (and a too-large int) up front. - _strip_control kept variation selectors (U+FE00–FE0F, U+E0100–E01EF, category Mn) and enclosing combining marks (category Me), both invisible/overlay characters that reached a user_code / identity / URL path. Strip them, and cap a run of non-spacing marks (Mn) so a "Zalgo" stack can't smear over adjacent prompt lines; legitimate accents and a short combining run still render, and interleaved zero-width chars can't reset the cap. - The direct OidcDeviceAuth(...) constructor validated only the two endpoints, not the issuer (resolve_config does on the from_questdb path), so a confusable or malformed issuer constructed fine and then raised lazily from cache_key on the first token() call. Validate the issuer authority in __init__ so it fails fast at construction. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The in-memory cache_key rstrip('/')-ed the whole rendered URL, while the
on-disk _canonical_endpoint strips the trailing slash on the path
component. That whole-string strip disagreed with the store on two
endpoint shapes: a slash before a query ('.../token/?x' stayed split
from '.../token?x'), and a slash that is part of a query value
('...?redirect=a/' wrongly collided with '...?redirect=a'). Either case
keyed one identity two ways in memory vs on disk.
Move the trailing-slash stripping into _normalize_url, applied to the
path component (mirroring _canonical_endpoint), and drop the now-redundant
cache_key rstrip. The two keys now make identical identity distinctions.
Also add the missing TLS-verification coverage: a unit assertion that
build_ssl_context() enforces CERT_REQUIRED + check_hostname (so a
regression swapping in an unverified context fails a test), plus a
behavioural handshake test that the default context rejects an untrusted
self-signed cert while a custom-CA context still verifies and connects.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Address moderate findings from the PR #133 review: - Persist and re-check `issuer` in the token store so a token pinned to one issuer is never served from disk to a session pinned to another. Kept out of the file-name hash, so the cross-language file contract and existing token files stay byte-stable; isolation is enforced via the on-load identity re-check, matching the in-memory cache_key. - FileTokenStore.load() now returns None (not raises) for an unreadable entry: a directory/symlink-loop at the token path (stat.S_ISREG/errno guard) and a deeply-nested attacker file (RecursionError), restoring its documented contract and matching every other json.loads site. - _display_url escapes a host that still carries a URL-structural char after IDNA (e.g. the U+FF3C/U+FE68 reverse-solidus fold) as a visible \uXXXX, instead of a bare backslash a browser would read as a slash. - CHANGELOG: relabel the Python 3.8->3.10 bump as a breaking change with the whole-package blast radius, EOL rationale, and a migration note. Adds regression tests for each; full auth suite passes (308 tests). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Latest hardening round for the OIDC device-flow module, incorporating a
code-review pass over the working tree.
Transport (_http.py):
- _parse_retry_after: bound the digit-string length before int(). On
Python >= 3.10.7 int() raises ValueError past 4300 digits, and this runs
inside post_form before its own try/except, so a hostile IdP / on-path
proxy could leak a raw ValueError past the module's typed-error contract.
Only a short run of ASCII digits is accepted; anything longer reads as
absent and the caller's fixed back-off applies.
Token lifecycle (_device.py):
- Reject a blank (empty or whitespace-only) served token on both the
network and persisted paths: a run of spaces passed the printable-ASCII
gate and would be cached and sent as "Bearer <spaces>" instead of failing
once with the clear "missing required token" error.
- _normalize_url: re-bracket IPv6 literals to match the on-disk
_canonical_endpoint, so two distinct endpoints ("[::1]:9000" vs
"[::1:9000]") can't collapse to the ambiguous "::1:9000" and share one
in-memory cache entry while staying separate on disk.
Token store (_store.py):
- Refuse a symlinked store-dir leaf (lstat) so the plaintext token files
can't be redirected outside the owner-only directory.
- Treat a future-dated lock mtime as fresh (never steal a possibly-live
lock when our clock reads behind it) and raise the lock-stale floor to
300s for fsync/scheduling headroom above the ~240s network envelope.
Tests / example:
- Regression coverage for all of the above, including proving the
SQLAlchemy per-connection token fetch is non-interactive (raises
OidcInteractionRequired from a pool thread rather than prompting).
- Example: close the psycopg connection via contextlib.closing (psycopg2
leaves it open on __exit__).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The slow-path shared-cache promotion set self._tokens from the shared MemoryCache without moving self._last_persisted_refresh_token, unlike every other adopter (_adopt, _store, clear). That desync made _refresh_under_lock's `refresh_token == _last_persisted_refresh_token` gate misread a peer-rotated adopted token as "our save failed, in-memory is newer than disk" and skip the token-store re-read -- so it refreshed a token a peer had already rotated away (revoked) and re-prompted, while the peer's valid refresh token sat unused on disk. Reachable with a rotating-refresh-token IdP, a FileTokenStore, and multiple OidcDeviceAuth instances sharing the process-global cache across processes; fail-safe (never serves a wrong/expired token) but defeats the silent cross-instance/restart resume the store exists to provide. Adopt the cache token's refresh token into the marker, exactly as _adopt does for a disk load. Adds a regression test that fails without the fix. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
OidcDeviceAuth's docstrings reference :class:`~questdb.auth.Renderer` for the renderer= parameter, but Renderer was not exported, so the -nW (nitpicky, warnings-as-errors) docs build could not resolve the target and broke ./proj doc. Export Renderer from the package (__init__ + __all__) and autodoc it in api.rst, and document its callback contract (untrusted MITM-tamperable input, hold-the-lock/no-reentrancy, best-effort) so custom renderers are a proper public extension point. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Code review —
|
Address the level-3 review findings on the questdb.auth module. Security: - _tokenset_from_persisted: screen BOTH wire-bindable tokens (access_token and id_token) through _safe_token_or_none, not just the served kind, so a control/CRLF char in a non-served persisted token can no longer land in the TokenSet (and be re-persisted verbatim) — symmetric with the network path. - discover_device_endpoint_from_idp: reject a discovery document whose own 'issuer' does not match the pinned issuer (RFC 8414 3.3), trailing-slash- insensitive; absent/non-string tolerated, so it is strictly a tightening. Robustness / nits: - _http: correct the head-read watchdog docstrings to describe the residual open()->release() window honestly (rare, recoverable via a retried OidcNetworkError) instead of overclaiming it cannot occur. - _store.save: close the mkstemp fd if os.fdopen raises before taking ownership; document the orphaned-lock window at the _steal_stale_lock restore site; add docstrings to the FileTokenStore method overrides. - _device._normalize_url: keep an explicit :0 port (compare against None, not truthiness) to match _canonical_endpoint's identity distinctions. - _adapters._coerce_port: reject a non-integral float port instead of silently truncating it through int(). Tests / CI: - Bound the previously-unbounded joins in the in_lock serialization and sqlalchemy per-connect tests and assert is_alive()==False, so a deadlock regression fails cleanly instead of hanging the suite. - ci/pip_install_deps.py: install cryptography so the behavioural TLS-rejection test runs in CI rather than silently skipping. - New regression tests: discovery issuer mismatch/trailing-slash, persisted non-served token screening, and a non-integral float pg_port. Verified each fails without its corresponding source fix. Full suite: 319 pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Code review —
|
…an't diverge _safe_link_url vetted the scheme, userinfo and host of the device-flow verification URL but never read its port, so a URL with a non-integer or out-of-range port (e.g. "https://host:70000/device") was returned verbatim as the clickable href / webbrowser.open() / QR target. _display_url, however, drops a junk port it can't render — so the user would read "https://host/device" while the click / browser / QR went to a different port on that host. That breaks the shown-vs-opened invariant _safe_target exists to hold (the port-stripped display had made the divergence invisible). Read .port inside the existing try so a malformed port raises ValueError and the URL is rejected: it is then shown as inert, port-stripped text and never made clickable/opened/scanned. Valid explicit ports are unaffected. Add a regression test pinning the invariant. Also correct the CHANGELOG: pyproject.toml's authoritative requires-python already declared >=3.10 in the released 4.1.0, so pip has rejected 3.8/3.9 since then; this release only syncs the stale setup.py python_requires. Scope the "nothing written to disk" note to the default (opt-in FileTokenStore persists a refresh token). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ge gaps _request_device_code accepted a user_code / verification_uri that is a non-empty string of only control / zero-width / exotic-space characters: it passed the _str_or_none truthiness guard yet renders empty via _strip_control / _display_url, producing an "Open and enter code:" prompt with nothing to act on. Gate the two DISPLAYED fields on their post-strip value (via _verification_uri, exactly what the renderer shows) so such a response is rejected as non-conformant. device_code keeps the raw check (it is sent, not shown); user_code keeps _str_or_none first so a JSON number can't be coerced visible by _strip_control's str() fallback. Also close review-flagged test gaps: - strengthen the two clear() concurrency stress tests: seed a token whose id_token is DISTINCT from the mock-minted one, so a served value tells a stale cache-hit from a genuine re-acquisition — the seed == issued value made `token() != ID_TOKEN` a tautology a broken shared-cache CAS could pass; - FileTokenStore: a mid-write failure (fdopen/fsync/rename) cleans up its .tmp and raises; load() errno routing (ELOOP/ENOTDIR -> None, EACCES/EIO -> raise); a non-string file audience/issuer is rejected; a control-char refresh_token is kept (only url-encoded to the IdP, never a header); a real cross-PROCESS save/load round-trip over the on-disk format; - TerminalRenderer.on_waiting renders the MM:SS countdown; - import-check examples/oidc_device_auth.py (not in examples.manifest.yaml). Reword the test.py comment: the auth tests are pure-Python and run standalone, but test.py imports questdb.ingress first, so the aggregated CI run builds the extension. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…-prompt
Two edge cases found while reviewing the OIDC device-flow module.
- _validate_positive_number interpolated the offending value with {!r}.
repr() on an int with >4300 digits itself raises ValueError (CPython's
int->str limit, active on the 3.10 floor), so a huge-int timeout /
default_interval escaped the module's typed-error contract as a bare
ValueError instead of OidcConfigError. Fall back to a type description
when repr() can't run.
- When a silent refresh was exhausted, _acquire evicted the shared-cache
entry and ran the device flow unconditionally. A peer OidcDeviceAuth
sharing the process-global cache may have stored a valid token for this
identity during our refresh; evicting it and prompting was a needless
re-prompt. Re-read the shared cache before evicting and adopt a valid
peer token, syncing _last_persisted_refresh_token as the promotion path
does. (Checking after the evict would not work — evict() drops the peer
token first, so the re-read has to precede it.)
Both are covered by regression tests that fail without the fix.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…tests Minor findings from the review round (the Moderate fixes landed in the previous commit). - FileTokenStore.clear() now sweeps orphaned `<hash>*.tmp` siblings. save() writes the plaintext token into a temp before its atomic rename; a hard crash in that window left it behind (0600, never read back) and clear() removed only the .json, so a forgotten credential lingered on disk. - _is_stale() uses lstat, so a symlink planted at the lock path is judged by the link's own mtime rather than a target a co-tenant keeps fresh; the residual future-dated-mtime case is documented as integrity-safe. - Doc accuracy: scope "auto-refreshed" to sqlalchemy_engine (psycopg_connect captures the token once at connect time); document the adapters' host / pg_port / database params and OidcConfigError; correct FileTokenStore.load "never raises" (it raises on a hard I/O error); frame the 0600 file mode (not the 0700 directory) as the content protection; note the cross-process clear-vs-save race is not limited to the lock-failed path. - Tests: _pg_module v3/v2/none selection order, an invalid-UTF-8 HTTP body via json(), a FIFO at the token path, and the groups_in_token=False refresh fall-back (mirroring the existing groups=True test). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two credential-path hardening fixes found in review.
PG host guard (_adapters.py): replace the deny-list of conninfo
metacharacters with a positive allow-list of the characters a real
hostname / IPv4 / IPv6-literal can contain. The deny-list missed ',' —
a libpq MULTI-HOST separator ('host=a,b' tries both), so a tampered URL
could steer the connection, and the '_sso' token sent as the password,
to an attacker host that merely reads next to the real one — and '/',
which libpq treats as a Unix-socket directory. The allow-list closes the
whole redirection/injection class at once. Underscore stays allowed,
matching _render._SAFE_HOST_RE.
HTTP body read (_http.py): read1() (used for the chunked-dribble
watchdog) does not enforce Content-Length — on a body that declares N
bytes but delivers fewer then closes, it returns the short data then a
clean EOF, so a truncated (yet still JSON-parseable) token / config
response was handed back as a complete 200. http.client leaves the owed
count on resp.length, so treat a truthy length at EOF as a truncation
and raise OidcNetworkError. Chunked bodies (length None) and non
-http.client readers (no length attr) are unaffected.
Add regression tests for both (comma/unix-socket/userinfo host
rejection; truncated Content-Length via a unit stub and a real socket)
and update the cross-reference comments to the renamed _LEGAL_HOST_RE.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
resolve_config tracked endpoint provenance by identity (`is not None`)
but chose the value by truthiness (`or _resolve_endpoint(...)`), so an
empty-string override — a common "unset" sentinel like
token_endpoint=os.environ.get("TOK", "") — was stamped caller-explicit
(trusted) while its value came from the untrusted /settings response.
That false provenance skipped the plaintext-channel guard and both
issuer pins, letting a tampered /settings route the device code and
refresh token to an attacker origin. Normalize empty->None up front so
an empty override behaves exactly like an omitted one.
FileTokenStore.load() stat'd the path then did a blocking open(): a
co-tenant swapping the regular file for a FIFO in that window hung the
open forever, pinning the acquisition-lock-holding thread. Open
O_NONBLOCK and re-validate the opened fd with fstat, closing the TOCTOU.
_try_refresh_coordinated's post-failure fall-through re-refreshed with
the stale tokens argument; a custom TokenStore whose in_lock raised
after its action already rotated the refresh token would replay the
spent token and trip the IdP's reuse detection. Re-consult the freshest
in-memory token first, and document the in_lock contract OidcDeviceAuth
relies on. The bundled FileTokenStore was already immune to both.
Adds regression tests for all three.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…mation M1: a non-string issuer= reached urlparse (via resolve_config's authority vetting and the discovery-URL build) before OidcDeviceAuth.__init__ could type-check it, so from_questdb leaked a raw AttributeError/TypeError instead of an OidcError, escaping the module's typed-error contract. safe_urlparse now maps TypeError/AttributeError too (also covering a non-string QuestDB URL), and resolve_config type-checks issuer early with the same clear message the direct constructor already raises. M2: the /settings-vs-IdP-discovery endpoint confirmation used raw string equality, so a legitimate split-origin IdP (Google/Auth0/Azure) whose two sources spelled one endpoint with a trailing slash, an explicit default port, or a case difference was wrongly rejected by the issuer-origin pin. Compare on the canonical endpoint form instead (reusing _canonical_endpoint, the same canonicalization the cache key and store key use), so a differing query stays a distinct routing target and an unparseable value fails closed rather than raising. Adds regression tests for both. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… marks
m1 (RFC 8628 §3.5): a non-conformant `429 {"error":"slow_down"}` was handled by
the transient 429/5xx arm, which runs before the dedicated slow_down arm, so a
low Retry-After made the poll interval DROP right after the IdP asked to slow
down. The 429/5xx arm now passes at_least_increment when the body carries
slow_down, so the interval still increases by at least 5 (floor-bounded either
way); a plain 429 keeps honoring Retry-After verbatim.
m2: groups_in_token reached the OidcDeviceAuth constructor un-coerced. A truthy
non-bool (e.g. 2, from an env read without a cast) is used truthily in memory,
but the store hashed the file as groups=1 while _parse_and_verify compared the
raw value, so a persisted entry failed its OWN reload (`True != 2`) and
re-prompted every restart. Coerce it to bool in __init__ (mirroring
from_questdb), and make the store self-consistent for a direct TokenStoreKey:
_serialize writes a boolean and _parse_and_verify compares bool-to-bool.
m3: seven invisible Default_Ignorable non-spacing marks (category Mn) — the
combining grapheme joiner (U+034F), the Mongolian free variation selectors
(U+180B-U+180D, U+180F) and the Khmer inherent vowels (U+17B4, U+17B5) —
survived _strip_control's "keep accents" rule and could hide payload in a
user_code / identity / URL, like the FE00-FE0F variation selectors already
stripped. Add them to _STRIP_EXTRA.
Adds regression tests for all three.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Code review —
|
Summary
New
questdb.authmodule that lets you sign in interactively to OIDC-securedQuestDB Enterprise from Python — including from remote kernels (JupyterHub,
SageMaker, Colab, VS Code-remote, containers) that have no local browser.
It runs the OAuth 2.0 Device Authorization Grant (RFC 8628)
entirely client-side: you authorize in any browser (laptop or phone), while the
kernel only makes outbound calls to your identity provider (IdP). The resulting
token is presented to QuestDB over the auth paths it already supports — HTTP
Authorization: Beareror PG-wire_sso— so no server change is required.Pure Python, built on the standard library (
urllib);questdb.ingressisnever imported at module load, and all optional deps are lazy.
Two ways to use it
Just the token — no extra dependencies; works with PG-wire, raw HTTP, the
ingestion
Sender, or any client:On first use you get a sign-in prompt (clickable link in Jupyter, plain text on
a terminal). Re-running is silent: the token is cached and refreshed silently as
it nears expiry.
PG-wire adapters — feed the auto-refreshed token in as the
_ssopassword(require
acl.oidc.pg.token.as.password.enabled=trueon the server):For REST (
Authorization: Bearer) and ingestion (the ILPSender), there is nodedicated adapter — take
auth.headers()/auth.token()and wire it inyourself, e.g.
Sender.from_conf("https::addr=…;", token=auth.token()).Highlights
/settingsendpoint (acl.oidc.*),falling back to the IdP
.well-known/openid-configuration. Anything passedexplicitly overrides discovery; discovery can also be skipped entirely.
(
groupsEncodedInToken ? id_token : access_token), requesting theopenidscope automatically when the id_token is sent. A completed grant that is
missing the required token kind fails once with a clear error rather than
caching an unusable token.
refresh_token; re-prompts only when the refresh token is missing/rejected.A transient failure — a network blip or an IdP
5xx/429, on a poll or arefresh — is surfaced and retried, not re-prompted, while a terminal IdP
rejection fails fast with a clear error. A lock serializes refresh so parallel
cells/threads don't double-prompt. The cache is an always-on, process-global
in-memory store: re-running a notebook cell reuses the token, a kernel
restart re-prompts once. By default nothing is written to disk; pass
token_store=FileTokenStore.at_default_location()to opt into persistenceso a restarted process resumes from a saved refresh token — an owner-only
(
0600) plaintext file per identity under~/.questdb/oidc-tokens/— insteadof re-prompting.
the auto-refreshed token as the
_ssopassword (sqlalchemy_enginere-supplies a fresh token on every new pooled connection). That per-connection
injection is non-interactive — it reuses / silently refreshes the cached
token but refuses to start the device flow from a pool thread (raising
OidcInteractionRequiredrather than blocking the pool on a browser prompt),so you sign in once up front. REST and the ingestion
Senderarebring-your-own-client via
headers()/token().OidcInteractionRequiredinstead ofhanging under papermill / cron / CI.
qrcode) with aplain-text terminal fallback that degrades gracefully on a non-UTF-8 console
(the verification URL and code always print, even if the emoji can't be
encoded — it no longer silently swallows the whole prompt). A device-
authorization response missing the verification URI is rejected rather than
rendering a blank prompt, and the "expires in N min" message reflects the
token's real lifetime (its JWT
exp), not the cache's re-validation clamp.OidcErrorsubclass(
OidcConfigError,OidcNetworkError,OidcInteractionRequired,OidcDeviceFlowError,OidcTimeoutError) — malformed config, HTTP/URL errorsand bad server payloads are mapped to these rather than leaking raw
ValueError/AttributeError/http.clientexceptions. Even a hostilenon-string
error/error_descriptionin an IdP response is coerced, so itcan't escape the contract as a raw
TypeError.Security
httpsis required; plaintext is allowed only to a loopback address.insecure=Truepermits plaintext to a non-loopback QuestDB host only — itnever downgrades the IdP (so the device code and refresh token are never
sent in cleartext) and never disables certificate verification.
origin, and HTTP redirects are refused — urllib doesn't strip the
Authorizationheader across a cross-origin30x, and the origin check onlysees the pre-redirect URL, so a silently-followed redirect could leak the token
or refresh token. A
30xon those endpoints surfaces as an error instead (andredirect-refusal is pinned on the HTTPS opener, the production path that
carries the bearer / refresh token).
issuer=/discovery_url=to pin the IdP. Thediscovery origin is never derived from a server-supplied token endpoint, so a
tampered
/settingscan't redirect the device-code / refresh-token POSTs. Thepin is required before trusting credential endpoints that arrive over an
untrusted plaintext
/settings(only reachable withinsecure=True), andbefore discovering endpoints from the IdP. On a path-based multi-tenant IdP
(e.g. Keycloak
…/realms/{realm}) it also scopes endpoints to the issuerpath, rejecting
..traversal even when hidden by percent-/double-encoding,a backslash, or a matrix
;param./settingsconfig is trusted — discovery readsthe nested
configobject and ignores the user-writablepreferencessibling(which the web console persists via
PUT /settings), so a user who can write apreference can't smuggle an
acl.oidc.*override (e.g. a redirected tokenendpoint) into the resolved config.
expires_in/intervalareclamped (sane lifetime cap, interval in
[5s, 60s], never sleeps past thedeadline), so a hostile or buggy IdP can't time the flow out before the first
poll or stall the loop while it holds the acquisition lock. Transient poll
failures (network, IdP
5xx/429, or a non-JSON proxy error) keep polling tothe deadline; a terminal rejection — including a non-JSON
4xxfrom a WAF orproxy — stops immediately rather than polling on to a misleading timeout. A
plain
429/5xxhonours the IdP'sRetry-Afterwhen present — even on anon-JSON body from a proxy/WAF — else the RFC 8628
+5sstep, both clamped tothe poll-interval bounds; a
slow_downonly ever increases the interval, soit never polls faster right after the IdP asked it to slow down (RFC 8628 §3.5).
and bidi/zero-width characters are stripped from untrusted device-response
fields (
verification_uri,user_code, IdPerror_description, JWT-derivedidentity) before they reach the TTY or notebook, so a MITM'd response can't
inject ANSI escapes or reorder a URL's host. The verification link is shown
with its host in IDNA/punycode form, and is only made clickable / opened in
a browser / encoded as a QR when that host is a plain ASCII name — so a
homoglyph dot (a fullwidth
.that IDNA folds to a real., resolving to adifferent registrable domain) or an embedded
user@hostcan't masquerade asthe trusted host, and the link shown, the URL opened, and the QR all use one
vetted target so they can't diverge. The Jupyter renderer additionally
HTML-escapes its output and scheme-vets the link (no
javascript:/data:).TokenSetis immutable (frozen) and keepsthe access/id/refresh tokens (and the
sub) out of itsrepr, so a tokencan't leak into a log line or traceback. The PG adapters avoid logging the
token / DSN and validate the PG-wire host and port: a host carrying
connection-string metacharacters (
;/=/ whitespace / control / an IPv6zone-id
%) or a non-integer port is rejected with a typed error, so atampered URL can't inject libpq conninfo parameters.
HTTPS_PROXY,REQUESTS_CA_BUNDLE,SSL_CERT_FILE, andca_bundle=.Dependencies & Python support
token()/headers()need nothing beyond the standard library.sqlalchemy,psycopg/psycopg2,qrcodeandIPythonare imported lazily, only whenused. The declared minimum Python is corrected to 3.10 (
setup.py>=3.8→
>=3.10,docs/installation.rst>=3.9→>=3.10), matching the floor4.1.0 already adopted when it dropped Python 3.9; stale 3.8 references are
removed (
RELEASING.rst,proj.py).Docs, examples & tests
docs/auth.rst(:ref:oidc_auth), API reference indocs/api.rst(
OidcDeviceAuth, the two PG-wire adapters,OidcConfig,TokenSet, and thetyped-error classes), index/installation updates, and a
CHANGELOG.rstentry.examples/oidc_device_auth.py.test/test_auth.py— pure-Python (no compiled extension required), wired intothe suite via
test/test.py. Covers the device flow, refresh, non-interactivecontexts, discovery, the insecure-
/settingsguard, PG adapters (host/portvalidation), config helpers, endpoint validation, cache-key derivation, the
HTTPS redirect-refusal path, multi-threaded and cross-instance cache-contention
stress tests, and transport/renderer security (control/bidi/homoglyph/userinfo
handling).
Also in this PR (CI & build robustness)
Bundled fixes to keep the matrix green, independent of the auth feature:
test_parquet_roundtripinstead of hardcoding
object(pandas 3 reads back the new string dtype); keep32-bit wheel targets on the consistent pandas 2 / numpy 1 stack (pandas 3 ships
no 32-bit wheels) so the dataframe tests actually run there.
-SNAPSHOTjava client via thelocal-clientMaven profile (newci/templates/detect-local-client.yml), invoke Maven directly (the Maven taskcan't parse JDK 25), and pass the JPMS module-access flags the server needs.
(narrowed to the specific broken-pipe / connection-reset / connection-aborted
errors, not all
ConnectionError), and skip the readonlyAZP_ENHANCED*agentvar in the wheel-build env re-export.
review-prreview skill (.claude/skills/review-pr/).