Skip to content

Fix TEI stale socket retries - #3375

Open
oldnicke wants to merge 1 commit into
vectorize-io:mainfrom
oldnicke:fix/3284-tei-stale-socket
Open

Fix TEI stale socket retries#3375
oldnicke wants to merge 1 commit into
vectorize-io:mainfrom
oldnicke:fix/3284-tei-stale-socket

Conversation

@oldnicke

Copy link
Copy Markdown
Contributor

Summary

Fixes #3284.

Remote TEI embedding requests can intermittently fail after idle periods when the pooled HTTP client reuses a socket that has died underneath it. This change makes TEI transport handling more resilient by:

  • retrying retryable transport-layer OSErrors such as EBADF within the existing TEI retry budget,
  • setting an explicit short keepalive expiry on TEI HTTP clients so idle pooled sockets are recycled sooner, and
  • applying the shared TEI transport retry handling to both embeddings and the TEI reranker client.

Tests

  • uv run pytest hindsight-api-slim/tests/test_tei_embeddings.py hindsight-api-slim/tests/test_tei_retry.py -q
  • uv run pytest hindsight-api-slim/tests/test_tei_cross_encoder.py -q
  • ./scripts/hooks/lint.sh

Notes

  • No API schema changes.
  • The regression test covers a first-request EBADF followed by a successful retry.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

tei: intermittent 'Bad file descriptor' on embedding requests — stale pooled connections lack health checks

1 participant