Skip to content

Non-finite float metadata (inf/nan) is silently accepted and then dropped, losing metadata entries #7751

Description

@warter666

What happened?

validate_metadata() accepts any Python float, including inf and nan. These values pass client-side validation, but they are not JSON-serializable, so they are silently dropped during persistence/serialization.

Consequences:

  1. Silent metadata loss — adding {"score": float("inf"), "source": "a"} persists source but silently drops score; adding {"score": float("inf")} as the only key returns metadatas: [None] for that record. The data is gone after a client restart too.
  2. Inconsistent validation surface$gt/$lt filters against the dropped key silently miss the record, which is confusing to debug because the write appeared to succeed.

Reproduction (chromadb 1.5.9, PersistentClient)

import chromadb, tempfile

pc = chromadb.PersistentClient(path=tempfile.mkdtemp())
c = pc.get_or_create_collection("t")
c.add(ids=["1"], embeddings=[[0.1, 0.2]],
      metadatas=[{"score": float("inf"), "source": "doc-a"}])
print(c.get(ids=["1"])["metadatas"])
# [{'source': 'doc-a'}]   <- 'score' silently gone

Same for nan, for the update() path (validate_update_metadata), and for floats inside list-valued metadata (_validate_metadata_list_value).

Expected behavior

Writes containing non-finite floats should fail loudly with a ValueError at validation time, mirroring how nested dicts are already rejected. (Alternatively: document + sanitize, but rejecting seems consistent with the existing strictness.)

Happy to open a PR with the fix + tests if the approach sounds good.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions