What happened?
validate_metadata() accepts any Python float, including inf and nan. These values pass client-side validation, but they are not JSON-serializable, so they are silently dropped during persistence/serialization.
Consequences:
- Silent metadata loss — adding
{"score": float("inf"), "source": "a"} persists source but silently drops score; adding {"score": float("inf")} as the only key returns metadatas: [None] for that record. The data is gone after a client restart too.
- Inconsistent validation surface —
$gt/$lt filters against the dropped key silently miss the record, which is confusing to debug because the write appeared to succeed.
Reproduction (chromadb 1.5.9, PersistentClient)
import chromadb, tempfile
pc = chromadb.PersistentClient(path=tempfile.mkdtemp())
c = pc.get_or_create_collection("t")
c.add(ids=["1"], embeddings=[[0.1, 0.2]],
metadatas=[{"score": float("inf"), "source": "doc-a"}])
print(c.get(ids=["1"])["metadatas"])
# [{'source': 'doc-a'}] <- 'score' silently gone
Same for nan, for the update() path (validate_update_metadata), and for floats inside list-valued metadata (_validate_metadata_list_value).
Expected behavior
Writes containing non-finite floats should fail loudly with a ValueError at validation time, mirroring how nested dicts are already rejected. (Alternatively: document + sanitize, but rejecting seems consistent with the existing strictness.)
Happy to open a PR with the fix + tests if the approach sounds good.
What happened?
validate_metadata()accepts any Pythonfloat, includinginfandnan. These values pass client-side validation, but they are not JSON-serializable, so they are silently dropped during persistence/serialization.Consequences:
{"score": float("inf"), "source": "a"}persistssourcebut silently dropsscore; adding{"score": float("inf")}as the only key returnsmetadatas: [None]for that record. The data is gone after a client restart too.$gt/$ltfilters against the dropped key silently miss the record, which is confusing to debug because the write appeared to succeed.Reproduction (chromadb 1.5.9, PersistentClient)
Same for
nan, for theupdate()path (validate_update_metadata), and for floats inside list-valued metadata (_validate_metadata_list_value).Expected behavior
Writes containing non-finite floats should fail loudly with a
ValueErrorat validation time, mirroring how nested dicts are already rejected. (Alternatively: document + sanitize, but rejecting seems consistent with the existing strictness.)Happy to open a PR with the fix + tests if the approach sounds good.