Repository navigation
Surface tool-definition changes for servers in the registry #6734
Description
Activity
On item 2, the comparison between the current tool list and the approved one needs a preimage both sides compute the same way, or two honest records disagree on whether anything changed. One is pinned with test vectors: each served tool restricted to name, title, description, inputSchema, outputSchema and annotations, RFC 8785 canonicalized under a profile label, SHA-256, keyed by tool name, with a manifest digest folded over the per-tool digests. https://github.com/AgentAvow/AgentAvow/tree/main/docs/standards/tool-manifest-digest-vectors-v1
The fixture pins an unmodified tools/list, so a check script can be tested against known bytes, and per-tool keys mean an admin sees which tool changed, not only that something did. Free to use as the oracle for the admin flag. A sealed daily record like heldfast's gives the history; a shared preimage is what makes two records comparable byte for byte.
Thanks. Two notes. Heldfast's lock already keys digests by tool name, so an admin sees which tool changed, and its preimage already covers name, title, description, inputSchema, outputSchema and annotations (plus icons when present). The difference is encoding: I ran your three DeepWiki fixtures through both, your derivation reproduces your signed digests, and heldfast's differ for the same tools (snake_case keys, defaults for missing fields, no profile label). So for the admin flag, ToolHive only needs one consistent digest of its own. A shared preimage matters if you also want to look a server up in an external record, and if that is useful I can publish a second digest under whichever profile you name next to the native one.
Yes, that would be useful, and thanks for running both. Agreed that ToolHive needs only one consistent digest for its own flag. The shared one earns its keep across records: with a second digest under
agentavow.mcp-tool-definition.v1next to your native one, a drift in your daily record and a signed AgentAvow grade can be checked against each other without trusting either. The v1 set now also carries thirteen key-encoding pairs (percent-encoding and the length cut), which a second implementation passed 60/60 last night; if your run of those and the three digests agrees, I'd count it as an independent implementation and say so on the vectors' README.One request: your repo's licence is all-rights-reserved with no carve-out for docs/standards/tool-manifest-digest-vectors-v1. Could you add a LICENSE file to that folder (CC0 or Apache-2.0)? Until then I'll run your vectors from a local checkout without copying them. A permissive licence also lets other implementers vendor the fixture.
Done: https://github.com/rufat325/heldfast/blob/main/docs/TRANSPARENCY.md#comparing-with-another-record. heldfast reproduces your published digests and key encoding for the pinned fixture (3 of 3 digests, 13 of 13 key pairs, as of 2 October 2026), from our own implementation of the rules, with none of your files copied. It makes no claim about your grades. Per your offer, please word any README mention that narrowly. One change from what I said: I am not storing the profile digest in every catalogue entry. It would add roughly 0.6 GB a year of permanent history to a sealed repository, and it can be recomputed from the definitions already in tools/ (research/feed/profile_index.py does it, and I found no native digest that maps to two profile digests).
Licence added: both vector folders (
docs/standards/tool-manifest-digest-vectors-v0and-v1) are Apache-2.0 as of AgentAvow/AgentAvowca6b5551, under the repository licence's subdirectory carve-out, so vendoring the fixture is fine now.Thanks for the TRANSPARENCY.md run. The README mention is worded as you asked, in the "Separate implementations" section of the v1 README: heldfast reproduces the per-tool digests and the key encoding from its own implementation and makes no claim about AgentAvow's grades.
Thanks for the question, @rufat325. Whether an admin gets told when a registry server changes its tool descriptions or schemas is a fair thing to ask.
We have decided not to build this around a third-party record or feed. We would rather keep the project free of integrations tied to specific external services or digest formats, so we are closing this one rather than leaving it open as a place to compare them.
Thanks for taking the time to write this up. Closing this one out.
ToolHive's README says it gives security teams audit logs and lets enterprises self-host their MCP registry. When a server in that registry changes its tool descriptions or schemas, whether between versions or in place on a remote server, does ToolHive tell an admin? I couldn't find it. If it exists, please point me to it.
If not, here is a concrete proposal. heldfast (Apache-2.0) keeps an open, sealed daily record of the tool definitions returned by 29,000+ servers in the official MCP registry, hosted and npm, with each change graded quiet or review: https://github.com/rufat325/heldfast/blob/v0.2.3/docs/FIRST-WEEK.md
I'll send a PR for either one.
For Stacklok's enterprise customers: a supported daily feed with a webhook for the servers they've approved, free for 30 days. If that would be worth paying for, tell me what you'd need.