Add tool call parser for MiniCPM5 - #1899
Open
dasashreeya wants to merge 1 commit into
Open
dasashreeya wants to merge 1 commit into
dasashreeya wants to merge 1 commit into
Conversation
MiniCPM5's chat template asks for <function name="fn"><param name="p">value</param></function> with no outer wrapper, which _infer_tool_parser did not recognize, so mlx_lm.server warned that the model does not support tool calling and returned the raw XML as content. Add a parser for that format (CDATA values, single or double quotes, parallel calls as consecutive blocks) and detect it from the template.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1885
openbmb/MiniCPM5-1B-MLXships a chat template that asks the model for<function name="fn"><param name="p">value</param></function>with no outer wrapper._infer_tool_parserintokenizer_utils.pypicks a parser by substring-matching the template and has no branch for that format, sohas_tool_callingis False, the server logs "Received tools but model does not support tool calling", and the raw XML comes back incontentwithfinish_reason: "stop".mlx_lm.server --model openbmb/MiniCPM5-1B-MLX # any /v1/chat/completions request with `tools`On main: the warning,
tool_calls: null, and<function name="get_weather"><param name="city">Paris</param>…</function>incontent. With this change:finish_reason: "tool_calls"andtool_calls: [{"name": "get_weather", "arguments": "{\"city\": \"Paris\", \"unit\": \"c\"}"}], in both streaming and non-streaming mode. Parallel calls (consecutive<function>blocks) come back as two entries, thinking mode still separatesreasoningfrom the call, and a tool-result round trip renders the assistanttool_callshistory correctly through the template.The parser sets
tool_call_start = "<function name="andtool_call_end = "</function>", unwraps<![CDATA[...]]>values (the template's escape for<,&and newlines), accepts single or double quotes, and types arguments from the tool schema using the helpers already inminimax_m2(same pattern aslagunareusingglm47). Detection is'<function name="' in chat_template and '<param name="' in chat_template; no other template in the repo's parsers uses those markers.Tests:
minicpm5added to the two generic cases intest_tool_parsing.py, plustest_minicpm5covering the marker-stripped text the server passes, quotes and whitespace, CDATA, no schema, parallel blocks, output truncated bymax_tokens, no params, empty values, and the no-call error. They fail on main (the module does not exist) and pass here; the full suite passes (312 passed, 1 skipped) and pre-commit is clean.Per CONTRIBUTING: I directed the investigation, code, and tests using Claude Code and verified the behavior end-to-end against the model on my M4 Pro. I reviewed the diff and take responsibility for every line.