Do hosted open models support tool calling reliably? How to test one in five minutes
Updated 13 November 2026
A coding agent is a loop of tool calls. The model asks to read a file, run a command or edit a line, the agent does it, and the result goes back. If the model gets a tool name wrong or sends arguments that aren't valid JSON, that turn fails, and an agent makes dozens of turns a task. So tool calling matters more for agents than almost any other model feature.
What we know and what we haven't measured
NeverMetered's API is OpenAI- and Anthropic-compatible, and coding tools that rely on tool calling work through it: Claude Code, Codex CLI, OpenCode and others have setup guides. Qwen's model card reports agentic coding scores that depend on the model calling tools correctly, with the Claude Code harness. The card doesn't document a tool-call format or a success rate, and we haven't published one of our own.
What failure looks like
- The model answers in prose when it should have called a tool, so the agent has nothing to run.
- It calls a tool that doesn't exist, or the wrong one for the job.
- The arguments aren't valid JSON, or a required field is missing.
- It repeats the same call after the result came back, which is the start of a loop. See why agent bills get so high.
A five-minute test
This sends the same tool-calling request 10 times and counts how many replies call the right tool with valid JSON arguments. It needs curl and jq, and it uses 10 requests, so check that your plan's allowance covers it. The request has one tool, read_file, and asks the model to open a file.
- 1
Save the request
Put this in a file called
request.json.request.json{ "model": "qwen3.8-27b", "max_tokens": 512, "messages": [ {"role": "user", "content": "Open the file src/index.ts so I can see it."} ], "tools": [ { "type": "function", "function": { "name": "read_file", "description": "Read a file from the project and return its contents.", "parameters": { "type": "object", "properties": {"path": {"type": "string", "description": "Path relative to the project root"}}, "required": ["path"] } } } ] } - 2
Run it 10 times
Set your key in
NEVERMETERED_API_KEYfirst. The test passes when the first tool call isread_fileand its arguments parse as JSON with apath.for i in $(seq 1 10); do curl -s https://api.nevermetered.com/v1/chat/completions \ -H "Authorization: Bearer $NEVERMETERED_API_KEY" \ -H "Content-Type: application/json" \ -d @request.json \ | jq -e '.choices[0].message.tool_calls[0].function | .name == "read_file" and (.arguments | fromjson | has("path"))' >/dev/null \ && echo pass || echo fail done | sort | uniq -c - 3
Read the result
A line like
10 passmeans every reply was usable. Anyfaillines are worth looking into: rerun one request without thejqfilter and read the raw reply to see whether the model answered in prose, picked another tool or sent broken arguments.
We ran exactly this script against the live endpoint on 13 November 2026 and got 10 passes out of 10. That's a small sample of the easiest case, so it shows the basics work and nothing more. Make the test harder before you trust it. Add three or four tools, give a task that needs two of them in sequence, and use the tool definitions your own agent sends. A real agent session is the best test of all, and the free plan is enough to run one.
If calls are failing
- Check the output cap first. A reply that's cut off mid-arguments is invalid JSON. See the truncated file writes guide.
- Keep tool descriptions short and specific, and keep the number of tools small. Models choose better from five clear tools than from fifty vague ones.
- Look at the Requests page for the failing call's finish reason. See reading your usage and requests.
- If your tool lets you set a temperature, use the vendor's recommended value for the model rather than a low one you chose for a different model.
Does Qwen3.8-27B support function calling?
Is tool calling worse on open models than on Claude or GPT?
Will my agent stop if one tool call fails?
- Qwen3.8-27B model card (Hugging Face) (checked 13 November 2026)
- How NeverMetered runs (checked 13 November 2026)
- NeverMetered pricing and plan limits (checked 13 November 2026)