Can an open model replace Claude for coding? Refactors, agents and where it falls short
Updated 13 November 2026
For a lot of everyday coding work, yes. For the hardest problems, not yet, and it depends on the task more than on the model's name. Here's what the published evidence supports, where an open model like Qwen3.8-27B tends to struggle, and how to decide what to hand it.
What the evidence supports
Qwen's model card reports 61.7% on SWE-bench Pro, 73.0% on Terminal Bench 2.1 and 90.3% on LiveCodeBench v6, with the agent tests run through the Claude Code harness. On the same card Opus 4.6 Max scores higher on Terminal Bench (78.2) and NL2Repo-Bench (47.6 against 42.3), and lower on LiveCodeBench (88.8). So a 27-billion-parameter open model is within reach of Opus 4.6 Max, an older frontier model, on the agentic coding tests that Qwen chose to publish. That's a real result, and it's also a vendor's own claim about its own model, measured on the full-precision build rather than the quantized one we serve. See the model page for the full table.
What it doesn't show is how the model behaves on your codebase, on tasks it wasn't tuned for, or against the newest frontier models. For that you have to try it.
Where an open model needs more help
- The hardest, longest problems, where the best frontier models still lead. The model page says the same.
- Vague tasks. A frontier model is more likely to infer what you meant from a one-line request. A smaller model does better with a clear, scoped instruction.
- Speed. Generation on a flat plan is steadier than it is fast. How it runs has the measured figures.
- Images. NeverMetered's model is text only, so a screenshot in the prompt won't work.
Which tasks to route where
Running a large refactor with an open model
A refactor is hard for any model because the change has to stay consistent across many files, and one inconsistency breaks the build. The published repo-level score is the closest thing to a measure: NL2Repo-Bench measures repo-level code generation, and Qwen's card has it at 42.3 against Opus's 47.6. That's a decent result and still below the frontier model, so give the model help rather than a single big instruction.
- 1
Commit before you start
A clean git state means any bad run is one
git checkoutfrom gone. Work on a branch. - 2
Ask for a plan first
Have the agent list the files it intends to change and why, then read it. Correcting a plan costs nothing. Correcting 40 edited files does.
- 3
Work in small slices
One module or one pattern at a time, not the whole codebase. Each slice should end with the tests running.
- 4
Let the tests judge
Tell the agent to run the test suite after each slice and fix failures before moving on. A model that can read real test output corrects itself well.
- 5
Review the diff yourself
Skim every slice's diff before the next one. Mistakes that compile and pass are the ones that survive.
If a slice keeps failing in the same way, that's the moment to hand that one task to a frontier model and go back to the flat plan after. It's also why a long refactor suits a flat rate: the retries and re-reads that make it expensive per token don't cost extra. See why agent bills get so high.
Can Qwen3.8-27B handle multi-file edits?
Is an open model good enough for production code?
Should I stop using Claude entirely?
- Qwen3.8-27B model card (Hugging Face) (checked 13 November 2026)
- How NeverMetered runs (checked 13 November 2026)
- NeverMetered pricing and plan limits (checked 13 November 2026)