Local Tool-Calling Models — Complete Visual Guide

A real head-to-head test, the current open-weight benchmark landscape, and what actually fits on consumer hardware · August 2026

Head-to-Head · Today’s Test

Qwen3.5-4B vs Llama-3.1-8B — A Real Head-to-Head Test

While running opensre (an AI-SRE investigation tool) locally through oMLX — a macOS-native MLX inference server — on two Mac Minis, the same real Kubernetes investigation task was pointed at two different open-weight instruct models in turn. Same oMLX server, same tool schemas, same alert. Only the model changed. The outcome was not close, and it was not the one the specs would predict.

Same task, same tools, same server — two very different runs

Qwen3.5-4B — full ReAct loop, correct grounded diagnosis Llama-3.1-8B — no tool calls, invalid enum output, no diagnosis

✅ Qwen3.5-4B-MLX-8bit

FamilyAlibaba Qwen, general instruct

Size / quant4B params · 8-bit · ~5GB

Total time872s

Tool calls made6 distinct real calls

Evidence gathered17 items

Final validity score1.0

Correctly identified an ImagePullBackOff caused by a missing imagePullSecrets configuration — verified by cross-referencing the actual Kubernetes API error text, and confirmed with real differential diagnosis: comparing a healthy pod that had imagePullSecrets configured against the broken pods that did not.

correct & fully grounded

❌ Meta-Llama-3.1-8B-Instruct-8bit

FamilyMeta Llama, general instruct

Size / quant8B params (bigger) · 8-bit · ~8.3GB

Total time172s (5x faster)

Tool calls made0

Evidence gathered0 items

Final result”Unable to determine root cause”

Skipped the ReAct tool-calling loop entirely and tried to answer straight from the alert text. Even that failed: it picked an invalid value for a closed-enum schema field — a literal_error Pydantic structured-output validation failure — on a large multi-dozen-value enum.

no tool calls, wrong

Benchmark Landscape

The Tool-Calling & Agentic Coding Benchmark Landscape

Today’s local test compared two general-purpose instruct models on one task. The broader open-weight field also publishes dedicated tool-calling and agentic-coding benchmark numbers — a different, larger set of models, some purpose-tuned for exactly this behavior. Every figure below is attributed; full links are in Tab 3.

Function-Calling / Tool-Use Benchmarks

Model	BFCL v3 Score	Tool-Use Success Rate
GLM-4.5	77.8%	90.6%
Qwen3	71.9%	—
Kimi K2	71.1%	86.2%
Qwen3-Coder	—	77.1%

BFCL v3 is the Berkeley Function-Calling Leaderboard v3. GLM-4.5 leads both measures reported here; note the tool-use success rate for the Qwen family is reported against the Qwen3-Coder variant specifically, not the base Qwen3 BFCL number, so treat those two Qwen rows as related but distinct model variants rather than one apples-to-apples line. source 1

Coding-Specific Benchmarks

Model	Headline Score	Params	License	Notes
Qwen3-Coder-480B	69.6% SWE-bench Verified	480B	Apache 2.0	Supports 358 programming languages
DeepSeek-V3.2	~70% SWE-bench Verified	685B total	MIT	128K context, best price-performance
Kimi K2.6	58.6% SWE-bench Pro / 66.7% Terminal-Bench 2.0	—	—	Sustained 4,000+ tool calls over a 13-hour autonomous session
Kimi K3	LiveBench Coding Avg 81.45 / Agentic Coding Avg 57.58	—	—	Newer, larger generation; current LiveBench leader on both metrics
GLM-5.2	Leads single-node-fitting models	—	—	Ahead of MiniMax M3, DeepSeek V4 Pro, MiMo-V2.5-Pro
Claude 4 Sonnet (reference)	70.4% SWE-bench Verified	—	—	Chinese open models are within 5–6 points of this

sources 2–4

Licensing

📜

Apache 2.0 — most permissive

Qwen3 and Gemma 4 both ship under Apache 2.0, the most commercially permissive license in this set.

📜

MIT — also permissive

DeepSeek-V3.2 is MIT licensed, comparably permissive to Apache 2.0 for most commercial use.

Reading This Table

These are the current leaders on dedicated tool-calling and agentic-coding benchmarks — GLM-4.5 for raw function-calling accuracy and tool-use success rate, Qwen3-Coder and DeepSeek-V3.2 for SWE-bench-style coding correctness, Kimi K2.6/K3 for sustained long-running agentic sessions, GLM-5.2 for whatever still fits a single node.

None of this landscape says anything yet about what runs on small local hardware. That’s Tab 3.

Hardware Reality · Sources

The Hardware Reality — What Actually Fits

Checked directly against real model sizes from the oMLX/exo catalogs actually available on these two Mac Minis today: Qwen3-Coder-480B-A35B is 276GB, Qwen3-Coder-Next is 54–65GB even quantized, and GLM-4.7-Flash — GLM’s smallest variant — is still 18–25GB. The two Mac Minis’ actual usable memory ceilings, measured today via oMLX’s own memory-guard errors, were 4.00GB and roughly 8.3GB.

Model size vs. what two real Mac Minis can actually hold (log scale)

Ran on a Mac Mini today Too large for either Mac Mini tested ┅ Amber dashed = measured memory-guard ceiling

Size vs. Fit — Reference Table

Model	Disk Size	Fits 4.00GB Mac Mini	Fits ~8.3GB Mac Mini
Qwen3.5-4B-MLX-8bit	~5GB	No	Ran today
Meta-Llama-3.1-8B-Instruct-8bit	~8.3GB	No	Ran today, at the ceiling
GLM-4.7-Flash (GLM’s smallest)	18–25GB	No	No
Qwen3-Coder-Next	54–65GB (quantized)	No	No
Qwen3-Coder-480B-A35B	276GB	No	No

---

Sources

• Tool-calling benchmark figures (GLM-4.5, Kimi K2, Qwen3 / Qwen3-Coder — BFCL v3 and tool-use success rate) medium.com/@cognidownunder — Kimi K2 vs Qwen3-Coder vs GLM-4.5
• Coding benchmark landscape and licensing (Qwen3-Coder, DeepSeek-V3.2, licensing notes) mindstudio.ai/blog — Best Open Source LLMs for Agentic Coding 2026
• Coding model comparison (SWE-bench Verified figures, license details) morphllm.com — Best Open Source Coding Model 2026
• Kimi K2.6 / GLM-5.1 / Qwen3.6 Plus / MiniMax M2.7 coding comparison (Kimi K2.6 SWE-bench Pro, Terminal-Bench 2.0, sustained tool-call session) atlascloud.ai/blog/guides — Kimi K2.6 vs GLM-5.1 vs Qwen3.6 Plus vs MiniMax M2.7
• Best open-weight LLMs for agentic coding (Kimi K3 LiveBench figures, GLM-5.2 single-node leadership) howaiworks.ai/blog — Best Open-Weight LLM for Agentic Coding