Models

Every model tested on this hardware tier with its full configuration disclosure, measured results and scenario recommendations. Recommendations are opinions grounded in theresults and findings.

1. Qwen3.6-35B-A3B-MTP

Qwen/Qwen3.6-35B-A3B-MTP-GGUF

Default local model of the benchmark.

Configuration

  • params total: 35B MoE
  • params active: ~3B
  • quant: GGUF (vendor quant, Q4-class)
  • quant source: vendor GGUF, Lemonade defaults
  • ctx: 65536
  • sampling: temp 1.0, top_p 0.95, top_k 20, min_p 0, preserve_thinking (vendor thinking preset)
  • spec decoding: MTP head embedded; draft acceptance ~0.66-0.95
  • backend: Lemonade 2026.45.3 / llama.cpp b11846 / Vulkan

Results (variant baseline):235/243 (96.7%), sum of runs 69 min —full per-task detail.

Scenario recommendations

  • → live coding agent — good best score-per-minute on both machines; responsive
  • → fire-and-forget — good short thinking traces, highest accuracy of the MoEs
  • → trust its own reports — avoid never reports plan deviations (0/3 on code-3); verify claims

2. GLM 5.3 Flash (remote)

https://openrouter.ai/z-ai/glm-5.3-flash

First full 9-task pi run (2026-11-16): 235/243 (96.7%) in 9 min wall, 366k prompt / 17.6k completion tokens — the most economical run on record. Comparisons vs local models are score-only (wall time not comparable).

Configuration

  • params total: ? (not disclosed)
  • params active: ?
  • quant: provider-side fp8 (Z.AI endpoint pinned)
  • quant source: OpenRouter provider pin, allow_fallbacks off
  • ctx: 1,048,576
  • sampling: server defaults; pi sends reasoning_effort: medium
  • spec decoding: provider-side
  • backend: OpenRouter -> Z.AI fp8

Results (variant flash-zai):235/243 (96.7%), sum of runs 9 min —full per-task detail.

Scenario recommendations

  • → reference tier — good the remote reference: 235/243 on the full grid; prior runs 124/125 with OpenCode
  • → cost-insensitive work — good no local load; 1M context
  • → trust its own reports — ok prior minimal-loop runs never reported the code-3 deviation

3. Recursive Flint 40B-A2.5B

unsloth/flint-GGUF

Only score loss vs K2-7B was one analysis timeout (-7). SWE-bench Verified 74.2% (vendor).

Configuration

  • params total: 39.8B MoE (256 routed experts + 1 shared)
  • params active: ~2.5B
  • quant: UD-Q4_K_XL (Unsloth dynamic), 23.3 GiB
  • quant source: unsloth/flint-GGUF (base: RecursiveSI/flint)
  • ctx: 65536
  • sampling: temp 1.0, top_p 0.95, repeat_penalty 1.0, q8_0 KV cache (vendor card)
  • spec decoding: none
  • backend: Lemonade 2026.46.2 / llama.cpp b11846 / Vulkan

Results (variant baseline):230/243 (94.7%), sum of runs 186 min —full per-task detail.

Scenario recommendations

  • → fire-and-forget — good strong accuracy and the best sustained decode on an iGPU (23-27 tok/s)
  • → time-boxed jobs — ok huge reasoning appetite (240k tokens/batch); budget ~2x wall clock of peers
  • → long-running sessions — good barely throttles (12% droop vs ~50% for bigger-active models)
  • → trust its own reports — avoid never reports spec deviations (0/3)

4. K2-Horizon-7B (dense)

IFM/K2-Horizon-7B-GGUF

Supersedes the 36B MoVA sibling on this hardware: same wall time, better score, ~3x less memory.

Configuration

  • params total: 7B dense
  • params active: 7B
  • quant: Q4_K_M
  • quant source: IFM official GGUF
  • ctx: 65536 pinned (see limitations: loaded at ~164k on this batch)
  • sampling: temp 1.0, top_p 0.95, repeat_penalty 1.0, reasoning_effort high (vendor: always high)
  • spec decoding: none
  • backend: Lemonade 2026.46.2 / llama.cpp b11846 / Vulkan (batch ran at ~164k ctx, see limitations)

Results (variant thinking-high):230/243 (94.7%), sum of runs 151 min —full per-task detail.

Scenario recommendations

  • → fire-and-forget on modest hardware — good top-tier accuracy for its size; think time is the price
  • → live coding agent — ok decode 12-14 tok/s fresh, slower on the throttle plateau
  • → destructive trim plans — avoid proposed unsafe deletions in 3/3 trim-plan runs
  • → trust its own reports — good reports spec deviations 2/3 - best with its 36B sibling

5. K2-Horizon-MoVA-36B-A4B

bartowski/K2-Horizon-MoVA-36B-A4B-GGUF

Kept as reference; first MoE to report the code-3 deviation (3/3).

Configuration

  • params total: 36B MoE
  • params active: ~4B
  • quant: Q4_K_M (bartowski), q8_0 KV cache
  • quant source: bartowski (modelscope mirror)
  • ctx: 65536
  • sampling: temp 1.0, top_p 0.95, reasoning_effort high, flash-attn, q8_0 KV
  • spec decoding: none
  • backend: Lemonade 2026.45.3 / llama.cpp b11846 / Vulkan

Results (variant thinking-high):226/243 (93.0%), sum of runs 147 min —full per-task detail.

Scenario recommendations

  • → destructive trim plans — good only tested model with clean safe-deletes discipline at this size
  • → general agent work — superseded same wall time but 4 points below its own 7B sibling; 3x more memory

6. JetBrains Mellum2.1-12B-A2.5B-Thinking

bartowski/Mellum2.1-12B-A2.5B-Thinking-MTP-GGUF

Cleanest negative result of the bench: substance present, output-form discipline absent.

Configuration

  • params total: 12B MoE (64 experts, 8 active)
  • params active: ~2.5B
  • quant: Q4_K_M, MTP head merged in same file
  • quant source: bartowski static quants (KLD 0.075 vs BF16, vendor-published)
  • ctx: 65536
  • sampling: temp 0.6, top_p 0.95 (vendor card)
  • spec decoding: MTP draft head; draft acceptance ~46% on this workload
  • backend: Lemonade 2026.46.2 / llama.cpp b11846 / Vulkan

Results (variant baseline):158/243 (65.0%), sum of runs 157 min —full per-task detail.

Scenario recommendations

  • → live sub-agent bursts — ok fastest cold decode after Qwen3.6-MTP thanks to the draft head
  • → structured-report tasks — avoid fails the structured answer form (parsed_json ~13 times); scores collapse to 65%
  • → honest deviation reporting — avoid never reports spec deviations (0/3)