MiniCPM5 1B small model landscape swarm control#

Executive summary#

Sam Witteveen argues that MiniCPM5-1B is the strongest current option for what Andrej Karpathy calls the cognitive core. That is the small, fast model that sits in the middle of a multi-agent system and decides what happens next. Instead of sending every thought through a large frontier model, a distilled 1B model handles planning, tool choice, and context, and you only call bigger models when you need specialized generation.

The survey through mid-2026 found MiniCPM5-1B unusually strong on the mix that matters for swarms: token efficiency, reasoning, tool use, and context length. Other 1 to 2B models have trade-offs, but MiniCPM5-1B gives the most balanced package for agent work.

The cognitive core thesis#

Karpathy splits the stack into three layers:

Why 1B parameters is the sweet spot#

• Fast enough for interactive loops
• Small enough to run on a phone, on the edge, or on a cheap high-throughput server
• Large enough to keep the reasoning that appears around this scale
• Can be quantized to 4-bit with small loss
• Fits in CPU memory, so you can run it without a GPU

Why not smaller (800M, 500M)#

Below about 1B, tool use gets unreliable and multi-step reasoning falls apart. Routing decisions start to hallucinate more often.

Why not larger (7B+)#

Above 7B you add latency, cost, and ops overhead that works against the idea of a light core.

MiniCPM5-1B is the first model at this size to show all three requirements together:

1. Strong reasoning at tiny scale
2. 128K context for agent memory
3. Reliable tool use for orchestrating other models and APIs

Model profiles#

MiniCPM5-1B (OpenBMB)#

Token efficiency is where the gap shows up. Sam tested a structured extraction task side by side:

MiniCPM5-1B (think mode):
  "To extract the date, I need to find the transaction record...
  [reasoning chain]... Answer: January 15, 2025."
  ~80 tokens total
 
Qwen 3.5 2B Reasoning (chain of thought):
  "Let me think about this step by step...
  First, I need to identify if there's a transaction...
  Looking at the text, I can see...
  [extended reasoning]... Therefore, the answer is...
  January 15, 2025."
  ~3,100 tokens total

That is about 31 times fewer tokens for the same answer.

On the AA Omniscience leaderboard for small-model general intelligence, MiniCPM5-1B ranks at the top. It scores above models two to ten times its size.

Limitations to keep in mind:

• Quality can slip after about 64K in very long contexts, with occasional token bleeding
• Creative writing is weaker than larger models
• Multilingual ability is not as strong as Qwen
• The ecosystem around it is smaller than the Llama family

Latency and deployment:

MiniCPM-4.6 (MiniCPM-o, OpenBMB)#

This is the previous version. It kept vision input that the text-only MiniCPM5-1B removed.

Useful when the core needs to see screenshots, UI states, or diagrams.

Qwen 3.5 2B Reasoning (Alibaba)#

The extra verbosity makes it a poor fit for a core where token efficiency matters. It makes more sense when you care about raw reasoning depth more than speed or cost.

Qwen 0.8B (Alibaba)#

This size falls below the threshold where tool use stays reliable. Multi-step reasoning and instruction following become inconsistent. It can work as a single-purpose agent, but not as the orchestrator.

LFM-1.2B Thinking (Liquid AI)#

Interesting technically, but the lack of GGUF, llama.cpp, and vLLM support makes it less practical for production swarms right now.

GLM 5.2 (Zhipu AI)#

Not a cognitive core option. Included here because MiniCPM distillation may draw from this class of model.

Gemma 4 (Google)#

The contrast helps put MiniCPM5-1B in perspective. Gemma 4 is about 80 times larger, yet MiniCPM5-1B holds its own in the narrow role of a core.

Historical references#

The smallest size that could serve as a viable core dropped from 8B in 2024 to 1B in 2026.

Swarm control suitability analysis#

Evaluation criteria#

Model rankings for swarm control#

Recommended swarm architecture#

Cognitive Core: MiniCPM5-1B (1.1B)
  - Task decomposition
  - Agent selection/routing
  - Tool call orchestration
  - Context management
  - Result synthesis
       |
   ----+----
   |    |    |
   v    v    v
Specialist A (Frontier)   Specialist B (Mid-size)   Specialist C (Small)
  - Creative / writing      - Code gen / analysis     - Classify / extract
  - Strategy                - Planning                - Validate

Why MiniCPM5-1B wins for swarms#

The advantage shows up in the loop. Every agent turn burns tokens, so a model that uses 31 times fewer tokens lets the context last much longer before you need to summarize or evict.

It also gives you per-step control. Think mode is useful for planning, no-think mode is faster for routing, and you can choose per call.

It runs on small hardware. The quantized build works on a [[Raspberry Pi 5]], which means you can keep orchestration air-gapped with no API dependency. See [[Local AI Agent Hardware Builds - Budget Tiers VRAM Guide]] for build options.

And the cost gap is large at scale:

At those rates the core layer is about 500 times cheaper with the 1B model, and still good enough to orchestrate.

Deployment strategies#

See [[AMD Strix Halo vs Nvidia DGX Spark Local AI Hardware 2026]] for edge hardware comparisons and [[NVIDIA DGX Station - Trillion Parameter Desktop AI Deep Dive]] for frontier-tier infrastructure.

Monitoring considerations#

When you run a small model as the core, watch these four signals:

1. Tool call failure rate - small models can loop or emit malformed calls
2. Context saturation rate - how fast the window fills per session
3. Agent routing accuracy - whether tasks go to the right specialist
4. Delegation threshold - when to hand off to a frontier model

Risks#

• Model maintenance. Will OpenBMB keep improving MiniCPM5-1B? Apache 2.0 guarantees you can keep using it, but community support could thin over time.
• Distillation quality at scale. Fine-tuning on your own swarm data can upset the balance from the original distillation.
• Hallucination in long runs. Very long sessions with 100+ tool calls can accumulate context drift. This affects all models, but it tends to show sooner at 1B.
• Security. Small models are more open to [[prompt injection]] in agent loops. You will need guardrails.

Future watch (mid-2026)#

• OpenBMB next release - it may bring back vision or push context to 256K
• Liquid AI ecosystem growth - if LFM tooling matures, it could challenge MiniCPM5-1B
• Llama 4 mini - Meta may ship a sub-3B competitor
• Apple on-device LLMs - iOS models could shift the edge picture

For hardware to run these models, see [[AI Inference Finetuning PC Build India 2026]] and [[AI PC Build Tiers India 3L to 10L 2026]].