Best local AI models by VRAM tier#

If you are buying hardware or trying to make the card you already own useful, the right question is not how big a model you can download. It is what actually fits and runs well on the VRAM you have.

This guide is organized by VRAM, from 4GB to 384GB, with one pick per tier that balances size, speed, and availability as of July 2026. Every model here is real, open-weight, and on Hugging Face. I checked the numbers against model cards, papers, and benchmarks and noted where they are rounded.

4-8 GB: Nanbeige4.2-3B - the small model that beats bigger ones#

You can run this on a laptop GPU or even on 2GB quantized. It is a 3B model (4B including embeddings) built as a Looped Transformer and trained from scratch on 28T tokens. The interesting part is not the size, it is the scores.

Spec	Detail
Parameters	3B non-embedding, 4B total
Architecture	Looped Transformer - runs its layers more than once
SWE-Bench Verified	63.6% (often rounded to 63%)
GPQA Diamond	87.4 (often rounded to 87)
Context	256K
License	Apache 2.0
Size quantized	~2.4GB

On most benchmarks it beats Qwen 3.5 9B and Gemma 4 12B despite being a fraction of the size. That is why it is the pick for this tier.

The model comes from Nanbeige LLM Lab in China (Boss Zhipin). One caveat from replication work by WebBrain: it does not win in every agentic tool-use test against Qwen 3.5 9B, but across the broader suite the lead holds.

Sources: arXiv:2607.22083, Hugging Face model card, AI Weekly, RuntimeWire.

8-24 GB: Bonsai 27B (Prism ML) - a 27B model that fits on a phone#

This tier is where most desktop GPUs live, and Bonsai is the most unusual pick.

Spec	Detail
Base	Qwen3.6-27B, compressed to ternary weights (-1, 0, +1), about 1.58 bits per weight
Size	5.9GB ternary (94.6% score retention), 3.9GB 1-bit (89.5% retention)
Context	262,144 tokens
License	Apache 2.0
Team	Caltech founders, backed by Khosla Ventures, Cerberus, Google, Samsung
Performance	163 tok/s on RTX 5090, 11 tok/s on iPhone 17 Pro - first 27B on a phone

The video describes this as "retraining every weight to be one of three values." PrismML calls it post-hoc compression, but the technical detail is Quantization-Aware Training with a Straight-Through Estimator. In practice the model was trained to live with ternary values, not just rounded after the fact. The wording is simplified, the result is correct.

The 95% retention figure you see quoted is 94.6% for the 5.9GB build, so a touch overstated, but not by much.

Sources: PrismML announcement July 14, 2026, Hugging Face collection, PrismML docs.

24-96 GB: Qwen 3.6 27B + ThinkingCap - less rambling, same score#

If you have 24GB or more, Qwen 3.6 27B dense is a strong default. It scores 87.8 on GPQA Diamond and is 54GB at full precision (27B x 2 bytes at FP16).

The variant to look at is ThinkingCap from BottleCap AI, co-founded by Tomas Mikolov of Word2Vec. It does not make the model smarter, it makes it less verbose. The claim is about 50% fewer reasoning tokens on average, up to 10x in some cases, for the same score. That is useful when you pay per token or need faster completions.

This one is harder to verify independently because the numbers are self-reported by BottleCap AI. The Mikolov connection is real, and the idea is plausible, but there is no third-party replication yet.

Sources: Qwen Hugging Face, BottleCap AI model card.

96-192 GB: Laguna S 2.1 (Poolside) - the best open coding model you can run on one DGX Spark#

This is the most documented pick.

Spec	Detail
Parameters	118B total, about 8B active per token (256 routed experts, top-10 + 1 shared)
Release	July 21, 2026, Poolside, San Francisco
Context	1M tokens (1,048,576)
License	OpenMDW-1.1, commercial use allowed
Terminal-Bench 2.1	70.2%
DeepSWE	40.4%
Runs on	Single DGX Spark at 50-77 tok/s (NVFP4 or INT4), Nvidia tuned

Laguna S 2.1 built a browser engine from an empty folder in about 50 minutes in Poolside's public trajectory, and it was trained on about 30T tokens in under nine weeks. It beats DeepSeek-V4-Pro-Max (64.0%) on Terminal-Bench 2.1. No Western open-weight coding model has scored higher, though closed models like GPT-5.6 Sol (88.8%), Claude Fable 5, and Kimi K3 still lead by 15 to 18 points.

Poolside published trajectories at trajectories.poolside.ai, so you can see the runs.

Sources: Poolside Hugging Face model card, Benchgen, MarkTechPost, explainx.ai.

192-384 GB: Motif-3 (Motif Technologies) - 314B from Seoul#

The top tier is Motif-3 from Motif Technologies, a Moreh subsidiary in Seoul with about 30 people.

Spec	Detail
Parameters	314B total, about 13B active per token (384 routed experts, 8 active + 1 shared)
Context	256K (262,144)
Size	About 300GB at 8-bit
Runs on	Two DGX Sparks
Ranking	AAII 44, #3 open source behind Kimi K3 and GLM 5.2
Origin	Korean sovereign-AI program Dokpamo, with 768 B200 GPUs from the government

Motif says it was built from scratch, not a fine-tune, but there is no paper to check. The important caveat the video does not mention is the license. Motif-3 is a Beta under a non-commercial research license. Commercial use needs written permission, and the final version was expected around early August 2026. Standard benchmarks like MMLU and GPQA have not been published, and the AAII 44 figure is press-sourced.

Sources: Artificial Analysis, Orcarouter, AI Weekly, Chosun Ilbo, Motif Hugging Face.

What this means if you are choosing today#

• None of these replace GPT-5.6 Sol or Claude Fable at the frontier. GPT-5.6 Sol is at 88.8% on Terminal-Bench 2.1, well above Laguna S 2.1 at 70.2%. Local models trade absolute capability for control, privacy, and cost.

• DGX Spark is 1.2 kg, 128GB unified, about $3,999 MSRP. Ollama and LM Studio both run with one install and one command, so trying these models is low friction.

• The rule that works well is simple: pick the biggest model that fits your VRAM without spilling to system RAM. All five above are available on Hugging Face as of July 2026 and run where listed.

Small notes on rounding: Nanbeige 63% is 63.6% and 87 is 87.4, Bonsai 95% is 94.6%. None of this changes the ranking.

Checked against Hugging Face model cards, arXiv:2607.22083, PrismML docs, BottleCap AI, and Poolside trajectories. Video source: Claude Code, "Best Local AI Models For Your GPU" (YouTube, July 25, 2026, 14:15, ~32K views as of July 27).