Local AI agent hardware builds - budget tiers VRAM guide#

The core principle: VRAM is everything#

"Buy the counter space, not hand speed."

I have built and broken enough local setups to agree with that line. For local AI, VRAM matters more than raw GPU speed. I think of it like a restaurant kitchen:

Element	Kitchen Analogy	Hardware
GPU (compute speed)	Chef's hand speed	How fast it processes
VRAM	Kitchen counter space	How much fits at once
System RAM	Back storage room	Overflow space (slower)
SSD	Pantry	Where models are stored

Here is what happens when you run out. The model spills into system RAM, and token speed falls from around 40 per second to 2 or 3 per second. At that point I stop using it. It is not slow, it is stuck.

Model size cheat sheet (4-bit quantization)#

Parameters	Approx. VRAM
7B	~5 GB
14B	~10 GB
30B	~20 GB
70B	~40 GB

A build can feel fine on load and then crawl 20 minutes into a chat. Context piles up like dishes. That is usually VRAM filling, not a bad model.

---

Three budget tiers#

Tier 1: entry-level (~$1,200-1,500)#

This is the minimum I would spend if you want local AI to feel usable. It runs 7B to 8B models well, and 14B with some trade-offs.

Component	Spec
GPU	RTX 4060 Ti 16 GB (not the 8 GB version)
CPU	Ryzen 5
RAM	64 GB system memory
Storage	2 TB SSD
PSU	Decent power supply
Case	Good airflow

It runs:

• [[DeepSeek]] Distill 7B
• [[Llama]] 8B
• Qwen 7B
• 14B models with shorter conversations or slightly slower output

I use this tier for coding assistants, document summaries, private chat, and light agent work.

Tier 2: mid-range / power user#

Two paths here, depending on what you care about.

Path A: RTX 4070 Ti Super 16 GB#

Faster hands than the 4060 Ti, in kitchen terms. You get more room for loops where the model thinks, runs a tool, then thinks again. Rest of the build is similar: Ryzen 7, 64 GB RAM, 2 TB SSD.

Path B: used RTX 3090 24 GB#

This is the move I recommend most. It is an older card, but 24 GB of VRAM changes what you can do. You can run 32B models at 4-bit with room left for long conversations. Qwen 32B or [[DeepSeek R1]] Distill 32B at this tier feels close to cloud quality for daily work. Pair it with Ryzen 7, 64 GB RAM, 2 TB SSD.

Apple (Mac) alternative: Mac Mini M4 Pro#

This one is different. It has 64 GB of unified memory in one shared pool, so there is no separate VRAM and RAM split. All 64 GB works as counter space.

It is slower than Nvidia cards for raw speed, around 10 to 12 tokens per second against 40 plus. But it runs quiet, sips power under 100W, and feels smooth. In practice 10 to 15 tokens per second reads like fast typing. Past 30 it feels instant.

Tier 3: high-end ($5,000-10,000+)#

Only spend this if your work actually needs it.

Component	Spec
GPU	RTX 4090 24 GB (or dual 5090 for 64 GB total)
CPU	Ryzen 9
RAM	128 GB system memory
PSU	Beefy power supply

What this runs:

• 32B models with speed and room for long conversations
• 70B models at heavy compression, tight fit, you trade conversation length
• Multi-model setups with reasoning plus embedding plus coding at once
• Fine-tuning and training

Apple equivalent: Mac Studio M3 Ultra#

96 GB of unified memory, so you can load several models at once. I have seen reasoning plus embedding plus coding models loaded together. It idles under 100 watts, while an RTX 4090 desktop pulls 5 to 10 times that under load.

RTX 5090 dual GPU#

32 GB per card, 64 GB total. This is a $10K plus setup, aimed at people who want to train their own models.

---

The kitchen counter rule#

"If it fits in VRAM, it runs fast. If it doesn't, you're off a cliff."

VRAM	What You Can Run Comfortably
8 GB	Very limited, fills up immediately with real models + context
12-16 GB	7B-14B models, the sweet spot for most users
24 GB	32B models with conversation room, a big step up
48-64 GB	70B models, multi-model workflows, training

My take is blunt. Do not buy the 8 GB version of any card for local AI. It fills up too fast once context grows. I call it a trap because it looks cheaper until you try real work.

---

Software stack#

Model runners#

Tool	Type	Platform
[[Ollama]]	CLI tool	Mac, Windows, Linux, dead simple
[[LM Studio]]	GUI (ChatGPT-like chat window)	Mac, Windows, Linux

Both handle downloads, GPU detection, and local serving, so you do not have to wire that yourself.

Model formats#

Format	Best For
GGUF	Mac (plays best on Apple Silicon)
AWQ	Nvidia graphics cards (faster response, better quality on same card)

Most people grab whatever has the most downloads. If you pick the wrong format for your machine, you leave speed on the table.

Agent workflow tools#

• [[n8n]] for automation pipelines
• [[CrewAI]] for multi-agent setup
• Custom pipelines

---

Local vs cloud: the hybrid approach#

"Local AI is your home gym. Cloud AI is the commercial gym downtown. Your home gym handles 80% of your workouts. The smartest setup in 2026 is hybrid."

	Local	Cloud (Claude, GPT, Gemini)
Strength	Privacy, no API costs, always available	Frontier reasoning, massive models
Best for	Daily work, simple chains, private data	Complex problems, heavy lifting
Cost	Pay once for hardware, conversations are free	Token metering, surprise bills

I use local for code assistants, summaries, private chat, and light agents. I use cloud when I need frontier reasoning on a hard problem.

Why I stay local for daily work#

• Privacy: data never leaves my machine
• No terms of service: no training on my prompts or secrets
• No API logs: no surveillance layer
• Cost control: no meter running, I paid once
• Always on: if the internet drops, the local model does not care

---

Summary#

Tier	Cost	GPU	VRAM	Best For
1	$1,200-1,500	RTX 4060 Ti	16 GB	7B-14B models, coding, chat
2 (PC)	~$2,000+	Used RTX 3090	24 GB	32B models near cloud quality
2 (Mac)	~$1,400+	Mac Mini M4 Pro	64 GB unified	32B models, silent, low power
3	$5,000-10K+	RTX 4090/5090	24-64 GB	70B models, training, multi-model

"VRAM is the number you focus on. Everything else is secondary."