Notes from Sai Dutta Abhishek Dash — AI infrastructure & security engineer — on LLM gateways, code-security agents, self-hosted systems, and the open-source tools he ships.
VRAM is everything for local AI agents: the 4-bit cheat sheet runs 7B at about 5GB, 14B at 10GB, 30B at 20GB, and 70B at 40GB. Three builds cover it, from a $1,200-1,500 RTX 4060 Ti 16GB starter to the used 24GB-card sweet spot that runs 32B models at near-cloud quality for daily work.
How Heretic removes refusal behavior from open-weight LLMs: my evaluation philosophy, Optuna TPE search learnings, the ARA next-generation ablation, why single constant vectors fail on MoE models, and what publishing 34 abliterated models taught me about attention-versus-MLP placement across Qwen and Gemma families.
A plain-English map of startup funding: venture-scale vs durable profitable companies, what actually happens at pre-seed, seed, Series A and beyond, how dilution compounds round over round, and the four term-sheet clauses (liquidation preference, protective provisions, pro rata, drag-along) that decide who really controls the company.
Claude Code ranked the best local AI model for each VRAM tier from 4GB to 384GB - Nanbeige 3B at 4GB to Motif-3 314B at 384GB. Fact-checked July 2026: all five picks are real, available, and the 5.9GB Bonsai 27B even runs on an iPhone.
Why about 1B parameters is the sweet spot for agent swarms: MiniCPM5-1B and six rival small models profiled from roughly 200 t/s on a $0.01/hr T4 down to 5 t/s on a Raspberry Pi 5, with the swarm economics at 1K, 100K, and 10M agents per day.
How to erase yourself from the internet with Claude - a 5-step playbook that finds every exposure, removes you from data brokers, deletes old accounts, buries what you cannot delete, and locks down what is next.
The memory wall ends here: NVIDIA's DGX Station pairs 252GB of HBM3e at 7.1TB/s with 496GB of LPDDR5X into one 748GB coherent pool on GB300 Grace Blackwell Ultra, adds NVFP4 quantization and 800Gbps ConnectX-8 networking (two boxes pool about 1.5TB), at roughly $94,930.
Kun Chen's agentic engineering system: a captain-and-crew model with parallel agents, isolated worktrees, a no-mistakes validation pipeline, and overnight autonomous loops. The result is 40-50 production-ready, tested PRs per day.
AMD's $1500 Strix Halo runs the same 120B model as Nvidia's $4700 DGX Spark, only 13% slower. But DGX Spark wins prefill by 3x and compute by 13x. Full benchmarks, India pricing, and a hands-on check on 2x DGX Spark GB10 with a 30B Q4 model.
