7 posts on SDAD Writes.
Epoch AI reviewed 15 major AI benchmarks and flagged 9 as flawed, including Humanity's Last Exam, where a hand-audited sample of 48 questions found 22 defective. I took their findings and ran them: filtering the broken items out of a live leaderboard, and the scores barely move. Both findings are true and they are not in tension.
A complete voice AI stack, speech to text, a 2B language model, and text to speech, running fully local on a 2013-era CPU server with no GPU. I benchmarked every stage, audited the marketing claims, and the "150 tokens per second" pitch came out 13.6x slower in reality. Here is what a local voice assistant actually costs and does.
I audited every claim on the Spark-X2.5 model cards and stress-tested both the 4B and 1.7B on agentic coding tasks over two Tesla T4s. The 4B is a genuinely usable small coding agent. The 1.7B is an assistant, not an agent. And both models have a thinking-verbosity tax worth knowing about.
I ran the first head-to-head of the two open "System One" decision models, Laya and Jev, on 751 byte-identical inputs: same states, same questions, same seed, question hashes verified before comparing. Jev wins most suites for $0.008. Laya wins where it counts for a free model. A fourth column with Qwen 1.5B under constrained decoding completed the picture.
I stress-tested Ternary Bonsai 2 27B on two Tesla T4s: 15 tokens per second on a single weak GPU, the full 262k context loading in 26GB of VRAM, and a complete todo webapp built and self-tested inside a local coding agent. The claims mostly check out, with a few catches.
I benchmarked LFM2.5 350M and Qwen3.5-2B on my iPhone SE 2 (2020) with 3GB of RAM using the PocketPal app. One model hits 70 tokens per second. The other needs 22 minutes for a single pass. Here is where the line is.
Real Qwen3.8-27B numbers on the DGX Spark GB10: the memory wall caps single-stream decode at 11-12 tok/s, and speculative decoding is the only way out. SGLang with DSpark hits 34 tok/s, veloGB10 reaches about 40 tok/s, and NVFP4 keeps quality within noise of FP8.
