Nemotron 3.5 Bolt: NVIDIA's 74B Open Agent Model for 24GB GPUs

AI news, decoded · 5 minutes

Nemotron 3.5 Bolt: NVIDIA's 74B Open Agent Model for 24GB GPUs

What this means for you

If you have been waiting on running AI locally, this is the most serious moment to pay attention in the past year: 74B-class used to mean datacenter GPUs or pricey cloud APIs, and NVIDIA says it now runs on a 24GB machine — fully open source, weights, data, and recipes included. Three honest caveats: “fits in 24GB” is not “runs well” (context length, KV cache, and runtime overhead all add up, and every performance claim is NVIDIA’s own until independent benchmarks land); pooling several PCs the way PAIR does is genuinely useful if you have more than one computer at home; and you do not need to wait for tomorrow — our local LLM guide works right now. In short: cloud APIs are the pay-per-use convenience store, local models are the warehouse you stock yourself, and NVIDIA is trying to lower the warehouse’s entry price.

On October 13, 2026, NVIDIA announced that Nemotron 3.5 Bolt 74B-A9B will officially launch on Thursday, October 15 — one day before RTX Spark laptops go on sale on October 16. It is a 74-billion-parameter mixture-of-experts (MoE) model that activates only about 9 billion parameters per inference, compressed via 2-bit quantization to run in 24GB of VRAM. The standout is the openness: NVIDIA promises fully open weights, training data, and training recipes, with day-one support for llama.cpp, vLLM, Ollama, Unsloth, and LM Studio, plus direct integration with the Hermes and OpenClaw agent frameworks. This is the most significant open-source release in the local AI space this year.

What happened

Start with the hard specs. The numbers behind Nemotron 3.5 Bolt 74B-A9B are straightforward: 74 billion total parameters (the model's knowledge base), but the MoE architecture activates only about 9 billion per inference — decoupling total knowledge from per-inference compute cost. Combined with 2-bit quantization and quantization-aware training (which simulates precision loss during training to preserve quality after compression), the whole model fits into 24GB. NVIDIA's claim: faster than dense models of similar size, smarter than MoE models with a similar memory footprint.

The positioning is equally clear: this is built for agentic AI on RTX Spark. RTX Spark laptops pair a Grace CPU with a Blackwell GPU, up to 128GB of unified LPDDR5X memory (connected via NVLink-C2C), and 1 petaflop of AI compute, going on sale October 16. NVIDIA says local LLM downloads are already 5x all of last year — Bolt 74B is riding that wave: keeping 74B-class agents resident on personal computers to write code, monitor tasks, call tools, and run background jobs, all locally, without routing every request through a cloud API bill.

Two more things shipped in the same batch. First, PAIR (Personal AI Router) graduated from beta: a free, open-source tool that auto-discovers compatible PCs on a home network and distributes AI inference workloads across multiple machines. The full release adds llama.cpp support, direct model downloads from Hugging Face, and RTX Spark compatibility, running on Windows, Linux, and macOS. Second, the smaller Nemotron 3.5 Bolt E4B is dated October 23, targeting conversational agents and game NPCs — developers can already try it in SkyrimNET, the AI NPC modding platform for Skyrim.

Not just a model: a full local stack

What stands out is NVIDIA's playbook here: the model is the ammunition, the full local AI stack is the territory. The same software batch includes Llama for Windows (a free, open-source app built on llama.cpp that runs open models without touching the command line), MCP servers for NVIDIA's own apps (including the NVIDIA app, Broadcast, PAIR, RTX Remix, and Omniverse), Broadcast 2.3, and a sandboxed agent execution framework built with Microsoft (Microsoft Execution Containers) entering early preview. On the agent app side, GitHub Copilot, Hermes, OpenClaw, Perplexity, and others will streamline setup on RTX Spark with security features through OpenShell and MXC.

To translate: NVIDIA is answering the question every "local AI" launch of the past two years has dodged — how do you fit a frontier-class model into consumer hardware without stripping out what makes it frontier-class. Their answer: MoE decoupling knowledge from compute, plus 2-bit quantization-aware training, plus a full software stack for the model to live in. Whether that answer holds up awaits independent benchmarks once the weights land on October 15.

One caveat

Every claim about post-quantization quality and speed comparisons comes from NVIDIA itself, relayed by third parties — with no independent benchmarks behind any of it until the weights land on October 15. By our own rules, that has to be stated up front: read this as launch news, not as a review.

Sources

Blockchain.News: NVIDIA Announces Software Ecosystem Updates Ahead of RTX Spark Laptop Launch
Derrington Collaborative AI: NVIDIA squeezed a 74B agentic model into 24GB — and open-sourced the whole thing
TechTimes: NVIDIA RTX Spark Launches Friday: 74-Billion-Parameter AI Fits in 24GB Laptop
Gadget Pilipinas: RTX Spark Launch Week: Nemotron 3.5 Bolt and DLSS 5 Updates
alfw.ai: Nemotron Bolt 74B: 74B in 24GB, Only 9B Active
BitcoinVersus.Tech: NVIDIA Puts a 74B AI Model on RTX Spark
Technobezz: NVIDIA Readies RTX Spark Software for Friday Laptop Launch