Nvidia's Switchyard router reshuffles AI models mid-task, cutting task costs to a third in its own tests
A new open model and an open router that ships alongside it are Nvidia's bet that the cost of AI agents gets solved at two layers, not one.
Written by OutOfToken AI
August 12, 2026 · 5 min read · Synthesized from reporting by VentureBeat · How this works
Every enterprise running AI agents at scale hits the same wall: route everything to a frontier model and the bill balloons, or build custom logic to send easy tasks elsewhere and inherit a maintenance project. Nvidia thinks it has a fix for both sides of that tradeoff at once. On Tuesday, the company released Nemotron 3.5 Lightning, a 30-billion-parameter open mixture-of-experts model, alongside NeMo Switchyard, an open-source router that reassigns models mid-task as an agent's workflow evolves.
A model and a router, sold as one idea
Nvidia says Lightning delivers up to 4x faster output than comparable models in its class, completing agentic tasks roughly 30% faster than Qwen3.6-35B at matching accuracy. Paired through Switchyard, the company says the combination holds frontier-level task completion while cutting benchmark costs to roughly a third of running Anthropic's Opus 4.8 alone. Neither half of that pitch works without the other: a fast, cheap model has no leverage without something to route around it, and a router is only as good as what's in its pool.
Why the routing decision can't stay fixed
Model routing itself isn't new — OpenRouter, LiteLLM and standalone routing startups already split traffic across providers. What Switchyard targets is narrower: the right model for a task changes as the task unfolds, not just at the start. Kari Briski, Nvidia's vice president of generative AI, described routing strategies that react to an agent's shifting state, from tool results to errors to a step turning out simpler than expected, rather than locking in a model choice upfront.
""It has many types of routing strategies," Briski said. "In some cases you want to go with a model like Lightning for really efficient tasks, and the router will actually choose Lightning if it's set up in your pool of models.""
Built to slot into existing stacks, not replace them
Nvidia split its partners into two camps: agent frameworks that call Switchyard directly, including Cognition, LangChain and Nous Research, and gateway providers that built Switchyard support into their own products, including Kong, LiteLLM and OpenRouter. That distribution strategy matters as much as the technology. Real-world results back it up — LangChain reported a 74% cost cut across 145 multi-turn tasks by routing just 7% of calls to a frontier model, Ramp cut costs 58% while matching frontier performance, and Cognition's integration into Devin Desktop landed within a few percentage points of frontier accuracy at roughly 28% lower mean cost.
Lightning isn't trying to win on raw intelligence
On the Artificial Analysis Intelligence Index, a general capability benchmark, Lightning scores 24 — tied with gpt-oss-120b and behind Nemotron 3 Super, Gemma 4 31B, Claude 4.5 Haiku and Mistral Medium 3.5, all at 30. Nvidia isn't claiming a capability crown; its actual claim, drawn from PinchBench data covering coding, research and file-management tasks, is that Lightning matches Qwen3.6-35B's accuracy roughly 30% faster and beats Gemma 4 26B at similar completion time. That's a speed-to-accuracy tradeoff, not a general-intelligence win, and Nvidia is positioning Lightning specifically as the fast, cheap end of a routed system rather than a standalone frontier contender.
The release lands in a crowded open-weight moment
Alibaba, Moonshot, Zhipu and DeepSeek have all shipped competitive open models since spring, some near frontier performance while undercutting US labs on size or price. Meta added its own 30-billion-parameter open agentic model, Muse Glimmer, into the same window. Asked how Lightning compares to Chinese models directly, Briski didn't offer a head-to-head benchmark, instead pointing to openness and customizability as Nvidia's differentiator — a signal that the fight here isn't purely about raw scores anymore.
Switchyard's real competition isn't other open models — it's Not Diamond, which already powers OpenRouter's Auto mode, and RouteLLM, the open-source framework out of UC Berkeley and LMSYS. Neither ships its own model, and Nvidia is betting that owning both the model and the routing layer under one open license is the thing a router-only or model-only rival can't replicate. Whether that combined bet holds will depend less on benchmark charts and more on how well enterprises can tune routing policies against workflows that keep changing shape.
Editorial Note
The research strongly corroborates the article's core claims about Switchyard's cost savings (one-third of Opus 4.8), the 4x speed improvement, and specific partner cost/performance metrics from Cognition and Ramp. The sources confirm the product specifications and real-world deployment results. However, the research does not contain detailed technical benchmarks comparing Lightning directly to Qwen3.6-35B, leaving one specific performance claim unverified.
Claim Tracker
AI-assessed
Source 3 (AI Weekly) confirms: 'Nemotron 3.5 Lightning activates only 3B of 30B parameters per inference step, delivering 4x output speed over comparable dense models.'
Source 4 (NextPlatform) states: 'the chart below shows how Switchyard delivers better accuracy at a third of the cost of Anthropic's Opus 4.8 model alone.' Source 2 also confirms frontier-level accuracy maintenance with cost reduction.
Source 5 (Nvidia Technical Blog) confirms: 'the implementation routed between Opus 5 and Kimi K2.7. It delivered near-frontier performance, achieving 50.6% at a $3.11 mean cost—within 2.8 percentage points of Opus 5 accuracy at approximately 28% lower mean cost.'
Source 3 (AI Weekly) lists real-world deployments: 'Ramp cut costs 58% and runtime 33%.'
The research provided does not contain specific benchmark data comparing Lightning's performance to Qwen3.6-35B at matching accuracy. The article cites PinchBench data but the sources do not detail this specific comparison.
Ask AI about this story
// discussion
sign in to join the discussion