Where NVIDIA Is Going: What Its Open-Source Code Reveals

Six months of NVIDIA's public code changes point to a new business: selling AI agent work by the megawatt, with its own software deciding which chip does each job.

Executive summary

NVIDIA is quietly changing what it sells. It is moving from selling chips to selling a managed result: AI answers for software agents, delivered at the lowest cost per unit of electricity. Its own software decides, moment by moment, which kind of chip does which part of the work.

This is not in NVIDIA's keynotes. It shows up in the code. Between June and November 2026, NVIDIA's engineers made about 14,000 public changes to ten of its most important open-source projects. Read together, those changes tell a clearer story than any press release.

Four things stand out:

• Software, not hardware, now decides how work is split between chips. NVIDIA's new Groq-based chip is publicly described as a helper in a few fixed setups. The code lets a compiler divide each AI model piece by piece between NVIDIA's GPUs and the new chips.
• Every AI model is getting a small "speed-up" companion model. NVIDIA is building the tools to make these companions for its own models and for rival labs' models. Whoever makes the companion has a say in which chips run the model.
• Electricity has become the budget. New controls let a data center set a total power limit. The software then spreads work to get the most output inside that limit.
• The customer being served is an AI agent, not a person typing. The software now recognizes agents, keeps track of their ongoing sessions, and routes them to keep their memory warm.

One more change runs underneath all of this: AI agents now write a growing share of NVIDIA's own software. In some projects it is more than a quarter of all changes. That lets NVIDIA support every new AI model on day one without hiring in proportion.

The bottom line for executives: the right way to compare AI infrastructure is shifting. "Whose chip is fastest" matters less than "how much useful agent work do I get per megawatt." NVIDIA is building the software that makes that second number its advantage.

Why read the code

Public code shows what a company is actually building, months before it shows up in marketing. Engineers have to write down, in exact terms, what the product will do. That makes the record harder to spin than a slide deck.

NVIDIA publishes a lot of its software openly on GitHub. Cloud providers, AI labs and enterprises build on it, so NVIDIA has to develop much of it in public. Every change carries a date and a short description, and often the actual design notes.

We looked at ten of NVIDIA's most active projects, covering June through November 2026:

We counted recurring themes month by month, then read the design notes and examples behind the most telling changes. Where the code went beyond what NVIDIA has said publicly, we checked the press coverage to confirm the gap.

Background: why the economics of AI changed

The money in AI is moving from building models to running them. Training a model happens once. Running it, which the industry calls "inference," happens every time anyone or anything asks it a question. As AI agents take over long, multi-step tasks, the volume of inference is exploding.

Running a model has two very different phases:

• Reading the request ("prefill"). The model takes in the whole prompt at once. This is heavy, parallel number-crunching, and GPUs are excellent at it.
• Writing the answer ("decode"). The model produces its reply one word-piece (a "token") at a time. Each step needs the whole model pulled from memory again. Speed here is limited by how fast memory can be read, not by raw computing power. GPUs are less efficient at this phase.

That second phase is why NVIDIA made its biggest deal in years. In December 2025 it licensed the technology of chip startup Groq and hired its engineers, in a deal Tom's Hardware put at $20 billion. Groq's chip, which NVIDIA calls an LPU ("language processing unit"), keeps the model in very fast on-chip memory. Each chip has 512 MB of that memory, read at 150 TB per second, versus 22 TB per second for the main memory on each of NVIDIA's next-generation Rubin GPUs. That makes it very fast at writing answers.

NVIDIA unveiled the result, the Groq 3 LPX rack, at its GTC conference in March 2026, with shipments planned for the third quarter. Publicly, NVIDIA has described three fixed ways to pair it with GPUs (Tom's Hardware, Hot Chips 2026):

1. GPUs read the request; LPUs write the answer.
2. GPUs handle one part of each step (attention and memory); LPUs handle another part.
3. A small model on the LPU guesses ahead; the GPU checks its guesses.

Analysts have also cited a reference fleet mix of three GPUs for every one LPU (Spheron). That is the public picture. The code shows NVIDIA going well past it.

Signal 1: software decides how the work is split

What the code shows. NVIDIA's data-center software, Dynamo, has a hybrid mode its engineers call "Cyborg." In this mode a compiler, the software that prepares a model to run, cuts the model into numbered pieces. A setting then lists which pieces run on the GPU, piece by piece. Anything not listed runs on LPUs. On November 10 a change titled "consume compiler-owned LPX placement units" made the compiler the authority on where each piece goes.

A second example, from November 6, runs a speed-up model and the main model as a pair, both on LPUs, with no GPU involved. Public descriptions present the LPU as a helper to the GPU. This shows it can also run a whole model by itself.

Why it matters. The three public setups and the 3-to-1 ratio look like starting defaults, not limits. In practice, the mix of chips behind any given model becomes a dial that software turns. NVIDIA can tune that dial per model, per customer, and over time, without anyone buying different hardware.

Signal 2: every model gets a speed-up companion

The idea. Writing an answer one token at a time is slow. A common fix is a small "drafter" model that quickly guesses the next several tokens. The big model then checks all the guesses in one pass, keeping the ones it agrees with. When the guesses are good, answers come out several times faster.

What the code shows. NVIDIA is building out a newer drafter design called DSpark, first published by DeepSeek in July 2026. Its key feature is a built-in confidence score for each guess (NVIDIA NeMo docs). Low-confidence guesses can be checked carefully; high-confidence ones need less effort. Across three projects:

• TensorRT-LLM added scheduling based on that confidence score (November 3) and re-ranking on the chip itself (November 5).
• Model-Optimizer made training these drafters 3.49 times faster (October 23).
• NVIDIA is building drafters for other labs' models, including DeepSeek-V4, Kimi-K3 and MiniMax-M3.1. Its own Nemotron models already ship with three different drafters, and the official guidance recommends DSpark for NVIDIA's desktop DGX Spark (Classmethod).

Why it matters. Drafters are exactly the kind of small, fast model that suits NVIDIA's LPU. If NVIDIA supplies the drafter for a popular open model, it shapes where that model's answers get written. That is a quiet way to pull rival labs' models onto NVIDIA hardware, without the labs having to do anything.

The confidence score matters too. It gives the scheduling software a running estimate of how much checking each token needs. In other words, it lets software spend compute only where it pays off.

Signal 3: electricity becomes the budget

What the code shows. Over the summer, Dynamo gained three power controls:

• A service that enforces a power limit on every GPU in every server (June 25).
• A planner setting for a total power limit across a whole deployment, with "budget-aware scaling" that adds or removes machines to stay inside it (August 4). NVIDIA's own example sets that limit at 5,200 watts.
• A rule that a server only takes traffic once its power limit is actually in force (October 20).

Why it matters. For most large AI operators, the scarce resource is no longer chips. It is power: grid connections, permits and cooling. A data center has a fixed number of megawatts, and it cannot easily get more.

Once power is the hard limit, the question every operator asks changes. It stops being "how fast is this chip?" and becomes "how much useful output do I get from the megawatts I have?" NVIDIA's marketing already speaks this way: it claims an LPX rack paired with Vera Rubin delivers up to 35 times more throughput per megawatt than its previous generation. The code is what turns that claim into a dial operators can set.

Signal 4: the customer is an AI agent

What the code shows. Dynamo has been rebuilt around a new kind of user: the AI agent working through a long task. Recent changes:

• It recognizes requests that come from agents and records what triggered each one (August 10).
• It reads metadata from popular coding agents, including OpenAI's Codex, and can record sessions from Anthropic's Claude Code as test traffic.
• It added "session affinity": every turn of an agent's session goes back to the same machine, which still holds the session's working memory. A global version, backed by Amazon's DynamoDB database, arrived November 5.
• It tracks that working memory across thousands of slots, so it can be reused instead of rebuilt.
• NVIDIA's performance test suite for TensorRT-LLM now includes dedicated agent workloads ("AgentX") on its GB300 systems.

Why it matters. A chatbot answers one question and stops. An agent may make hundreds of model calls in one task, each carrying a growing history of everything it has done. Re-reading that history from scratch on every call is the single biggest waste in serving agents.

By keeping each session on one machine and reusing its memory, NVIDIA cuts that waste. It also changes the natural unit of business. The sensible thing to measure, and eventually price, is the agent session rather than the individual request.

Signal 5: AI agents are writing NVIDIA's software

What the code shows. In August, TensorRT-LLM brought in a framework called "agent-flow." It uses AI coding agents, Anthropic's Claude Code or OpenAI's Codex, to add support for new AI models and tune how fast they run. That job has traditionally been done by hand by specialist engineers.

More broadly, many NVIDIA changes now credit an AI co-author in the record. Share of changes crediting an AI co-author, in the peak month:

These are lower bounds. They count only changes where the AI tool is named.

Why it matters. A big part of NVIDIA's advantage has always been software: new models simply run best on NVIDIA. Keeping that true is labor-intensive, because a major new open model now arrives almost every week. Automating the work means NVIDIA can offer day-one support for every new model without growing its team at the same pace. That widens the gap with chip rivals, who must do the same work with fewer engineers.

Putting it together: the strategy

Each signal on its own looks like ordinary engineering. Together they describe one plan: sell AI agent work as a service, measured by output per megawatt, with software choosing the chips behind every session.

Each lever does one job. Session routing stops agents from paying twice for the same history. The compiler puts each piece of a model on whichever chip does it cheapest. Drafters, and their confidence scores, cut the cost of writing each answer. The power planner keeps the whole fleet inside its electricity limit. And AI-written support code makes sure every new model runs this way from day one.

Why this is hard to copy. A rival can build a faster chip. It is much harder to build all five levers, make them work together, and keep them current as new models arrive weekly. NVIDIA's real product is becoming the system, and the system is mostly software.

What NVIDIA has not said. Its public message is about the chips and racks, plus a headline efficiency number. The code suggests the bigger bet is on the orchestration layer. If it works, customers will judge NVIDIA less by any single chip and more by what a megawatt of NVIDIA infrastructure delivers for their agents.

Caveats and what to watch next

This is a reading of the direction the code points, not a confirmed NVIDIA strategy. Keep three limits in mind:

• Public code is not the whole picture. NVIDIA keeps much of its work private. We saw what it chose to build in the open.
• Some features are experimental. The piece-by-piece GPU/LPU split, for example, is labeled experimental in the code. Experimental features can be dropped.
• We read change titles, design notes and examples. We did not run the software or measure its performance.

Signals to watch over the next two quarters:

• Whether NVIDIA or its cloud partners start pricing by agent session or by task rather than by token.
• Whether the GPU/LPU split moves from "experimental" to a standard, documented option.
• How many major open models launch with an NVIDIA-made drafter on day one.
• Whether rival chip makers ship comparable power-budget and session-routing software, or stay focused on chip speed.
• Customer reports of real LPX deployments, which began shipping in the second half of 2026.

Looking ahead: NVIDIA in 2029

Up to here, everything comes straight from NVIDIA's code. What follows is opinion. If these trends hold, here is where we think NVIDIA ends up by late 2029, and what would have to go wrong for us to be wrong.

1. What NVIDIA sells: outcomes, not units

Chips and racks will still make up most of NVIDIA's revenue. But they will stop being the headline. The pitch to large customers becomes: "Give us your power envelope, and we will deliver the most agent work per megawatt."

That changes how deals are structured. Today a customer buys a count of GPUs and figures out the rest. In 2029, we expect NVIDIA's largest customers to buy a system with performance commitments attached: a guaranteed level of agent throughput for a given amount of power.

We also expect NVIDIA, or its cloud partners, to offer pricing by agent task or by session, at least for the biggest clients. The session-routing and power-budget software described above is exactly what such pricing would need. Once a provider can measure cost per session reliably, selling per session is a short step.

2. The hardware: a mix of chips, chosen by software

GPUs stay at the center of NVIDIA's data centers. Around them, we expect two or three kinds of specialized chip:

• The LPU line, for writing answers quickly.
• Probably a chip or memory tier built to store agents' working memory between steps, since keeping that memory warm is the biggest waste in serving agents.
• Possibly chips sized specifically for drafter models.

The fixed 3-to-1 GPU-to-LPU ratio will be gone. NVIDIA's compiler decides how each model is split across chip types, and that split shifts as models and workloads change. Customers mostly stop caring which chip did which part of the work, much as businesses stopped caring which physical server ran their website once the cloud arrived.

3. The moat moves up a layer

For two decades, NVIDIA's moat has been CUDA: the software layer that makes AI code run best on NVIDIA chips. Rivals could build fast chips, but developers' code was written for CUDA.

By 2029 we think the moat sits one level higher, in orchestration. That is Dynamo and whatever it grows into: the software that routes agent sessions, splits models across chips, schedules drafters and keeps the fleet inside its power budget. These pieces are deeply tied together and tuned against every major model. A rival with a faster chip would still have to rebuild the whole stack, and keep it current.

AI-written code widens this gap. As agents take over more of the routine work of supporting new models, NVIDIA can offer day-one support for every major release without hiring in proportion. Rivals with smaller teams fall further behind with each new model.

4. Quiet influence over open models

If NVIDIA keeps supplying the drafters, optimizations and launch-day support for leading open models, it becomes the default way those models get served. The lab releases the model; NVIDIA makes it fast.

We would not be surprised if major open-model labs co-design new releases with NVIDIA before launch, the way game studios work with console makers today. That gives NVIDIA early sight of where models are heading, and a say in design choices that favor its hardware.

5. What could prove this wrong

• Hyperscalers' own chips. Google, Amazon, Microsoft and Meta keep investing in their own silicon. If they also build orchestration software as good as NVIDIA's, its system advantage shrinks back to the chips alone.
• Power gets easier. If grid capacity grows faster than expected, "work per megawatt" matters less and plain cost per token matters more.
• Regulation and export controls. Trade rules have already disrupted NVIDIA's sales at times. Tighter or shifting controls could split the market by region and slow a single global system.
• A new kind of model. A design that does not benefit from drafters, or from splitting work across chip types, would blunt several of these advantages at once.
• Customer pushback on lock-in. The more NVIDIA sells a complete system, the more customers may worry about depending on one vendor. That could push them toward open alternatives, even at a performance cost.

6. Milestones that would confirm the forecast

If this forecast is right, expect to see these along the way:

1. 2027: NVIDIA's piece-by-piece GPU/LPU split ships as a standard, documented feature. At least one major cloud offers agent pricing by session or task.
2. 2027 to 2028: most major open-model launches arrive with an NVIDIA-built drafter and same-day optimized support.
3. 2028: NVIDIA reports a growing share of revenue from software and services tied to its systems, and highlights output per megawatt in earnings calls.
4. 2029: large customers routinely sign deals stated in power and throughput, not in chip counts.

The bottom line

In 2029, NVIDIA still sells silicon. But its power comes from the software that decides how every watt in an AI data center gets used. For executives, the practical lesson is to start judging AI infrastructure the way NVIDIA increasingly will: by useful agent work per megawatt, not by the spec sheet of any single chip.

Glossary

Sources

• Tom's Hardware: NVIDIA's $20 billion Groq deal produces its first chip
• Tom's Hardware: Hot Chips 2026, NVIDIA presents Groq 3 LPX architecture
• Counterpoint Research: GTC 2026, NVIDIA's LPU strategy
• Spheron: NVIDIA Groq 3 LPU explained
• NVIDIA NeMo AutoModel: DSpark documentation
• Classmethod: DSpark, DFlash and MTP compared on DGX Spark
• Code history, June to November 2026: Dynamo, TensorRT-LLM, Model-Optimizer, Megatron-LM, NeMo, CCCL, cuda-python, Warp, KAI-Scheduler, CUDA-Q