Enterprises Are Buying AI Coding Tools Faster Than They Can Prove They Work

Three things happened inside a single 48-hour window this week, and none of them were coordinated, which is exactly what makes them worth reading together. Amazon told its software engineers to stop treating AI token counts as a badge of productivity. Anthropic launched a suite of tools designed to tell enterprise customers whether their Claude Code spending is actually buying anything. And engineering-analytics firm Jellyfish published research showing that AI coding spend per engineer has roughly tripled over six months without a corresponding shift in what those engineers are doing with their time. Read separately, each is a minor trade story. Read together, they describe an industry that adopted AI coding agents far faster than it built the instruments to tell whether the money is working.

Amazon's leaderboard problem

The clearest signal came from inside Amazon. In a memo reported by Fortune, Terk Terkowitz, a vice president and distinguished engineer at Worldwide Amazon Stores, told engineers that counting tokens consumed or lines of code generated is a poor way to measure productivity, and that turning those numbers into explicit targets backfires: it inflates computing costs and rewards busywork over judgment. Terkowitz's framing was blunt. Metrics built around raw AI usage, he wrote, 'are disconnected from what matters,' and chasing them — what Amazon engineers had taken to calling 'tokenmaxxing' — 'is at odds with frugality,' one of the company's own long-standing leadership principles.

The memo wasn't abstract. It was a response to a real, specific failure mode: Amazon had been running an internal, unofficial leaderboard called Kirorank that ranked engineers by their AI usage. According to Fortune's reporting, some engineers started running AI agents against trivial or meaningless tasks purely to climb the rankings, burning compute for no productive purpose. Amazon shut the leaderboard down back in May. The October memo is Amazon's attempt to codify the lesson under a new framing of what it calls the 'era of the Renaissance Developer' — engineers who pair AI with their own judgment rather than defer to it. Tellingly, Terkowitz didn't throw out usage metrics entirely — he said they still 'correlate well with the outputs that matter' and told managers to compare their most and least active AI users, the 90th and 10th percentiles, rather than chase a single team-wide number. The point wasn't that measurement is useless. It's that the measurement Amazon had been running on was actively gameable, and gaming it was expensive.

What Amazon is now asking for instead is less a new metric than a change in posture: letting engineers push back when AI tooling isn't actually speeding up their work, rather than treating usage itself as the deliverable.

The industry catches up to the same question

Amazon's internal reckoning landed the same week Anthropic moved to address a version of the same problem from the vendor side. According to TechCrunch, Anthropic rolled out new enterprise reporting tools that let customers' managers review a team's AI-assisted work over a given stretch of time — what got produced, what it cost, and where the workflow broke down — drawing on usage analytics and employee surveys rather than token counts alone. Anthropic is pairing that with pooled group budgets, so organizations can manage spending across a team rather than guessing per-seat. The company says it won't use the reporting data to train its models by default, an answer to a predictable enterprise objection. The clear subtext, as TechCrunch frames it, is return on investment: large customers are asking Anthropic to prove the spend is worth it, because right now most of them can't prove it themselves.

The research backing up that anxiety came from Jellyfish, the engineering-analytics firm, in the same article and in its own published research. Jellyfish's figures, drawn from roughly 276,000 of its customers, show Claude Code and other Anthropic-model tools now account for close to a third of all proposed code changes submitted for review, with related spending up tenfold since the start of the year — now equal to about 2% of total engineering cost. Jellyfish chief scientist Nicholas Arcolano put the growth rate in blunt budgetary terms: 'Even if you're saying 2% is still small, that 10x growth is growth that most CFOs didn't plan for.' Jellyfish's AI product lead, Nikolas Albarran, added that Anthropic's traction with enterprises owes less to novelty than to incumbency — the company 'was just there with great enterprise motions,' and that once a team has adopted Claude Code, 'it's going to be harder and harder to switch.'

Jellyfish's separate research resource adds the number that ties the whole week together: an analysis of more than 878,000 developer-weeks between March and August 2026 found that AI spend per engineer roughly tripled over that span — while what engineers actually spent their time working on barely moved. The firm's own framing is a warning aimed at 2027 budget season: a flat monthly AI line item is 'the riskiest assumption in your plan,' because the spend compounds whether or not output is compounding with it. The same research found that larger engineering teams hit a ceiling on the benefit of added AI spend at roughly half the usage level smaller teams do, while organizations where adoption is daily and consistent merge roughly twice as many pull requests per engineer as those with patchier adoption.

Spend first, instrumentation second

None of these three threads describes an AI coding backlash. Amazon isn't retreating from AI tooling, Anthropic is selling more of it with better reporting attached, and Jellyfish's own data shows adoption climbing, not stalling. What they describe instead is a sequencing problem: enterprises bought agentic coding tools and scaled spending on them well before they had reliable ways to tell whether that spending was changing what engineers produced. Amazon built an internal leaderboard that measured the wrong thing and got gamed within months. Jellyfish's customers tripled their AI coding budgets without a matching shift in work allocation. And Anthropic, reading the same signal from its own customer base, is now selling the instrumentation that should have shipped alongside the spend in the first place. The lesson landing across all three, independently, is the same one: counting tokens was never the hard part. Knowing what they bought is.