In Defense of Tokenmaxxing // BRXND Dispatch vol 123
Why I think the moral panic around token leaderboards is the wrong bogeyman.
Another week, another article about the dangers of tokenmaxxing. This one comes courtesy of the New York Times and even introduces “tokenminning” as the antidote to the old way of thinking. But once again, the article’s example companies are countable on one hand: Meta, Amazon, and Uber. Reading these articles, it’s easy to believe that every big company faces a token reckoning, but I believe (and have decent evidence from the hundreds of conversations I’ve had over the last six months with Fortune 500 companies) that the vast majority of large enterprises are still way on the wrong side of the ledger when it comes to driving AI adoption.
BRXND NYC is coming up on 11/5, and early bird tickets are currently $749. Get yours early, we will sell out.
So while clearly token leaderboards don’t represent the long-term answer, I want to offer a defense of the practice of tokenmaxxing (and token maximalism generally), as by far the best approach to learning how to get the most out of these models. The thing I think most of these articles miss is that when it comes to driving change inside organizations (and society), people, not technology, are the bottleneck. If you fail to address that, then nothing else really matters.
One of the questions I get most often from customers—along with friends and family—is “what can I do to catch up in AI?” My answer for the last four years has basically been the same: get yourself the most expensive subscription you can and try to use as much of it as possible. It used to be ChatGPT; now it’s Codex and Claude Code/Cowork, but the advice stays the same. Specifically, I suggest you use it so much that you max out your token budget on their biggest plans.
To give a sense of just how many tokens that is, here’s a recent SemiAnalysis estimate on what you get out of both companies’ most expensive plans (the “max possible spend” number is their best guess for the value you can get out of the plan if you were to use it to its fully allowable extent):
If you’re really trying to wrap your head around AI, nothing replaces the hands-on keyboard experience. At the very first BRXND in 2023, I talked about building AI fingerspitzgefühl (fingertip feeling), and despite the years in between, I still think it’s the most critical piece of the puzzle, and I credit quite a bit of my own understanding and success with AI to just having been lucky enough to be able to put in a thousand-plus hours in 2022.
The reason nothing replaces hands-on also hits on a big misconception about AI: that you can just hand things over to it. While Fable and Sol get us closer than we’ve been before, for the most part, when you watch people who get the most out of these systems, you see them subtly steering them to success. Oftentimes it’s such small adjustments that even they don’t notice. It can be the way they ask the question or when they choose to interrupt, but the outcome is that by making one- or two-degree adjustments, they are experts at keeping the harness on track. The most insidious AI failure case isn’t it writing bad code or a bad doc, it’s fundamentally doing the wrong work.
Which brings me back to “tokenmaxxing,” the pejorative term people have come to use for companies that encourage their employees to use AI by any means necessary and often track them in leaderboards. Here’s my general view on the practice:
When people talk about tokenmaxxing, they’re mostly talking about a) the token leaderboards and b) a few reports from companies like Uber of massive AI spend.
I obviously don’t think leaderboards are the best long-term solution, but I’m not sure they’re the worst short-term solution (more on that in a bit).
While there are a few stories of massively inflated AI spending, I’d venture to guess that the vast majority (90%+) of the Fortune 500 should be spending far more on tokens than they do today.
When we hear about these companies spending real money on tokens, it’s happening via harnesses like Claude Code/Cowork and Codex, not ChatGPT, Claude, or Copilot. More importantly, that inflated spend mostly isn’t coming from tools built with AI, partly because those tools, by their nature, are much easier to optimize spend around.
The vast majority of the Fortune 500 doesn’t give their employee base access to platforms like Codex and Claude Code/Cowork.
Most people who work in large companies don’t want to change the way they work.
That last point matters a lot. If you’re trying to turn a massive organization, you are usually better off over-rotating than under-rotating. This is also why, in the short term, a blunt instrument like a leaderboard might actually work in motivating people to dive in. Obviously, in the long term, you want to see value from these tokens, but remember that the average Fortune 500 employee is currently using significantly fewer tokens than they should. Participation before optimization.
The conclusion of all this seems very clear to me: for a small group of technology companies, getting past tokenmaxxing really matters. They’ve convinced a large enough swath of employees to integrate these tools into their lives that they’re now ready to graduate to thinking about the return on those tokens. But for the other estimated 90-95% of large enterprises, the greater concern is that they’re currently token minimalists in a world where the return on knowledge and capability can be exponential. In part, this is a function of point number five, as most of these companies still haven’t opened access to the very best AI tooling (and I’m not quite sure what the path to that looks like). Like all things AI, the biggest mistake pundits make is forgetting that people are the bottleneck. This transition, despite the breakneck speed, will still take a long time to fully sweep through the corporate world, and we should all be wary of claims that we’ve entered a new phase.
Since you’ve made it to the bottom, one more reminder that BRXND NYC is coming up on 11/5, and early bird tickets are currently $749. Get yours early, we will sell out.