On the path to becoming a Zen Master of AI, it is dangerously easy to assume that the possibilities are infinite. They are not. The Universe has built a couple of firm boundaries into the fabric of space-time—one of which almost always manifests precisely around lunchtime.
This is the exact moment ChatGPT politely informs you that you have utterly exhausted your daily credit allowance, suggesting that your time would be far better spent extending your lunch break, perhaps until next Tuesday. The other boundary is your local computer’s processing power. While you aren’t strictly trying to calculate the Ultimate Question of Life, the Universe, and Everything on a rubber-keyed ZX Spectrum, there still comes a point where the fans emit a frantic roar, followed shortly by a desperate plea to shut down.
The second issue will be dealt with in a future post. For now, let’s take a look at token use.
The Grand Vision Meets the Token Bill
So there you sit, watching your AI agent process your latest ticket, its status reassuringly set to “Planning next moves.” What now? You’ve already scrolled through social media to discover that Aunt Jo is having a delightful time in the Bahamas while the rest of the world is rapidly spiralling toward collapse. Having exhausted your options, you decide you might as well tackle another task or—failing that—attempt the noble pursuit of self-improvement.
It was during precisely one of these moments of idle existential dread that I wondered: why not use multiple AI agents to take a raw idea and run with it?
The idea was simple: let a swarm of digital assistants chart out the plan and slice it into bite-sized pull requests. They would write the prompts, write the code, review the code, ponder what tests were necessary, write the test prompts, run the tests, fix their own mistakes, and re-review everything in a closed, beautifully self-contained loop.
It sounded magnificent on paper: a self-orchestrating assembly line of tireless digital helpers.
In reality, what I had actually an expensive loop of AI models enthusiastically high-fiving each other in a private chat room while quietly incinerating my entire monthly token allowance in approximately forty-two minutes.
When my kids were aged 2 and 4, I remember thinking how cool it was going to be when they start talking to each other - when they did start to talk to each other I was surprised at what garbage they 'd come out with. Was this my Agents. Or maybe a group of giggling hallucinating teenagers, while spending there parents cash.
Left to their own devices without strict token hygiene, Agent A writes a thousand words of speculative architecture. Agent B dutifully ingests all thousand words, adds twelve hundred words of polite praise, and suggests two lines of CSS. Agent C then re-reads the entire epic saga from the top, ponders the philosophical implications of flexbox, and submits the whole thing back to Agent A.
Before you know it, you are paying API fees for two artificial intelligences to have a polite, recursive debate about whether semicolons are truly necessary in modern JavaScript.
Where Do All the Tokens Go?
To stop your tokens vanishing faster than free pastries in an office kitchen, you have to understand the three main culprits.
The Context Snowball: Every time you reply in an ongoing chat, the model does not just read your newest message—it re-reads every word, code block, and tangent from earlier in the thread. By turn fifteen, asking “can you rename this variable?” may require sending thousands of tokens of chat history just to change x to user_id.
The Death Star Crumpet Problem: Using a top-tier reasoning model to write a regular expression, format JSON, or rename variables is the equivalent of firing up the Death Star laser just to toast a crumpet. It gets the job done, but the energy bill is catastrophic.
Monolithic Pasting: Pasting an entire repository—or even a large file—when the bug lives in a handful of lines forces the model to parse imports, comments, and boilerplate before it can focus on the actual problem.
The Zen Master’s Rules for Token Efficiency
1. Match the Tool to the Task (And Squeeze Free Allowances)
Don’t use the same AI for your project planning that you use to write blog posts or refactor code.
Creative Writing & Content: Use a model tuned for prose and conversational flow (like Gemini, which happens to come with generous free web tiers and massive context windows).
Architectural Planning: Use high-reasoning frontier models, but only for the initial blueprint.
Routine Coding & Formatting: Offload syntax checks, unit tests, and minor refactoring to fast, lightweight models—or tap into the daily free allowances across different AI platforms to stretch your paid tier further.
2. Slay the Megathread
Once an assistant completes a discrete task—such as outlining a ticket or generating a test plan—capture the final artefact, summarise the key decisions, and start a fresh thread for implementation. Do not drag every brainstorming tangent into the coding phase.
3. Constrain the Output
LLMs are naturally chatty. Unless instructed otherwise, they may deliver a 400-word introduction before handing over the two lines of code you requested. Constrain them explicitly:
“Output only the diff or changed function block—no conversational preamble.”
“Summarise the status in three bullet points, under 60 words total.”
4. Inject Context Surgically
Only provide what the model genuinely needs to solve the immediate problem:
Share the exact stack trace and the relevant lines of code, not the entire controller.
Store recurring coding standards, technology constraints, and project conventions in persistent system instructions rather than repeating them in every prompt.
5. Put Agents on a Short Leash
When experimenting with orchestration or automated feedback loops, enforce hard stop conditions:
Set a maximum iteration count (e.g., two review passes max).
Require structured output (like JSON or strict Markdown) so agents cannot wander into philosophical soliloquies.
Stop the workflow as soon as tests pass and acceptance criteria are satisfied.
Summary: Do Not Summon HAL for a Pocket Calculator
Becoming token-efficient is not about being stingy; it is about clarity, speed, and focus. When you prune inputs, constrain outputs, and choose the lightest tool capable of doing the job, your workflows run faster, cost less, and are far less likely to hallucinate.
Best of all, you might actually make it past lunchtime without an AI politely advising you to step away from the keyboard and go surfing ( although that’s not actually that bad an outcome ) .
No comments:
Post a Comment