I Stopped Using the Smartest AI Models for Coding — Programming Became Faster and Cheaper
For a while, I thought that the future of AI-assisted programming was obvious: use the smartest model available, give it the largest possible context window, and let the AI agent handle the rest.
I no longer think that way.
During my vacation, I had time to step back and experiment with different AI coding models, agents, APIs, and development tools. What I discovered was surprisingly simple:
The smartest AI model is not necessarily the most useful model for programming.
For many everyday software-development tasks, a smaller and cheaper model can be more efficient, especially when the developer already understands the codebase and knows how to describe the problem clearly.
And once model intelligence reaches a certain level, another factor starts becoming much more important:
speed.
The next major competition in AI coding may not be about who has the largest model or the biggest context window. It may be about who can give developers a useful result in the shortest possible time.
The Smartest AI Model Is Not Always the Best Coding Model
One of the biggest changes in my approach to AI programming has been realizing that raw model intelligence is only one part of the equation.
There are several factors that determine how useful an AI coding system actually is:
- Model intelligence
- Cost per task
- Number of tokens consumed
- Generation speed
- Tool efficiency
- Context management
- Reliability
- Quality of the developer’s instructions
- The coding agent or harness surrounding the model
This is why benchmarks such as the Artificial Analysis Intelligence Index can be interesting, but they should not be treated as the only measure of real-world productivity.
OpenAI’s current GPT-5.6 family, for example, includes multiple model tiers designed to balance capability, speed, and cost. OpenAI also reports that GPT-5.6 Luna is substantially cheaper than its larger models while remaining highly capable on coding and agentic tasks.
For a professional developer, however, the important question is not simply:
Which model is the smartest?
The more useful question is:
Which model completes my actual task with the least time, cost, and unnecessary context?
That is a very different question.
Cost Per Task Matters More Than Model Price
AI model pricing is usually discussed in terms of input and output tokens.
But developers do not actually buy tokens.
They buy completed tasks.
Imagine two models:
- Model A costs $1 per million tokens.
- Model B costs $10 per million tokens.
At first glance, Model A looks dramatically cheaper.
But suppose Model A needs 200,000 tokens and several additional attempts to complete a task, while Model B solves it in 30,000 tokens.
The cheaper model is no longer necessarily cheaper in practice.
This is why I find the concept of Cost per Task much more interesting than simply comparing token prices.
A better calculation is:
Cost per Task = Total Tokens Used × Token Price × Number of Attempts
The equation is simplified, but the idea is important.
If a cheaper model solves a task correctly on the first attempt, it can be dramatically more economical than a more expensive model that repeatedly reasons through the same problem.
Good Prompts Make Smaller Models Much More Useful
There is another factor that is easy to underestimate: the quality of the developer’s instructions.
There is a huge difference between asking:
“Fix this application.”
and saying:
“The authentication middleware returns HTTP 401 after token refresh. Inspect
src/auth/refresh.ts, identify why the refreshed token is not propagated to the request interceptor, implement the smallest backward-compatible fix, run the authentication tests, and report the files changed.”
The second prompt contains much of the problem definition already.
A capable developer knows:
- what the system is supposed to do;
- where the problem probably exists;
- what constraints matter;
- what should not be changed;
- how success should be verified.
This dramatically reduces the amount of reasoning the AI has to perform.
In other words, developer expertise can compensate for model size.
This is one reason experienced programmers may benefit differently from AI coding agents than beginners do.
A beginner may need the model to discover the entire problem.
An experienced developer can often give the model a precise problem definition and let it execute.
Smaller Frontier Models Are Becoming Good Enough for Many Coding Tasks
This is the part of the AI industry that I find most interesting.
The gap between the largest frontier models and smaller models is no longer as important for every programming task.
OpenAI’s GPT-5.6 documentation, for example, shows that its smaller Luna and Terra models remain highly capable across coding and agentic evaluations while costing considerably less than the flagship model.
That does not mean smaller models are universally better.
Complex architecture decisions, difficult debugging, unfamiliar codebases, security-sensitive work, and long-horizon reasoning can still benefit from more capable models.
But for everyday tasks such as:
- writing functions;
- modifying existing code;
- fixing straightforward bugs;
- writing tests;
- refactoring small components;
- updating configuration;
- explaining code;
- generating scripts;
- performing repetitive repository changes;
the difference can become much less important.
At that point, spending several times more for marginally better reasoning may not make economic sense.
The New Bottleneck: AI Coding Speed
Once models become capable enough, another problem becomes obvious.
They can be too slow.
This is especially noticeable when using autonomous coding agents.
You give an agent a task.
It reads the repository.
It searches for relevant files.
It analyzes the code.
It plans the changes.
It edits files.
It runs tests.
It reads the results.
It makes additional changes.
Then it repeats the cycle.
A five-minute wait can feel acceptable once.
But if you work with AI agents all day, waiting five to fifteen minutes for every substantial task becomes a serious productivity problem.
This is where tokens per second starts becoming a meaningful developer metric.
Why Tokens Per Second Could Become the Next AI Benchmark
Imagine two models that can both solve your programming task successfully.
Model A takes five minutes.
Model B takes forty seconds.
Even if Model A is slightly more intelligent, the second model may be more useful for your workflow.
This changes the economics of AI development.
A fast model lets you:
- Start a task.
- Receive the result quickly.
- Review the changes.
- Correct the requirements if necessary.
- Run another iteration.
- Continue programming.
The development loop becomes much tighter.
This is particularly important for agentic coding, where a model may make dozens of tool calls during a single task.
The faster each step happens, the faster the entire development cycle becomes.
DeepSeek and the Rise of High-Speed AI Coding
This is one area where Chinese AI companies have become particularly interesting.
DeepSeek’s current API platform provides the V4 family, including DeepSeek-V4.1-Flash, and the company explicitly positions its models for agentic workflows and developer applications.
The official DeepSeek API also supports integration with coding tools and agent frameworks, including compatibility with OpenAI-style APIs.
This makes the model layer increasingly interchangeable.
A developer does not necessarily need to rebuild an entire workflow just because the underlying model changes.
That is an important development.
The future coding stack may increasingly look like:
Editor → Agent Harness → Model Provider → Tools → Repository
rather than a single application controlling the entire experience.
The Coding Agent Is Becoming as Important as the Model
This leads to another discovery.
The model itself is only one part of an AI coding system.
The surrounding agent harness determines:
- what context the model receives;
- which tools it can use;
- how files are selected;
- how commands are executed;
- how previous interactions are stored;
- how errors are handled;
- how much system information is injected into every request.
And this can have a huge impact on performance.
A brilliant model surrounded by thousands of unnecessary tokens can sometimes be less useful than a slightly weaker model operating inside a clean, focused environment.
Why I Started Using Pi Coding Agent
This is why I became interested in Pi.
Pi describes itself as a minimal agent harness rather than a feature-heavy coding environment. Its core intentionally stays small and can be extended through TypeScript extensions, skills, prompt templates, themes, and packages.
Its philosophy is very different from the idea that an AI coding tool needs to contain every possible feature.
Pi’s core tools are deliberately simple:
readwriteeditbash
Additional capabilities can be added when they are actually needed.
This approach makes sense to me.
If the model already knows how to program, perhaps the agent should not constantly overwhelm it with unnecessary abstractions.
The goal should be:
Give the model the right context, the right tools, and the shortest path to the task.
Nothing more.
Context Is Becoming a Hidden Cost
There is another problem that is easy to miss.
Every additional instruction, tool description, repository file, previous conversation, and framework abstraction contributes to the context.
And context is not free.
Suppose I ask an agent:
“Hello.”
A human needs almost no information to answer.
But a coding agent may load:
- system instructions;
- tool definitions;
- repository instructions;
- project metadata;
- environment information;
- previous conversation;
- available skills;
- tool documentation;
- additional agent configuration.
Suddenly, a simple interaction can consume thousands of tokens.
The problem becomes much more serious when the agent is working on a large repository.
The Real Problem With Huge Context Windows
For years, the AI industry competed over context length.
First it was tens of thousands of tokens.
Then hundreds of thousands.
Then one million tokens.
And naturally the question became:
“What comes after one million?”
Two million?
Five million?
Ten million?
I am not convinced that continuously increasing the context window is the answer.
At some point, the problem stops being how much information can the model see?
The problem becomes:
Which information should the model remember?
That is a fundamentally different problem.
The Missing Piece: Long-Term Memory for Coding Agents
Imagine working on the same codebase with an AI agent for six months.
Today, the agent may repeatedly need to rediscover:
- where the authentication system lives;
- how the database layer is structured;
- which conventions the project follows;
- how tests are organized;
- which APIs are considered legacy;
- why certain architectural decisions were made;
- which files should never be modified;
- how deployments work.
Every new session can require part of this information to be reconstructed.
That is inefficient.
The ideal system would gradually build a persistent representation of the project.
Instead of starting from zero, the agent would already know:
“I have worked with this repository before.”
Stateful LLMs Could Change AI Programming
This is where the idea of stateful AI agents becomes interesting.
Today’s LLM applications are often effectively stateless between requests. The client or application has to provide the relevant conversation or context again.
For example, DeepSeek’s Responses API documentation explicitly describes the API as stateless and notes that clients need to provide conversation history for multi-turn interactions.
That architecture works.
But it is not necessarily the ideal architecture for long-running software development.
A truly persistent coding agent could maintain a long-term knowledge layer containing:
- repository architecture;
- developer preferences;
- historical decisions;
- previous bugs;
- test behavior;
- deployment knowledge;
- project conventions;
- frequently modified components.
The context window would still exist.
But the agent would no longer depend on repeatedly stuffing the entire history into that window.
Context Windows Could Become More Like Working Memory
This suggests a different architecture.
Instead of:
Everything → Context Window → Model
we could eventually have:
Long-Term Memory → Retrieval → Relevant Working Context → Model
The model receives only what it needs for the current task.
This could dramatically reduce unnecessary token consumption.
It could also make long-running coding agents much more practical.
A developer might work with an AI agent for months without requiring the agent to rediscover the entire architecture every morning.
Zed: When the IDE Becomes Less Important
My own programming workflow has also changed.
As AI agents became more capable, I started using fewer traditional IDE features.
I do not necessarily need a massive IDE to write every line of code manually when an agent can perform many of the repetitive operations.
What I actually need is often much simpler:
- fast source navigation;
- a good file tree;
- Git integration;
- excellent diff visualization;
- keyboard shortcuts;
- fast search;
- a responsive interface.
This is one reason I moved toward Zed.
Zed AI Code Editor is built around speed and AI-assisted development, and its current tooling allows developers to work with agents, review changes through diffs, and connect different agent systems.
Zed also works without an AI subscription, while developers can connect their own API keys or use external agents.
That makes it particularly interesting in a world where the editor and the AI model are becoming increasingly independent.
The Future May Be About Composable AI Development Stacks
I think we are moving toward a more modular development environment.
Instead of paying for one enormous application that bundles everything together, developers may choose individual components:
Editor
Zed, VS Code, Neovim, or another lightweight editor.
Agent Harness
Pi or another customizable coding agent.
Model
GPT, DeepSeek, Claude, Gemini, or another provider.
Tools
Git, shell commands, test runners, databases, browsers, deployment systems, and APIs.
Memory
A persistent project knowledge layer.
Infrastructure
The fastest and most cost-efficient model-serving provider available.
This architecture gives developers much more control.
It also means that the best tool today may not be the best tool six months from now.
AI Coding Is Moving From “Smarter” to “Faster”
I think the AI industry is entering a new phase.
The first phase was:
Can AI write code?
Then:
Can AI write useful code?
Then:
Can AI solve complete software-engineering tasks?
Now we are approaching another question:
How quickly can AI complete those tasks?
That changes everything.
If an agent can complete a task in 20 seconds instead of 10 minutes, the interaction between developer and AI becomes fundamentally different.
You no longer have:
Give AI task → wait → review.
You start to have:
Think → ask → receive → review → correct → repeat.
That is much closer to the speed of human thought.
The Importance of Developer Skill Is Not Disappearing
There is an important paradox here.
AI makes programming easier.
But better programmers may benefit even more from AI.
Why?
Because an experienced developer knows how to:
- define a problem precisely;
- understand architecture;
- identify bad assumptions;
- inspect generated code;
- design good tests;
- recognize security problems;
- control scope;
- provide useful context;
- reject unnecessary changes.
AI does not eliminate these skills.
In many cases, it amplifies them.
A developer who knows exactly what needs to change can use a smaller and cheaper model very efficiently.
A developer who does not understand the system may compensate by repeatedly asking increasingly powerful models to figure everything out.
That can become expensive very quickly.
What Happens Next?
I do not think the next major breakthrough in AI coding necessarily has to be a model with an enormous context window.
We may instead see improvements in several other areas:
1. Faster inference
More tokens per second means shorter development cycles.
2. Better coding-specific models
Models increasingly optimized for software engineering rather than generic conversation.
3. Smaller models with frontier-level performance
This would reduce the cost of running AI agents continuously.
4. Better agent harnesses
Minimal systems that give models exactly the tools and context they need.
5. Persistent memory
Agents that remember projects instead of repeatedly rediscovering them.
6. Better context selection
The ability to identify exactly which pieces of a repository matter for a task.
7. More modular development environments
Editors, models, agents, and tools becoming interchangeable components.
8. Local and hybrid AI
Some workloads running locally while more difficult tasks are delegated to remote frontier models.
The End of the “Bigger Context Is Always Better” Era?
Maybe.
A one-million-token context window is impressive.
But if an agent needs to repeatedly read hundreds of thousands of tokens just to remember how a project works, that is not necessarily an ideal architecture.
A better system might remember the important parts permanently and retrieve only the information relevant to the current problem.
That would make the effective memory of the agent much larger without forcing every request to carry the entire project history.
The future may therefore be less about:
“How large is your context window?”
and more about:
“How well does your AI remember?”
My New AI Coding Stack
My current philosophy has become much simpler:
Use the smallest model that reliably completes the task.
Then optimize the rest of the stack around it.
That means:
- precise prompts;
- minimal context;
- fast inference;
- lightweight tooling;
- efficient agent harnesses;
- persistent project knowledge;
- human review;
- escalation to a stronger model only when necessary.
For networking, infrastructure, operating systems, and developer workflows, I also find it useful to keep the AI’s output grounded in the actual technical environment rather than relying entirely on generic model knowledge. For example, a developer working with network architecture can combine an AI coding workflow with foundational resources such as a complete TCP/IP model guide on CorgiNetwork, a DNS fundamentals guide, or the site’s servers and infrastructure guide.
Similarly, AI-generated software that interacts with production systems should be treated as engineering work rather than blindly accepted output. Understanding cybersecurity fundamentals is still essential when AI agents can read files, execute commands, modify applications, and interact with infrastructure.
Conclusion: The AI Coding Race Is Changing
I used to think that better programming with AI meant using the smartest model available.
Now I think the equation is much more complicated.
The real objective is not maximum intelligence.
It is maximum useful work per unit of time and cost.
A smaller model that completes a task in 30 seconds can sometimes be more valuable than a significantly more expensive model that takes several minutes.
A minimal coding harness can sometimes be more effective than a feature-heavy agent if it keeps the context focused.
A fast editor can matter more than another layer of IDE functionality.
And a persistent memory system could eventually matter more than another million tokens of context.
The next generation of AI coding tools may therefore be defined by three things:
speed, efficiency, and memory.
The smartest AI is impressive.
But the AI that understands your project, remembers what it learned, uses only the tools it needs, and gives you the result almost immediately?
That could be the real future of software development.
So what does your AI coding stack look like today?
Are you using a full-featured coding agent, a minimal terminal harness, an AI-powered IDE, or a combination of several tools?
And more importantly:
What do you think comes next — faster models, better memory, better agents, or something completely different?