Let Your Agent Choose the Right AI Model and Effort
The newest AI model at maximum effort is not always the right choice. Let current documentation, task difficulty, and cost guide the decision.
A new model arrives, tops a leaderboard, and makes the default choice look obvious: select the latest release, turn reasoning effort to maximum, and assume the work will improve. That is convenient, but it is not how providers document their own models.
OpenAI’s model guidance recommends comparing accuracy, token use, and end-to-end latency, while adjusting reasoning effort to the task. Google similarly says that everyday work may need less thinking than difficult, multi-step problems. “Newest at maximum” is not a selection strategy. It is the absence of one.
Your code sets the ceiling
A narrow change in a familiar framework, backed by clear tests, may already be within an older model’s reliable range. Once the model can understand the relevant code and satisfy the checks, extra capability has little room to show. More reasoning can simply add tokens, latency, and opportunities to wander.
The latest model earns its premium when the task actually demands it: unfamiliar architecture, a large blast radius, ambiguous failures, weak verification, or consequences that make a plausible-but-wrong answer expensive. Model and effort should rise with the work’s difficulty and risk, not with the date on a release announcement.
Older does not mean obsolete. A supported older model may have predictable behavior, established prompts, lower latency, or better economics for your workload. A provider may also lower its price after demand shifts to a newer premium model, although that is an opportunity, not a promise. Support status still matters: providers also deprecate and retire models, so the current lifecycle documentation must outrank habit.
A benchmark is evidence, not a verdict
The chart makes the price problem visible. In its September 3, 2026 snapshot, Gemini 3.8 Flash High, GPT-6 Astra xHigh, and Claude Opus 5 Max each round to a 74% score, yet their reported average costs are $2.36, $6.52, and $11.84 per task. Similar headline performance spans roughly fivefold in price.
That does not make the benchmark useless. It shows why a benchmark becomes a “trust me bro” metric when someone treats its ranking as a universal answer. DeepSWE measures 113 tasks through one agent setup and one evaluation method. The leading uncertainty ranges overlap, individual task results changed sharply between v1 and v1.1, prices describe that run, and wall-clock time is not reported. None of it knows your repository, tests, subscription, review burden, or cost of failure.
Use leaderboards to form a shortlist. Let the agent choose from that shortlist using the task in front of it.
Give the decision to the agent
An agent can inspect the work, read the provider’s current guidance, check which models and effort levels its client actually exposes, and choose the least costly, least latent option likely to succeed. It can escalate for difficult or high-risk work and step back down for routine edits. That is more useful than preserving a static favorite.
The public model-effort-recommendation skill already implements this approach across multiple providers and agent clients. It targets official documentation, validates runtime model availability, refreshes its guidance on a 24-hour cycle, and retains a last-known-good cache if a refresh fails. Point your agent at the repository and ask it to install or implement the skill instead of hard-coding “use the latest model.”
