A year ago, choosing an AI model was mostly a quality decision.

Today, it is increasingly a resource-allocation decision.

The model landscape has fragmented into distinct layers of intelligence. At the top are frontier systems that can handle open-ended research, system architecture and difficult multi-step reasoning. A second tier of highly capable general-purpose models can already execute substantial engineering, research and design work. Beneath them is a rapidly improving pool of inexpensive models that are more than good enough for information collection, extraction, routine coding, transformation and many bounded analytical tasks.

For a growing share of work, paying for more intelligence does not materially improve the outcome.

Yet most AI workflows still behave as if one model should do almost everything.

That mismatch is becoming expensive.

The next important abstraction in AI may not be a better model. It may be a scheduler that allocates the minimum intelligence required to achieve the desired outcome.

We hit the problem before we had a name for it

Our own research and engineering workflow currently spans multiple model providers and multiple capability tiers.

The pattern became increasingly obvious.

Some tasks genuinely benefit from frontier intelligence:

A much larger set of tasks does not.

Routine searches, information extraction, formatting, repetitive coding, document transformation and many bounded subtasks can now be handled extremely well by much cheaper models.

And then there is a middle layer: substantial engineering and analysis that needs a strong model, but not necessarily the most expensive model available.

The problem is that today's agent products do not allocate these tiers particularly well.

In our own usage, Claude's delegation behavior feels more mature than Codex's, but neither product is sufficiently cost-aware for the way we want to work. The system still tends to inherit the model selected for the parent workflow rather than continuously asking whether the marginal intelligence is worth the marginal cost.

So a premium orchestrator can easily spend premium tokens on tasks for which cheaper intelligence would produce the same useful result.

At light usage, this barely matters.

At heavy usage, it becomes a line item.

The industry has already started solving the same problem

This is no longer just a power-user complaint.

Microsoft Foundry's Model Router explicitly separates workloads by complexity. Simple factual lookups can go to inexpensive models; tool orchestration can go to a mid-tier model; research synthesis and difficult multi-step reasoning can escalate to frontier models.

Microsoft says simple agent interactions typically represent 50–60% of traffic and can be routed to materially cheaper models while maintaining equivalent quality on those tasks.

AWS has reached a similar conclusion with Bedrock Intelligent Prompt Routing.

On one internal dataset, AWS reported 48% cost savings when routing between Claude 3 Haiku and Claude 3.5 Sonnet while maintaining Sonnet-level response quality. Replacing Haiku with Claude 3.5 Haiku increased the reported savings to 56%.

Microsoft has separately reported up to 60% cost savings in tests of its model router versus direct GPT-4.1 usage at similar accuracy.

These results are not directly comparable. The workloads, model pools and evaluation methods differ.

But the magnitude matters.

Model routing is not a rounding-error optimization.

For the right workload, it can materially change inference economics.

The wrong answer can be too much orchestration

Once teams discover this problem, the obvious response is to build a sophisticated multi-agent system.

We tried versions of that too.

There are increasingly capable frameworks that create planners, supervisors, researchers, reviewers, coding agents and specialized workers, sometimes across multiple model providers.

They can be impressive.

They can also introduce a coordination tax.

A planner interprets the task.

A supervisor rewrites it.

Workers receive duplicated context.

Workers send status updates.

A reviewer re-reads the output.

Another model synthesizes the review.

The workflow becomes sophisticated — but the sophistication itself consumes tokens.

Anthropic now explicitly warns about a version of this problem in its own guidance: powerful Claude models can overuse subagents for tasks where a direct operation would have been faster and sufficient.

The analogy to organizations is useful.

A company can become less productive by adding too much management.

So can an AI system.

Our conclusion was counterintuitive:

Simple but enough is often better than general and sophisticated.

Our workflow is deliberately boring

We do not try to build a universal AI organization.

We classify work into broad capability tiers and keep the communication bridge small.

Tier 3 — cheap execution intelligence

Use by default for high-volume bounded work:

Tier 2 — strong general intelligence

Escalate when the task requires more context or stronger reasoning:

Tier 1 — frontier intelligence

Reserve for tasks where additional intelligence actually changes the result:

The communication protocol is intentionally small.

A worker receives the minimum context needed.

It returns a compact artifact.

The artifact is evaluated.

If it is good enough, the process stops.

If it is not, the task escalates.

Cheap model → evaluate → stop

If uncertain or inadequate → stronger model → evaluate → stop

Only then → frontier model

This is less cinematic than a room full of autonomous agents.

It is also much easier to reason about economically.

The key metric is not token price

A cheap model that fails three times is not cheap.

A frontier model that solves a critical problem in one pass may be extremely cheap relative to the value produced.

The economic objective is therefore not:

minimum token cost

It is:

minimum cost per successful outcome subject to a quality and risk constraint.

That changes how the system should be designed.

A mature intelligence scheduler eventually decides not only the model, but also:

This starts to resemble the way cloud infrastructure allocates compute.

We no longer ask which server is "best."

We specify latency, reliability, cost and performance requirements, then let infrastructure allocate resources.

AI is heading toward the same abstraction.

The real moat may not be the router

Generic routing itself is unlikely to remain scarce.

AWS already offers it.

Microsoft already offers it.

Cloudflare's AI Gateway now supports dynamic routing, budget limits, provider/model fallbacks and routing based on request metadata.

Basic routing will increasingly become infrastructure.

The more interesting asset is the data generated by routing.

Every task creates a record:

task type → model → context → cost → latency → output → evaluation

Over time, a company learns which models reliably perform which parts of its own workflow.

That knowledge is highly domain-specific.

A generic cloud router can know that a prompt appears simple.

It does not automatically know that:

That mapping is proprietary operational knowledge.

The router can remain simple. The evaluation history becomes valuable.

This is where we think a real organizational moat can emerge.

A new enterprise control plane

Once AI spending becomes material, routing is only one component of the stack.

Companies also need:

Cloudflare increasingly describes AI Gateway as a control plane rather than merely an API proxy. Its current product supports dynamic routing, spend limits, fallbacks, cost tracking and policies across model providers.

Datadog's Agent Observability similarly calculates estimated LLM cost at the request, model and application level and currently supports cost tracking across 800+ models.

These products point toward a broader category:

AI FinOps + AI control plane.

The eventual KPI is not "how many tokens did we use?"

It is closer to:

What did this workflow accomplish, what intelligence did it consume, and was that the cheapest safe path to the result?

The market impact may be surprisingly quiet

This shift does not necessarily create a single giant "model routing company."

The larger impact may appear inside other businesses.

1. AI software gross margins

Two AI applications can produce nearly identical user experiences while having radically different cost structures.

One sends every task to its strongest model.

Another learns that 55% of its workload can use a cheap model, 35% needs a strong general model, and only 10% requires frontier intelligence.

If product quality is comparable, the second company has a structural gross-margin advantage.

This means orchestration quality may eventually become part of software financial quality.

Investors may need to care about metrics such as:

These metrics are largely invisible in today's financial reporting.

They may not remain invisible forever.

2. Hyperscalers become intelligence exchanges

AWS, Azure and Google Cloud have an unusual advantage.

They do not need to predict which single model wins.

They can host many models and own the layer that allocates traffic among them.

Microsoft's router already treats simple, medium and frontier workloads differently. Cloud platforms can increasingly make model selection disappear behind one endpoint.

This moves hyperscalers toward a familiar role:

abstract the underlying resource, schedule it efficiently, and collect economics across the entire pool.

They did it with servers.

They can do it with intelligence.

Importantly, routing-driven efficiency is not necessarily negative for aggregate inference demand.

If a workflow becomes 50% cheaper, companies can spend the savings on more agents, more automated research, more software actions and more machine-to-machine inference.

Lower cost per task can increase the number of tasks dramatically.

3. Frontier models can become the escalation layer

Routing looks superficially negative for premium-model vendors.

If half of requests move to cheaper models, the frontier provider loses token share.

But token share is not necessarily the right measure.

Frontier models may increasingly become something closer to scarce senior intelligence.

They receive:

In that world, the frontier model does not need every token.

It needs the economically important escalations.

At the opposite end, cheap models can capture enormous execution volume.

That creates a potential barbell:

massive cheap-model volume + smaller but extremely valuable frontier usage.

The strategically awkward layer may be models that are too expensive to dominate volume but not differentiated enough to own escalation.

4. Gateways, observability and security become more valuable

A multi-model world is operationally harder than a single-model world.

Companies need to answer:

That complexity increases the value of gateways, observability, identity, policy and AI security.

This is why products such as Cloudflare AI Gateway and Datadog Agent Observability are strategically interesting even if routing itself becomes commoditized.

They monetize the complexity created by model heterogeneity.

AI also changes the build-vs-buy equation

There is another effect we think is underappreciated.

Historically, internal software was expensive.

A team had to justify engineering time, maintenance, deployment and long-term ownership.

That favored generalized SaaS.

But AI has reduced the cost of building small internal tools so dramatically that the old trade-off is changing.

Our orchestration workflow is a good example.

We did not need a universal platform capable of handling every company's multi-agent architecture.

We needed a narrow tool that understood our tasks.

Building it was cheap.

Changing it is cheap.

If the model landscape shifts next month, rewriting it is cheap.

That changes the optimal design philosophy.

Traditional software encouraged:

generalize because rebuilding later is expensive.

AI increasingly allows:

specialize because rebuilding later is cheap.

This creates potential pressure on horizontal orchestration software whose main value proposition is generalized workflow abstraction.

The durable layers may instead be:

The generic middle can be squeezed from both directions.

Simple software may be the AI-native answer

One of the stranger effects of AI is that it simultaneously allows us to build much more complicated systems and reduces the need to do so.

If a tool takes months to build, we try to anticipate every future requirement.

If it takes hours or days, we can build exactly what we need now.

Use it.

Measure it.

Throw away the wrong parts.

Rewrite it.

The execution cost of experimentation has collapsed.

That makes simple, disposable, workflow-specific software much more rational than it used to be.

This principle applies beyond orchestration.

It is a broader adaptation to AI-era engineering:

Do not optimize for permanence when iteration is cheap.

What we are watching

The thesis is still early.

We would watch several signals to understand whether intelligence scheduling becomes a meaningful economic layer:

Workload mix

What percentage of agent interactions can cheaper models serve without degrading outcomes?

Microsoft currently describes 50–60% of typical agent interactions as simple enough for cheaper routing. The number should rise if smaller models continue improving.

Frontier escalation rate

Does the percentage of tasks requiring frontier intelligence fall even while the absolute number of frontier tasks rises?

That would support the barbell model.

Cost per successful task

Do AI-native applications begin disclosing or optimizing this internally rather than focusing on raw token cost?

Routing and gateway adoption

Do model routers, dynamic gateways and cross-provider control planes become default enterprise infrastructure?

Evaluation infrastructure

Do companies build proprietary datasets mapping task classes to model performance?

If so, orchestration becomes a learning system rather than a static rules engine.

Build vs. buy

Do small teams increasingly replace generalized AI workflow software with narrow internal tooling?

That would be an important signal for horizontal software economics.

The next abstraction

Today, developers still think in model names.

GPT.

Claude.

Gemini.

DeepSeek.

Kimi.

GLM.

Llama.

But model names may eventually become implementation details.

An application will specify:

quality target latency budget cost ceiling risk class tool requirements privacy constraints

The system will decide how much intelligence to buy.

That is the same abstraction transition computing has made repeatedly.

Applications stopped thinking about individual servers.

Schedulers took over.

AI is beginning the same process.

The models have already become heterogeneous.

The missing layer is the scheduler.

And the companies that learn how to allocate intelligence efficiently may have a quieter but very real advantage:

They will accomplish the same work with less compute — and then use the savings to do more work.

Sources

  1. Microsoft Foundry — Use model router with Foundry agents. Microsoft says simple agent interactions typically represent 50–60% of traffic and can route to cheaper models while maintaining equivalent quality for those tasks. https://learn.microsoft.com/en-us/azure/foundry/openai/how-to/model-router-agents
  2. Microsoft Foundry — Developer Essentials for Agents and Apps. Microsoft reported up to 60% cost savings with similar accuracy in tests comparing model router with direct GPT-4.1 usage. https://devblogs.microsoft.com/foundry/announcing-developer-essentials-for-agents-and-apps-in-azure-ai-foundry/
  3. AWS — Use Amazon Bedrock Intelligent Prompt Routing for cost and latency benefits. AWS reported 48% and 56% cost savings on internal routing tests while maintaining the response quality of the stronger model. https://aws.amazon.com/blogs/machine-learning/use-amazon-bedrock-intelligent-prompt-routing-for-cost-and-latency-benefits/
  4. AWS — Amazon Bedrock Intelligent Prompt Routing. https://aws.amazon.com/bedrock/intelligent-prompt-routing/
  5. Cloudflare — AI Gateway Dynamic Routing. Dynamic routes can enforce conditions, budgets, quotas and model fallbacks without changing application code. https://developers.cloudflare.com/ai-gateway/features/dynamic-routing/
  6. Cloudflare — AI Gateway Spend Limits. Spend controls can route traffic to cheaper fallback models when budgets are exhausted. https://developers.cloudflare.com/ai-gateway/features/spend-limits/
  7. Datadog — Agent Observability: Cost. Datadog currently calculates estimated LLM costs for 800+ models and can attribute spend across requests and applications. https://docs.datadoghq.com/llm_observability/investigate/cost/
  8. Anthropic — Prompting best practices. Anthropic warns that capable models can overuse subagents for tasks where direct operations are faster and sufficient. https://docs.anthropic.com/en/docs/build-with-claude/prompt-engineering/prompt-templates-and-variables

Perceptron Alpha research notes are for informational and educational purposes only. Company references illustrate potential industry effects and do not constitute investment recommendations.