When to use a decision model instead of an LLM

Decision models are pitched as a faster and cheaper way to make the structured choices we usually hand to an LLM. I've had a hunch they're cheap because they're worse, and that an LLM with structured outputs would beat them. This week Artificial Analysis launched the Decision Index, which puts both kinds of model through the same tests. So I dug into the data.

My hunch turned out to be half right, and the half I got wrong tells you when to use a decision model.

What decision models are

Decision models are a new class of model built only to pick answers from fixed lists. You give one a question and a list of possible answers, and it returns a probability for every answer. An LLM can answer the same question with structured outputs, which is how Artificial Analysis ran the LLMs in this benchmark.

The benchmark covers 11 decision models and 15 LLM configurations on 4 tasks. Knowledge questions and hard reasoning test what a model knows and how well it thinks, while safety classification and tool-call routing test the kind of fixed-choice decisions these models are built for.

The overall score hides the answer

The index blends all 4 tasks into 1 score, and on that score most LLMs rank above every decision model. Taken alone, it backs up my hunch that decision models are just worse LLMs.

The picture changes when you split the index by task.

Dot plot of scores on 4 tasks for 11 decision models and 15 LLM configurations. On knowledge questions the LLM median is 30 points higher. On hard reasoning it is 19 points higher. On safety classification and tool-call routing the 2 groups overlap and their medians are within 2 points.

LLMs win knowledge and reasoning by a wide margin, and those 2 tasks account for nearly all of the gap in the blended score. That's the half of my hunch that held up.

The other half didn't. On safety classification and tool-call routing, the 2 groups are about even. The best decision model beats every LLM tested at safety classification, and on tool-call routing, only Claude Opus 5.5 does better than the best decision models but at 50 or more times the cost per task.

Where they tie, decision models are much faster

Every decision model answers faster than every LLM on the 2 tasks where scores are even.

Dot plot of median seconds per task on safety classification and tool-call routing. All 11 decision models take between 0.06 and 0.54 seconds. All 15 LLM configurations take between 1.1 and 17 seconds, with medians of 1.5 and 1.7 seconds.

A typical decision model answers in about 0.2 seconds, while a typical LLM takes more than 1 second. Decision models are usually cheaper, too, though the cost ranges overlap more than the speed ranges do.

Dot plot of cost per 1 million tasks on safety classification and tool-call routing. The median decision model costs 5 to 6 times less than the median LLM. The 2 groups overlap, and 4 LLM configurations on each task cost more than 1,000 dollars per million tasks.

For a router in front of an agent or a moderation check on every message, that speed gap matters more than a small difference in accuracy.

Same model, different format

OpenAI's entries offer the cleanest comparison in the data because they hold the model fixed. GPT-6 Luna appears both as a decision model and as an LLM at 5 effort settings. The decision version scores about the same as the LLM version with reasoning off, and it answers several times faster. Turning on reasoning is what pulls the LLM version ahead.

Scatter plot of Decision Index score against median seconds per task for GPT-6 Luna. The decision model version scores 48.5 in 0.17 seconds. The LLM version with reasoning off scores 49.6 in 1.1 seconds. With reasoning on, the LLM version scores 55 to 61 in 1.5 to 2.2 seconds.

So the decision format by itself costs little accuracy. What you give up is the accuracy that reasoning buys.

How I'd choose

Use a decision model for routing, triage, moderation, intent detection, and tool-or-no-tool calls, especially when the input holds everything you need to decide and latency or volume matters.
Use an LLM when the right answer depends on facts the input doesn't contain or on multi-step reasoning.
Check the probabilities before you trust them. Artificial Analysis found decision models overconfident on the knowledge and reasoning tasks but better calibrated on tool calling and classification, which matches where they score well. Their probabilities are most useful for setting escalation thresholds on the tasks they're built for.

The best setup for most products is probably both, with a decision model in front handling the bulk of fixed-choice calls and an LLM behind it taking the requests that need knowledge or reasoning.

How I made these charts

The headline comparison in Artificial Analysis's launch article is a bar chart of the blended index for 9 decision models and 2 LLMs, with bars colored by company. I rebuilt it from the leaderboard data with evident-charts, my open source skill that helps coding agents make clear and honest charts, and used it for every chart in this post. The rebuild splits the index into its tasks, shows every entry on the leaderboard, and uses color to separate decision models from LLMs, which is the comparison that matters here.