Amazon Swapped Claude Sonnet 4.6 for Strands Decider and Cut Labeling Costs 98%

Short answer: Two Amazon engineers replaced Claude Sonnet 4.6 with Strands Decider, a small open-source “decision model,” for labeling tickets in one of Amazon’s biggest internal ticketing systems. The Strands team says the Gemma 4 E4B version matched or beat production Sonnet on Amazon’s own ticket test sets, was 78% faster per call, and cut cost from about $22,200 to about $500 per million tickets, a 98% drop. The lesson for builders is narrow but valuable: if your LLM is only picking one label from a fixed list, you may be paying for text generation you never use.

What did Amazon actually change?

According to the Strands Agents write-up by engineers Carr Onstott and Mehul Jaiswal, Amazon and AWS teams were using Claude Sonnet 4.6 on Bedrock to sort incoming tickets into 20 categories. The tickets covered customer escalations, capacity requests, automated alarms, security findings and more. Sonnet ran a long system prompt on every ticket and returned JSON with its reasoning and label.

The team swapped that step for Strands Decider, a model that cannot write text at all. It receives some state (the ticket), a question, and a list of options (the labels), and it returns a probability for each option plus a confidence score. They first tried the original 2B model, then moved to the Gemma 4 E4B build after it did clearly better on a test set, and tuned the confidence threshold a label had to clear before being applied. The authors say the change will save “millions of dollars of token costs in a year.”

The story hit the top of Hacker News, where it was one of the most discussed AI posts of the past day.

What is a decision model?

A decision model is a language model with its text-generating head removed. Strands Decider 2B, which the Strands team introduced as open source, takes a pre-trained Qwen3.5-2B “torso,” drops the LM head, and adds a small pointer head (just over a million parameters) that scores each answer option in one parallel pass. The torso is fine-tuned with a rank-16 LoRA adapter.

That design gives three trade-offs:

• It always answers from your options. No malformed JSON, no invented label.
• It returns a calibrated confidence score, which frontier LLM APIs do not expose directly. That lets you decide when to trust the model and when to escalate.
• It cannot reason through hard problems or write anything. The Strands team is explicit that it is unsuited for coding, chatbots and summarization.

The model supports three question types: choice (pick one option), noul (probability that a statement is true) and score (an expected value on an ordered scale). Typical uses the team lists include model routing, tool selection, guardrails, evaluations and policy classification.

How did Strands Decider compare with Sonnet 4.6?

Amazon ran both models on the same live tickets before switching. These are the numbers reported in the write-up:

The token drop came from two places. The decider outputs a single choice instead of a JSON answer, and its prompt is far shorter. Amazon’s original label descriptions alone were longer than the decider’s 4,096-token window, so the team rewrote each of the 20 labels as a one- or two-sentence criterion and added a “none” option. On long tickets the decider drops the oldest comments to fit.

Two caveats are worth keeping in view. The cost figures use published us-east-1 on-demand list prices and leave out Lambda costs. And the accuracy claim, that E4B beat production Sonnet, comes from Amazon’s own internal ticket sets, which are not public. We have not been able to verify it independently.

How was it deployed?

Apart from Amazon’s internal ticketing APIs, the pipeline uses standard AWS services, which is the part other teams can copy:

• Ticket events land in an SQS queue, which triggers a Lambda function.
• Lambda calls a SageMaker AI endpoint running Strands Decider on ml.g5.xlarge instances (one NVIDIA A10G, 24 GB). The model used about 19 GiB and one request took about 640 ms on one instance.
• The endpoint autoscales, runs network-isolated, and loads weights from S3 at start-up.
• AppConfig holds a three-way switch, CloudWatch carries logs and alarms, and the stack is defined in CDK with Beta, Gamma and Prod stages.

At about 640 ms per request, one instance handles roughly 5,600 tickets an hour, which works out to $250 per million tickets at full use. Amazon keeps headroom for traffic bursts, so the real figure lands around $500 per million.

Why does the three-way switch matter more than the model?

The authors say the part they would copy regardless of model is the rollout pattern. On every message, the labeling Lambda reads one of three settings:

• BEDROCK: Sonnet labels every ticket; the decider is not called.
• DUAL: both models run in parallel; Sonnet’s label is applied and compared with the decider’s.
• DECIDER: the decider labels every ticket; Bedrock is not called.

Running both on real traffic and judging every disagreement told them more than any offline evaluation, and rolling back was a 45-second config change. That shadow-mode approach is useful for any model swap, including moving between frontier LLMs.

One practical gotcha from the write-up: JSON key order changed the decider’s answer. Sending the same content with keys in a different order moved output probabilities by up to 0.19, so the team now sorts keys on every request. If you serve a decision model behind an API, check this before comparing runs.

Which Strands Decider model should you try?

Strands Decider now comes in five sizes. The original 2B is built on Qwen3.5-2B and runs on a local CPU or GPU, with a median latency around 115 ms on an RTX 3090. Four more were built on Gemma 4. The Strands team reports these results on JevBench v1’s 231 public tasks:

The team’s own advice is to start with E2B on your workload and only move up to E4B, 12B and then 26B-A4B if it misses your target. Adapters, readout heads and code are released under Apache 2.0, with weights on Hugging Face and code in the strands-decider GitHub repo. You can try it with pip install "strands-decider>=0.2.0" (Python 3.10 or later).

When should you not replace your LLM?

The Amazon engineers are clear about the limits. This worked because the task is a closed choice from a known catalog, the volume is high, and latency mattered. A decision model is probably the wrong fit if:

• your output needs explanation text, not just a label;
• your label catalog changes weekly;
• the useful context will not fit in a few thousand tokens;
• the decision needs multi-step reasoning over long documents, which the Strands team says these models struggle with.

The release notes also warn that confidence calibration was fit on short classification tasks, so thresholds should be tested on your own traffic, and decisions that block actions or route sensitive work should be logged and reviewed.

What should builders do now?

• Audit your LLM calls for hidden classifiers. Routing, tagging, triage, sentiment and yes/no guardrail checks are the obvious candidates.
• Build a test set from your own past decisions before picking a model size or threshold. Public benchmark scores will not tell you how it handles your labels.
• Ship it in shadow mode first. Copy the BEDROCK / DUAL / DECIDER switch so you can compare on live traffic and roll back in seconds.
• Keep the frontier model for the hard cases. A low-confidence score is a natural signal to escalate a ticket to the LLM instead of guessing.

Sources

• Strands Agents: Amazon saves millions by replacing Sonnet 4.6 with Strands Decider
• Strands Agents: Introducing Strands Decider 2B
• Strands Agents: Strands Decider adds four Gemma 4 models
• GitHub: strands-labs/strands-decider

Image credit: Strands Agents