Amazon Cut a Ticket Labelling Bill by 98 Percent. Only One Quarter of the Saving Came From the Model.

Abhishek Dash10 min read
A tall stack of glowing amber server invoices on a dark desk, split by a sharp blue blade, with tiny glowing green coins dissolving into light and a faint cloud of circuit traces in the background.

In short

AWS replaced Claude Sonnet 4.6 with Strands Decider on an internal ticketing workload and reported 98 percent lower cost and 4.7 times lower latency. The cost table is published and the arithmetic checks out at $0.0222 per ticket against about $0.0005. Roughly 70 percent of the saving comes from a 70 percent token reduction driven by task shape and a rewritten label catalogue, with the rest from the price gap between a per-token frontier API and a per-hour commodity GPU. The "millions of dollars" headline rests on a ticket volume the blog never states.

Key takeaways

  • Sonnet 4.6 labelled tickets at $0.0222 each on Bedrock; Strands Decider on a SageMaker ml.g5.xlarge came to about $0.0005 each, roughly 44 times cheaper.
  • Total tokens per ticket fell from 6,724 to 2,010, a 70 percent reduction, because output fell from 170 tokens of JSON and reasoning to a single choice token.
  • The 20 production label descriptions were longer than Decider's 4,096-token window and were rewritten as one or two sentence criteria plus a none option.
  • Amazon reports one instance handles about 5,600 tickets an hour at 640 ms per request, which would be $250 per million at full use, and about $500 in practice with burst headroom.
  • JSON key order alone moved output probabilities by up to 0.19; Amazon now sorts keys before every request.
  • Saving $2 million a year at the stated $21,700 per million tickets requires roughly 92 million tickets a year, a volume the blog never states.

AWS teams were labelling millions of internal tickets with Claude Sonnet 4.6 on Bedrock, mapping each ticket onto one of 20 categories. They replaced it with Strands Decider, a decision model, and reported a 98 percent cost reduction and a 4.7 times latency improvement. The engineering blog claims the change saves millions of dollars of token costs in a year.

The number is real and it is arithmetic from disclosed inputs: $0.0222 per ticket for Sonnet against about $0.0005 for the decision model. But the mechanism is not "a small model is cheaper than a big model". It is four separate changes that compound, and only one of them is the model itself.

The four changes

First, the task shape. Sonnet was generating JSON plus reasoning against a long system prompt, producing 6,724 tokens per ticket of which 170 were output. The decision model produces exactly one output token, the choice, and reads 2,009 input tokens. That is a 70 percent token reduction from task design, not from model size.

Second, the label catalogue was rewritten. Production label descriptions were longer than the decision model's 4,096-token window, so each of the 20 labels was rewritten as a one or two sentence criterion and sent as options of a single choice question plus a "none" option. The prompt is short because the catalogue was rewritten, not because the model is small.

Third, the deployment shape. Sonnet ran on Bedrock at $3.00 per million input and $15.00 per million output. The decision model ran on a SageMaker ml.g5.xlarge instance, one A10G with 24 GB, at $1.408 per instance-hour, using about 19 GiB and about 640 ms per request, autoscaling with traffic on an isolated network.

Fourth, and the part a reader should notice: the comparison is between a per-token API price and a per-hour instance price. The decision model's cost depends entirely on instance utilisation. Amazon says one instance handles about 5,600 tickets an hour at 640 ms per request, which would be $250 per million tickets at full use, but they keep headroom for bursts and in practice it comes to about $500 per million. So the 98 percent figure is computed against a deliberately under-utilised deployment.

The cost table

This is the entire case study in one table, and it deserves to be reproduced with its footnotes intact rather than reduced to a headline.

Claude Sonnet 4.6 (Bedrock, global endpoint) Strands Decider E4B (SageMaker, ml.g5.xlarge)
Price $3.00 per 1M input, $15.00 per 1M output tokens $1.408 per instance-hour
Per ticket $0.0222 about $0.0005
Per million tickets about $22,200 about $500

The Sonnet figure is computable from the token counts: 6,554 input at $3.00 per million plus 170 output at $15.00 per million is $0.0222. The Strands figure is not a token price at all. It is an instance-hour price divided by the tickets that instance served.

So the decomposition of the 98 percent is:

  • roughly 70 percent of the win from token count reduction, which comes from a one-token output replacing JSON plus reasoning and from a rewritten, much shorter prompt
  • the remainder from the price gap between a per-token frontier API and a per-hour commodity GPU
  • and the doubling from $250 to $500 that comes from running with burst headroom rather than at theoretical maximum utilisation

Amazon says the Lambda cost is also lower now because it runs for less time per ticket, and that this is left out of the published list prices. That is stated in the blog, not hidden.

The finding nobody quoted: JSON key order moves the output by 0.19

From the case study, in Amazon's own words:

"One thing that surprised us was that JSON key order changed the answer. Sending the same content with keys in a different order moved the output probabilities by up to 0.19. We now send every request with sorted keys, so if you serve a decision model behind an API, check this before you compare runs."

A probability shift of up to 0.19 from key ordering alone is large. It is not a rounding artefact. It means that a naive evaluation of a decision model, where the request serialiser happens to produce keys in a different order between the baseline and the candidate, is measuring nothing but serialiser behaviour.

This matters well beyond AWS. Decision models are being sold as a general architecture, and their entire pitch is that the reliability score makes them better than a frontier LLM for closed choices. If that score is sensitive to input serialisation at this magnitude, then every published calibration number for this model class should be treated as conditional on an unstated normalisation. Amazon fixed it by sorting keys. I found no discussion of this anywhere outside the AWS blog, and no third-party replication. That is a genuine open question in a new model class.

What the community pushed back on

Engagement from a last30days run: 18 evidence items across Reddit, Hacker News, YouTube and GitHub. Reddit returned 13 threads with 3,974 upvotes and 865 comments across r/aws, r/artificial and r/devops. GitHub returned the strands-agents SDK at 9,144 stars.

The sharpest question was arithmetic, and it was asked on Hacker News:

"At roughly $21,700 saved per million tickets, saving millions annually would suggest at least 92 million tickets a year to reach $2 million saved. Is that about right?"

This is the right question and it is worth answering precisely. At $22,200 per million tickets for Sonnet and $500 for the decision model, the saving is $21,700 per million tickets. To save $2 million you need roughly 92 million tickets a year. To save "millions" in the plural, say $5 million, you need roughly 230 million tickets a year. Amazon describes the workload as "one of the world's largest internal ticketing systems", and describes the ticket population as spanning "customer escalations, capacity requests, automated alarms, security findings, or anything else across the company". A company with automated alarms and security findings flowing into a 20-label system plausibly does process that volume, but the blog never states the ticket count, so the "millions" claim cannot be independently checked from the document. This is a gap in the primary source, not a contradiction of it.

The second pushback was about whether the comparison is apples to apples:

"One of the biggest issues with using a general LLM like Sonnet is calibration. I think Jev really helped bring this to light. With Sonnet, you get an answer and that's it. It's really not that great for classification, especially tagging..."

This one is more sympathetic to the case study than the case study itself. The argument is that Sonnet was never calibrated for this task, so it was doing a classification job without the tooling for it. If the task needs a probability per label and a frontier chat API cannot give you a reliable one, then a model that returns calibrated probabilities per option is a better tool for the job regardless of its size.

The third pushback was historical, and it is the one that most deserves a place in a post:

"About six years ago we built a ticket classification and routing model and pipeline. The initial version was trained using SVMs, it took a couple of minutes to train, and the ticket classification was just a few milliseconds. It's so fast..."

The claim is that a support ticket classification problem was solved with an SVM roughly six years ago, and that the interesting part of the 2026 story is not the model but the pipeline that feeds it. If that is right, then the correct reading of the AWS post is not "frontier models are obsolete for classification". It is "classification was never the frontier model's job, and the interesting work is in task shape, pipeline design and switching mechanisms". The AWS post itself supports this reading. Its lessons section is entirely about task shape, prompt length, catalogue stability and the three-way switch. The model is almost incidental.

The limits

The headline "98 percent less cost" understates what actually happened, in a way that flatters the model. If Amazon had kept Sonnet but rewrote the label catalogue into a compact choice format and dropped the reasoning requirement, most of the token saving would still be there. The blog does not report that counterfactual. The comparison is Decider with rewritten prompts against Sonnet with its existing long prompt. That is a fair comparison for a real migration, but it is not a measurement of model capability against model capability.

The cost figures are list prices, and the blog says so. "These are published list prices. They leave out the Lambda." They also leave out the engineering cost of the three-way switch, the AppConfig plumbing, the CDK stages, the rewriting of 20 label definitions, and the tuning of the confidence threshold at which a label gets applied. That threshold tuning is the part with a hidden cost: "Afterwards, we tuned how confident the model needed to be before applying a label." A decision model returns a probability, and choosing where to cut is a judgement call that a chat model does not force you to make explicitly. A post should say the migration was not free, not that it was free but cheaper.

The decision model has a smaller context window than the prompt it replaced. Production's label descriptions alone exceed the 4,096-token window. That means the workflow that fit Sonnet's long prompt does not fit Decider, so on long tickets Decider "drops the oldest comments to fit". Dropping the oldest comments from a support ticket is a substantive loss of context that the cost comparison does not price. Amazon discloses this plainly, which is good, but the 98 percent number does not carry a caveat about it.

Amazon's own limitation list excludes the case study. The launch blog says decision models "are significantly worse at solving complex problems than reasoning models" and are "unsuited for coding, chatbots, document summarisation". The case study's own lessons section repeats that this worked because the task was a closed choice from a known catalog. Read together, these say: decision models are for closed choices only. The 98 percent is not a general statement about model substitution.

The comparison is between a managed API and self-managed infrastructure. Sonnet ran on Bedrock with no instance to operate. Decider runs on SageMaker instances that someone has to size, monitor, patch, scale and keep isolated. Amazon has no incentive to describe this cost and mentions only the Lambda saving. A reader should treat the comparison as excluding operational cost.

"Just as well" is doing real work in the sentence. The blog says Decider "performed just as well at 98 percent less cost" and that on a separate set of tickets "the results showed E4B beating production Sonnet". It does not publish the accuracy figure, the sample size, or the disagreement rate between the two models in DUAL mode. Amazon says judging every disagreement on real traffic "told us more than any offline eval would have". That is a good practice and it is also a way to avoid publishing the number. The 98 percent is well evidenced; the "just as well" is asserted.

Where this could be wrong

This post is built on a vendor's own engineering blog, which is unusually complete for the genre and still a vendor's own engineering blog. Every figure above is either a published list price or a disclosed token count, both of which check out. But the claim "saves millions of dollars a year" reduces to a volume figure that the blog never states, and the community worked out that roughly 92 million tickets a year are needed to reach $2 million. Until Amazon publishes the volume, the headline number cannot be checked from the document that carries it. The JSON key order finding is the second weakest: disclosed, plausible, and completely unreplicated. No accuracy figure, no F1, no confusion matrix and no DUAL-mode disagreement rate were published. "E4B beating production Sonnet" is asserted without the numbers. Instance utilisation, patching, on-call, GPU availability and the cost of the dual-run period are all excluded. The true cost delta is less favourable than 98 percent and unknowable from the document.

On this page

Sources

  1. Amazon saves millions by replacing Sonnet 4.6 with Strands DeciderAWS, 2026
  2. Introducing Strands Decider 2BAWS, 2026
  3. strands-decider-2b on GitHubstrands-labs, 2026
  4. Strands Decider Gemma 4 modelsAWS, 2026

Frequently asked questions

What is Strands Decider?

Strands Decider is a decision model: a 2 billion parameter model built from a Qwen3.5-2B torso with the language model head removed and replaced by a pointer head of just over one million parameters, fine-tuned with a rank-16 LoRA. It cannot generate text and is unsuited to coding, chatbots and document summarisation. It returns a choice from a closed catalogue plus a reliability score.

Where did the 98 percent saving come from?

Roughly 70 percent from a token reduction driven by task shape: a one-token output replacing JSON plus reasoning, and a rewritten, much shorter prompt. The remainder from the price gap between a per-token frontier API on Bedrock and a per-hour commodity GPU on SageMaker. The doubling from $250 to $500 per million tickets comes from running with burst headroom rather than at theoretical maximum utilisation.

Is the comparison fair?

It is a fair comparison for a real migration but not a measurement of model against model. Sonnet ran with its existing long prompt and reasoning requirement; Decider ran with a rewritten compact catalogue. The blog does not report the counterfactual of Sonnet with the same compact prompt. The comparison is also between a managed API and self-managed infrastructure, and it excludes operational cost.

What is the JSON key order finding?

Amazon found that sending the same content with JSON keys in a different order moved output probabilities by up to 0.19. It now sorts keys before every request. No third-party replication exists.

Can the millions of dollars claim be checked?

No. At the stated $21,700 saving per million tickets, saving $2 million requires roughly 92 million tickets a year and $5 million requires roughly 230 million. Amazon describes the workload as one of the world's largest internal ticketing systems but never states the ticket count.