Bedrock structured outputs stop at Claude 4.6 — a prompt rule and error feedback for Sonnet 5

Amazon Bedrock rejects strict tool use on Claude Sonnet 5. A prompt rule and a tool_result error retry kept translation JSON valid without it.

The Amazon Bedrock icon, a white line-drawn brain of hexagonal outlines with circuit traces ending in small circles, on a teal rounded square, beside the words Amazon Bedrock in dark navy on a white card
On this page

Introduction

A document translation pipeline I work on started failing batches with ValueError: 応答要素の形式が不正: ['id', 'text']: a response element with the wrong keys. Claude Sonnet 5 was forced to answer through a tool whose schema asks for {id, translated_text}, and on some calls it named every key text instead, copying the input.

Strict tool use exists to make that impossible, and Amazon Bedrock rejects it for Sonnet 5 and every Claude model after Opus 4.6. A one-line prompt rule kept the JSON valid, backed by sending the validation error back to the model as a tool result. On Sonnet 5, both worked.

Where structured outputs stops on Bedrock

Structured outputs is the feature that compiles a JSON Schema into a grammar and lets the model sample only tokens the grammar allows, so the output cannot break the schema. Bedrock exposes it two ways, usable separately or in the same request:

  1. strict: true sits on a tool definition and holds the call’s arguments to that tool’s input_schema
  2. output_config.format carries a JSON Schema of its own for the model’s plain JSON answer, with no tool involved

Both give the same guarantee to a different target, and both were tried on Sonnet 5.

Sending the same minimal request to every Claude inference profile in ap-northeast-1 split the models at the 4.6 generation:

strict_probe.py
body = {
    "anthropic_version": "bedrock-2023-05-31",
    "max_tokens": 200,
    "messages": [{"role": "user", "content": "a に hello を入れて"}],
    "tools": [{"name": "t", "description": "x", "input_schema": schema, "strict": True}],
    "tool_choice": {"type": "tool", "name": "t"},
}
client.invoke_model(modelId=f"global.anthropic.claude-{m}", body=json.dumps(body))
Modelstrict: true on Bedrock
Haiku 4.5, Sonnet 4.5, Opus 4.5Accepted
Sonnet 4.6, Opus 4.6Accepted
Opus 4.7, Opus 4.8, Opus 5, Opus 5.5ValidationException
Sonnet 5, Sonnet 5.5ValidationException

Every rejection carried the same message, and output_config.format failed the same way on Sonnet 5 while Haiku 4.5 answered with schema-shaped JSON:

ValidationException: tools.0.custom.strict: Extra inputs are not permitted
ValidationException: output_config.format: Extra inputs are not permitted

The Sonnet 5 model card agrees: its Capabilities and Features table lists Structured outputs under Not Supported, on both the bedrock-runtime and bedrock-mantle endpoints. Fable 5 and 5.1 could not be tested at all; both refused the request with data retention mode 'default' is not available for this model.

Structured outputs and forced tool use by Claude model and platformA table of six Claude model groups against three columns. On Bedrock, strict tool use is accepted for Haiku 4.5 and the 4.5 and 4.6 Sonnet and Opus models, and rejected for Opus 4.7 and later and for Sonnet 5 and 5.5. On Anthropic’s API every group supports structured outputs. Forced tool_choice works everywhere except Sonnet 5.5 and Opus 5.5. Sonnet 5, the model used here, has forced tool use on both platforms but structured outputs only on Anthropic’s API.ModelBedrockstrict / output_configAnthropic APIstructured outputsAnthropic APIforced tool_choiceHaiku 4.5✓ Supported✓ Supported✓ SupportedSonnet 4.5 · Opus 4.5✓ Supported✓ Supported✓ SupportedSonnet 4.6 · Opus 4.6✓ Supported✓ Supported✓ SupportedOpus 4.7 · 4.8 · 5✗ Rejected✓ Supported✓ SupportedSonnet 5used here✗ Rejected✓ Supported✓ SupportedSonnet 5.5 · Opus 5.5✗ Rejected✓ Supported✗ Rejected
The same models on Bedrock and on Anthropic’s own API. The schema guarantee and forced tool use each drop out at a different generation.

Is the limit Bedrock’s or Anthropic’s?

Anthropic’s own API supports structured outputs on Sonnet 5. Its structured-outputs page lists claude-sonnet-5, claude-opus-5-5 and claude-sonnet-5-5 among the supported models, and notes that on Amazon Bedrock only a smaller set (Opus 4.6, Sonnet 4.6, Sonnet 4.5, Opus 4.5 and Haiku 4.5) has it.

Bedrock compiles that grammar itself, from the JSON Schema in each request. Its structured-outputs page states under “Request workflow” that “Amazon Bedrock compiles the grammar, which may take up to a few minutes”.

The grammar pipeline is Bedrock’s own infrastructure, so support is decided per deployment rather than per model. The same page warns that structured outputs “aren’t supported for Claude Haiku 4.5 through the India (in.anthropic) inference profile”, a model that has it everywhere else.

Why each new Claude generation arrives on Bedrock without it is suspected, not documented. The likeliest reading of those two quotes is that every model has to be enabled in Bedrock’s own grammar engine, and that has not happened for anything newer than the 4.6 generation. AWS gives no reason and no date, though I expect support to roll out over time.

The Anthropic SDK on Bedrock

anthropic[bedrock] ships an AnthropicBedrock client that speaks the Messages API, which reads like a way around Converse, Bedrock’s own model-agnostic chat API. It signs the request and sends it to the same bedrock-runtime InvokeModel endpoint. The probe above already used that body shape through invoke_model, and it is the request Bedrock refused.

Why forced tool use still returned the wrong key

tool_choice: {"type": "tool"} guarantees that the model calls the tool. It does not look at what goes inside the call. Anthropic’s structured-outputs page is explicit that without it, “Claude can generate malformed JSON responses or invalid tool inputs”, including “Missing required fields”.

The pipeline sends each batch as a JSON array of {"id", "text"} objects in the user message, 63 of them for one Word document, and asks for {"id", "translated_text"} back.

When the model wrote text for the first element, it wrote text for all 63. The values were proper English translations, so the model had done the work and mislabelled it.

The rate depended on the prompt, and that one Word document reproduced it on demand:

PromptCalls with every key named text
Without the soft length cap (translation about 2× the source)2 of 30
With that cap, as in production8 of 40

One call in five broke at production settings, and a plain retry of the same prompt broke at the same rate.

A prompt rule that names the forbidden key

The first fix is one line appended to the end of the system prompt, after every other instruction. It says to return results through submit_translations, to use the keys id and translated_text, and never to use the input key text.

system prompt, last line
- 結果は `submit_translations` ツールで返すこと。`translations` の各要素のキーは `id` と `translated_text` にする。入力のキー `text` を出力に使ってはいけない (訳文は `translated_text` に入れる)
English translation (untested)
- Return the results with the `submit_translations` tool. Each element of `translations` uses the keys `id` and `translated_text`. Do not use the input key `text` in the output (put the translation in `translated_text`)

A version that stated only the correct keys, without mentioning text, still broke:

RuleCalls with every key named text
None8 of 40
Correct keys only4 of 40
Correct keys, and text named as forbidden0 of 40

The rule costs 82 input tokens per call, on a 4,885-token prompt. Zero in 40 does not mean the rate is zero: 40 samples only bound the true rate below roughly 7.5%, so a second line of defence is needed for the calls that slip through.

Feeding the error back as a tool_result

Instead of sending the batch again, send the broken answer back with the error. That is how I had handled invalid SQL in a text-to-SQL project: run the query, return the database error, and let the model try again with it in view.

Anthropic documents the same move for tool calls: “continue the conversation forward with a tool_result that indicates the error, and Claude will try to use the tool again”.

On Bedrock’s Converse API that means appending the broken assistant turn unchanged and answering its toolUse with an error:

bedrock adapter
bad_use = next(b["toolUse"] for b in bad_content if "toolUse" in b)
messages = [
    {"role": "user", "content": [{"text": user}]},
    {"role": "assistant", "content": bad_content},  # the response, unmodified
    {"role": "user", "content": [{"toolResult": {
        "toolUseId": bad_use["toolUseId"],
        "content": [{"text": feedback}],
        "status": "error",
    }}]},
]
client.converse(modelId=model_id, system=[{"text": system}], messages=messages,
                inferenceConfig=inference_config, toolConfig=tool_config)

The feedback string says what was wrong and what to do: the received keys were ['id', 'text'], each element needs id and translated_text, and every element should be submitted again. Keep the assistant content byte-for-byte, because with adaptive thinking on it can carry reasoningContent blocks that have to go back with it.

Every real broken response from the measurements above, all eight, was fixed on the first retry. A retried call costs about 72% more input tokens than the original and the same 24 seconds.

Retrying underneath the response cache

The pipeline caches each model response in DynamoDB, keyed by a hash of the complete prompt, so a redelivered job replays the batches that already succeeded instead of paying for them again. Both fixes above have to sit underneath that cache, or the cache undoes them.

A cache that saves a response before anything checks its shape stores the broken batch, and every redelivery replays it. The second and third SQS deliveries each logged a cache hit for the same key and failed with the same ValueError. Validating before saving, and again on read, closes that.

The second trap is a retry that changes the prompt: feeding the error back appends the broken response and the error message to the request. Sonnet 5 also accepts no temperature other than the default 1.0, so every repeated call returns a different translation.

Each batch’s prompt also carries the previous batch’s translated tail as context. A batch fixed by a different prompt is saved under a key the next delivery never builds, and the miss propagates.

A retry under a changed prompt misses the cache for every later batchTwo rows of five batches. In delivery 1, batch 0 is saved, batch 1 breaks and is retried with a changed prompt and saved under the retry key, batch 2 is saved, batch 3 fails with a Bedrock error and batch 4 is not reached. In delivery 2, batch 0 is a cache hit, batch 1 builds its original key and misses, producing a new translation, and batches 2 to 4 miss because their context changed.Batch 0Batch 1Batch 2Batch 3Batch 4Delivery 1SavedBroken, retriedwith a new promptsaved: retry keySavedBedrock errordelivery endsNot reachedDelivery 2Cache hitOriginal keymiss: newtranslationContext changedmissContext changedmissContext changedmissSQS redelivers the job; each batch rebuilds its prompt and looks up its keyEach prompt carries the previous batch’s translation, so one new translation moves every later key
A batch repaired under a changed prompt is stored where the next delivery never looks. Its fresh translation changes the context of every later batch in the same flow.

In a simulated five-batch document, with batch 1 broken once and the first delivery dying on batch 3, the second delivery called the model for batches 1 to 4 instead of 3 and 4.

Moving the retry inside the Bedrock adapter fixes both traps at once. The cache wrapper calls the adapter once with the original prompt; the adapter validates, feeds the error back if it has to, and returns either a valid response or the broken one. The cache stores only a valid response, under the original key.

The error-feedback retry runs inside the Bedrock adapter, below the response cacheThe translator calls the response cache, whose key is a hash of the original prompt and which saves a response only when its shape is valid. Inside the cache sits the Bedrock adapter. It makes a Converse call and checks the shape. A valid response is returned to the cache. An invalid one is sent back to the model as a tool_result error and the call is made again, at most once.TranslatorResponse cacheKey: hash of the original promptSaves a response only if its shape is validBedrock adapterConverse callCheck the shapeSend it back as atool_result errorValid responsevalidinvalidat most oncereturned to the cache
The cache sees one call with the original prompt. The validation and the error round trip happen inside the adapter.

Summary

ProblemWhat handles it
Bedrock rejects strict: true and output_config.format on Claude Sonnet 5 and every model after Opus 4.6Nothing on Bedrock’s side yet; validate in your own code
Forced tool use renames translated_text to text in 8 of 40 callsA system-prompt rule that names text as forbidden: 0 of 40
A call that still breaksReturn it as a toolResult error with the keys it should have used: 8 of 8 fixed
A response cache keyed on the promptValidate before saving, and retry inside the model adapter so the key stays the original

Structured outputs on Bedrock is a feature of the deployment, and the model card is where to check it before choosing a model. When it is missing, a validator that feeds its error back to the model buys most of the guarantee back for one extra call on the batches that need it.

References

Share this article

More

More