AI & LLM Index
How-to · howto

How to design prompts for reliable JSON

A practical guide to design prompts for reliable JSON, using data from the AI & LLM Index. The goal is a shortlist and a decision, not a tour of every option.

Before you start

This guide walks through design prompts for reliable JSON with a focus on decisions you can check. Before changing anything, write down the current behaviour, the failure you want to fix and the metric you will use to decide whether the change worked.

A small, fixed evaluation set is more useful than a large, vague plan. Ten to thirty representative examples are enough to catch obvious regressions and to compare two model choices.

Step by step

  • Define the input and output contract in one paragraph.
  • Collect examples that cover the common case and the awkward edge cases.
  • Choose the cheapest model that clears the evaluation set.
  • Add logging and a fallback path before you increase traffic.
  • Re-run the evaluation whenever the model, prompt or data changes.

Data from the index

For design prompts for reliable JSON, start with GPT-5.2 Pro as the quality candidate and GPT-5.2 Pro as the value candidate. The price difference between them is $0.000 per million input tokens, which is large enough to justify a small pilot.

Common mistakes

  • Optimizing the prompt before defining the evaluation.
  • Using a frontier model for a task a small model can handle.
  • Ignoring output price and retry costs.
  • Skipping the fallback path until the first outage.

Models to test

ModelProviderInput / 1MContextAvg score
GPT-5.2 ProOpenAI$21.322,000,00089.6
GPT-5.2 MaxOpenAI$21.391,000,00089.2
Claude Opus 4.5 ProAnthropic$20.98200,00089.1
GPT-5.1 ReasoningOpenAI$20.27400,00088.3
Claude Opus 4.5 ReasoningAnthropic$20.912,000,00088.2

Related

How to choose an LLM for productionHow-to How to evaluate an LLM before buyingHow-to How to reduce LLM API costsHow-to How to cut latency in an LLM appHow-to

FAQ

What is the most important factor in design prompts for reliable JSON?

Fit to the task beats raw benchmark rank. Define the failure modes, pick the benchmarks that resemble them, then test a small shortlist on your own examples.

How many models should I evaluate?

Three is usually enough: a quality candidate, a value candidate and a fallback. More candidates add work without changing the decision.

How do I control cost?

Compare blended 3:1 price, watch output tokens, use caching where available and route simple requests to a cheaper model.