Does Prompt Format (JSON, YAML, TOON, Markdown, XML) Affect LLM Performance?

Ever since I started building automations with LLMs, I've always tried to structure my prompts. I explained this a while ago in this talk (minute 48:30, and here's the Google Colab I used), where I said that models don't really "speak" JSON, Markdown, YAML or XML, but these structured formats can help you get more consistent outputs.

Specifically, I explained that writing prompts in structured formats like JSON or YAML helps guide the model more precisely, because instead of plain text, you give it keys and values that set clear parameters.

  • Advantages: Clarity (less ambiguity), flexibility (multiple parameters), consistency (a uniform format) and scalability (easy to plug into automated workflows or APIs).
  • Limitations: A learning curve, plus a possible rigidity that reduces the model's creativity on open-ended tasks.

But over time, and as models have evolved, I've realized that working this way isn't always better for every task. So I set out to check whether it was just me, or whether there's recent research on this and what it concludes.

Here's my analysis of the main studies.

Executive summary

The scientific evidence shows that format does have a significant impact on LLM performance, but the effect varies depending on the type of task, the model used and the kind of constraint applied.

Key findings

Factor Finding
Performance variation Up to 40% variation (He et al.) and up to 76 accuracy points (Sclar et al.) depending on the format used
Model size Larger models (GPT-4) are more robust to format variations
Task type Reasoning tasks suffer more under strict format constraints
Transferability The best formats do NOT transfer between different models

1. Does Prompt Formatting Have Any Impact on LLM Performance?

He et al., 2024 - arXiv:2411.10541

Methodology

They evaluated four formats (plain text, Markdown, JSON, YAML) on GPT-3.5 and GPT-4 using natural language reasoning, code generation and translation benchmarks.

Results

  • Up to 40% variation: GPT-3.5-turbo showed performance variations of up to 40% on code translation tasks depending on the template used.
  • Larger models are more robust: GPT-4 is significantly more robust to these variations than GPT-3.5.
  • Format preferences: GPT-3.5-turbo prefers JSON, while GPT-4 favors Markdown.
  • No universal format: No single format is optimal for every task and model.

2. Let Me Speak Freely? A Study on the Impact of
Format Restrictions on Performance of Large Language Models

Tam et al., 2024 - EMNLP Industry Track

Methodology

They compared free-form answers vs. structured formats (JSON, XML, YAML) on reasoning and classification tasks, using multiple LLMs including GPT-3.5-turbo, Claude, Gemini and LLaMA.

Results

  • Reasoning degrades: Structured format constraints (especially JSON-mode) significantly reduce LLMs' reasoning abilities.
  • Stricter constraints = more degradation: The stricter the format constraint, the bigger the performance drop on reasoning tasks.
  • Classification vs. reasoning: Classification tasks benefit from structured formats, but reasoning tasks suffer.
  • Proposed solution: Use a two-step approach: first generate a free-form answer, then convert it to a structured format.

3. Which Nested Data Format Do LLMs Understand Best?

Improving Agents, 2025

Methodology

They evaluated GPT-5 Nano, LLaMA 3.2 3B and Gemini 2.5 Flash Lite on nested data in JSON, YAML, XML and Markdown, using 1,000 questions over Terraform-like data with 6-7 levels of nesting.

Results

  • YAML came out on top: For GPT-5 Nano and Gemini, YAML gave the best results. LLaMA showed little sensitivity to format.
  • Markdown was the most efficient: Markdown used 34-38% fewer tokens than JSON, making it the most cost-efficient option.
  • XML performed worst: XML needed 80% more tokens than Markdown and had lower accuracy on most models.
  • JSON was problematic: JSON performed poorly with GPT-5 Nano and Gemini, which suggests you should avoid it as your default format.

4. Prompt Engineering for Structured Data

Elnashar, White & Schmidt, 2025 - Artif. Intell. Auton. Syst.

Methodology

They compared six prompt styles (JSON, YAML, CSV, Function Calling APIs, simple prefixes, hybrid CSV/Prefix) on ChatGPT-4o, Claude and Gemini, measuring accuracy, token cost and generation time.

Results

  • Differences between models: Claude consistently achieves the highest accuracy; ChatGPT-4o is the most efficient in tokens and time; Gemini strikes a balance.
  • Trade-offs: Simpler formats cut cost and time with little loss of accuracy; more expressive formats offer more versatility for complex data.
  • Common limitation: All the LLMs struggle with unstructured narrative data (e.g., personal stories), with ~40% accuracy.

5. Additional studies

5.1 FormatSpread (Sclar et al., ICLR 2024)

They proposed a method to quantify how sensitive LLMs are to format changes. They found differences of up to 76 accuracy points on LLaMA-2-13B from minor changes like spacing, punctuation or capitalization.

5.2 Mind Your Format (Voronov et al., 2024)

They evaluated 21 models (770M to 70B parameters) on 4 classification datasets. They showed that a bad template choice can drop the performance of even the best models to random-guess level. They proposed Template Ensembles as a solution.

5.3 TOON Format (2025)

TOON (Token-Oriented Object Notation) is a new structured format designed for token efficiency. In the original benchmark, TOON achieved 73.9% accuracy vs 69.7% for JSON while using 39.6% fewer tokens.

But keep in mind that these numbers come from the TOON repo's own benchmarks, which have significant methodological limitations. Independent tests by Improving Agents put TOON in 9th place out of 12 formats (47.5% accuracy) with GPT-4.1-nano, below JSON (52.3%) and far behind Markdown-KV (60.7%). With nested data and GPT-5-nano, TOON came last (43.1%), versus YAML (62.1%) and JSON (50.3%). An analysis using real billing data confirms that the actual token savings are around 25% (not the advertised 30-60%) and that TOON doesn't improve model accuracy. In short: the token savings are real but more modest, the accuracy advantage doesn't hold up outside the official benchmarks, and TOON is only really useful for uniform arrays of objects in high-volume pipelines. For nested or complex data, YAML and Markdown are still better options.

Synthesis: when does format matter?

Situation Does it matter? Direction of the effect Recommendation
Complex reasoning tasks YES - HIGH Strict formats hurt performance Free-form answer + conversion afterward
Classification tasks YES - MODERATE Structured formats help JSON/XML with a strict schema
Small models (<13B) YES - VERY HIGH High variability Test multiple formats
Large models (GPT-4+) YES - LOW More robust Any format works
Nested/hierarchical data YES - HIGH YAML > JSON > XML Prefer YAML or Markdown
Cost optimization YES - HIGH JSON uses 2x the tokens of TSV Markdown or CSV/TSV
Few-shot prompting YES - VERY HIGH The format of the examples is critical Keep it consistent

Conclusions

  1. Format DOES matter: The scientific evidence clearly shows that prompt format has a significant impact on performance, with variations of up to 40% (He et al.) and up to 76 accuracy points (Sclar et al.).
  2. The impact depends on context: There's no universally optimal format. The best format depends on the type of task (reasoning vs classification), the model used and your cost/latency requirements.
  3. Reasoning vs structure trade-off: There's a fundamental tension between getting structured outputs and preserving reasoning abilities. For complex tasks, the two-step approach mitigates this problem.
  4. Larger models are more robust: GPT-4 and similar models are less sensitive to format variations than smaller models.
  5. Practical recommendation: Always test multiple formats on your specific use case. Use YAML or Markdown for nested data and CSV/TSV for tabular data, and consider the two-step approach for critical reasoning tasks.

References

Natzir Turrado 07 January 2026

Share

Facebook Linkedin Twitter

More articles

It's Not GEO or AEO, It's Just SEO: How to Do SEO for AI

Every time a new concept or acronym pops up in the digital marketing world (GEO, AEO, LLMO, GAIO, LSO, LEO, etc.), the debate is whether we're looking at a real revolution or the same old strategy under a new name. In my experience, optimizing for large language models (LLMs) is, at its core, the logical […]

Read more

Web Agents and the Importance of Semantic, Accessible HTML

AI agents that browse the web, like the ones built on multimodal LLMs, run into complex HTML and interfaces designed for humans. Just like screen readers, many of these agents rely on the semantic structure and accessibility of the HTML to understand a page and act correctly. In this post, we'll look at how to […]

Read more