Prompt wording, format and detail measurably change what an AI model gives back. Across the studies below, a single added sentence lifted one model's math accuracy from 17.7% to 78.7%, formatting changes alone moved another's by up to 76 points, and prompts that left requirements unstated were twice as likely to break when the model or prompt changed. Real users mostly type short requests: the median first ChatGPT message in one public dataset was 17 words. This page collects 30 numbers like these, each checked against its primary source, with a line on what each one means for an ordinary prompt.

Last updated 9 November 2026. Every figure was checked against its source in October 2026.

How we chose these statistics

Each number below comes from a primary source we read: a peer-reviewed or preprint paper (arXiv, ACL, ICLR, ICML, NeurIPS, TACL), an official vendor document, or a large public dataset. We left out statistics that only appear in vendor blog posts or roundups without a paper behind them, and anything we couldn't find stated in the source itself. Many of these results come from older models such as GPT-3, so read them as evidence that wording matters, not as exact predictions for the model you use today. We'll add and retire figures as new research comes out.

How much does prompt wording change the answer?

  • Adding "Let's think step by step" raised text-davinci-002's accuracy on the MultiArith math set from 17.7% to 78.7%, and on GSM8K from 10.4% to 40.7% (Kojima et al., NeurIPS 2022). The model and the questions stayed the same; only the prompt changed.
  • A machine-searched variant, "Let's work this out in a step by step way to be sure we have the right answer," pushed the same scores further, to 82.0 on MultiArith and 43.0 on GSM8K (Zhou et al., 2022). Small differences in phrasing still mattered after the big gain.
  • A set of 26 prompting principles improved GPT-4's response quality by an average of 57.7% and its accuracy by 36.4% (Bsharat et al., 2023). The authors tested them on LLaMA-1/2 models and GPT-3.5 as well.
  • Adding emotional phrases to prompts (the authors call it EmotionPrompt) gave an 8.00% relative improvement on Instruction Induction tasks and 115% on BIG-Bench tasks (Li et al., 2023). In a 106-person human study, the same prompts scored 10.9% higher on average for performance, truthfulness and responsibility.
  • A study of 20 models across 39 tasks and 6.5 million instances found that results measured with a single prompt are brittle, and called for testing every model with several phrasings (Mizrahi et al., TACL 2024). If benchmark scores move with wording, so can your answers.

Does the format or order of a prompt matter?

  • Changing only small details of a prompt's formatting produced accuracy differences of up to 76 points for LLaMA-2-13B (Sclar et al., ICLR 2024). The sensitivity remained with larger models, more examples and instruction tuning.
  • GPT-3.5-turbo's performance on a code translation task varied by up to 40% depending on the prompt template, while GPT-4 was more robust to the same changes (He et al., 2024). Bigger models help, but they don't make format irrelevant.
  • The order of the examples in a few-shot prompt can make "the difference between near state-of-the-art and random guess performance" (Lu et al., ACL 2022). Picking a good order gave a 13% relative improvement across eleven text classification tasks.
  • Prompt format, the choice of examples and their order can swing GPT-3's few-shot accuracy from near chance to near state of the art; correcting for the model's bias improved average accuracy by up to 30.0 points (Zhao et al., ICML 2021). Instability was documented early and hasn't gone away.
  • Models perform best when the relevant information sits at the beginning or end of a long input, and significantly worse when it sits in the middle (Liu et al., TACL 2023). Put the thing you care about first or last.
  • Anthropic's documentation says that placing your question after long documents, at the end of the prompt, "can improve response quality by up to 30 percent in tests" (Claude docs, prompting best practices). Paste the material first, then ask.

Does being specific, or giving examples, help?

  • When a prompt leaves a requirement unstated, models infer it by default only 41.1% of the time (Yang et al., 2025). The rest of the time you get the model's guess.
  • In the same study, underspecified prompts were 2x as likely to regress when the model or prompt changed, sometimes with accuracy drops of more than 20% (Yang et al., 2025). A vague prompt that works today can stop working after an update.
  • Eight worked examples with written-out reasoning (chain-of-thought) were enough for a 540B-parameter model to reach state-of-the-art accuracy on GSM8K, beating a fine-tuned GPT-3 with a verifier (Wei et al., 2022). Showing the method can beat describing it.
  • Across 12 models, randomly swapping the labels in a prompt's examples "barely hurts performance"; the examples' format and the kind of input they show matter more (Min et al., EMNLP 2022). Examples mostly teach shape, so make sure the shape is the one you want.
  • Anthropic recommends including 3 to 5 examples for best results when you want to steer format, tone and structure (Claude docs, prompting best practices). The same page suggests wrapping examples in tags so the model can tell them apart from your instructions.

Are longer prompts better?

  • Google reports that its users' most fruitful prompts "average around 21 words with relevant context," yet the prompts people try are usually under nine words (Gemini for Workspace Prompting Guide 101, 2024). The same guide lists the four things to cover: persona, task, context and format.
  • Padding the same reasoning question to about 3,000 tokens dropped average accuracy across the tested models from 0.92 to 0.68 (Levy, Jacoby and Goldberg, ACL 2024). Length that doesn't carry information costs accuracy, well below the models' advertised limits.
  • Adding irrelevant sentences to grade-school math problems "dramatically decreased" model accuracy; telling the model to ignore irrelevant information recovered some of it (Shi et al., ICML 2023). Cut what the model doesn't need, or say that it can be ignored.

How do people actually prompt AI chatbots?

  • WildChat holds 1,039,785 real ChatGPT conversations from 204,736 users in 68 languages, with 52.94% of turns in English (Zhao et al., ICLR 2024). It's one of the largest public records of real prompts.
  • In WildChat's English first turns, 61.9% were "assisting or creative writing", 13.6% analysis, 6.7% coding and 6.3% factual questions (Zhao et al., ICLR 2024). Most people ask chatbots to make things, which is where vague prompts hurt most.
  • In one of WildChat's 14 data files (25,144 English first messages), the median first prompt was 17 words when counting one per user, and 32% were under 10 words; our count, from the public WildChat-1M dataset. Short requests are the norm.
  • The mean WildChat user turn is 295.58 tokens, against 69.83 in LMSYS-Chat-1M, (Zhao et al., ICLR 2024). WildChat's standard deviation of 1,609 tokens shows a long tail of very long pastes, so averages hide how short a typical prompt is.
  • LMSYS-Chat-1M records one million conversations with 25 models from 210,000 unique IP addresses, and its authors found the majority of questions related to coding and software (Zheng et al., 2023). It was collected on a model-comparison site, which likely draws a more technical crowd than ChatGPT does.
  • A 2025 NBER study of ChatGPT usage found non-work messages grew from 53% to more than 70% of use, and that "Practical Guidance", "Seeking Information" and "Writing" together make up nearly 80% of conversations (Chatterji et al., NBER, 2025). Writing dominated work-related use.
  • In Anthropic's privacy-preserving analysis of one million Claude.ai conversations, web and mobile app development was over 10% of all conversations, with writing, research and education each at 6% to 10% (Tamkin et al., 2024). The analysis covered both free and paid accounts.

Can AI rewrite prompts automatically?

  • Prompts found by an LLM optimiser beat human-designed prompts by up to 8% on GSM8K and by up to 50% on Big-Bench Hard tasks (Yang et al., ICLR 2024). A model can search phrasings faster than a person can.
  • Machine-generated instructions matched or beat those written by human annotators on 19 of 24 NLP tasks (Zhou et al., 2022). Prompt writing is a task models are already decent at.
  • Rewriting user prompts before they reach the model raised ChatGPT's win rate against its own unrewritten answers by 22%, and GPT-4's by 10%, with no model training (Cheng et al., ACL 2024). The same rewrites beat models tuned with PPO and DPO.
  • A survey of prompting research catalogued 58 text prompting techniques and 40 more for other modalities, plus 33 vocabulary terms (Schulhoff et al., 2024). Nobody is expected to memorise them, which is part of why automatic rewriting exists. A 2025 study of real conversations reached a similar conclusion: rephrasing ineffective prompts got better responses from a chatbot (Sarkar et al., 2025).

What this means for your prompts

Most of these numbers point the same way. Say what you want, who it's for and what shape the answer should take. Put pasted material first and the question last. Cut what the model doesn't need. If an answer matters, try a second phrasing, because the research shows that a single phrasing can mislead you about what the model can do.

If you'd rather not do that by hand every time, PrePrompt is a chrome extension that adds a Rewrite button to ChatGPT, Claude, Gemini and Perplexity. It launches soon with 30 free rewrites a month, no card required; join the waitlist to hear the day it goes live. It grades the prompt first and leaves a clear one alone.

TL;DR

  • Wording alone can move results a long way: "Let's think step by step" took one model from 17.7% to 78.7% on a math set.
  • Format and order matter too, by up to 76 accuracy points in one study; larger models are less sensitive but not immune.
  • Unstated requirements get guessed: models inferred them by default only 41.1% of the time.
  • Extra words help only when they carry context. Padding to about 3,000 tokens cut average reasoning accuracy from 92% to 68%.
  • Real people mostly type short requests to make something, and machine rewriting of prompts measurably improves answers in lab tests.

About the author. Tanuj writes PrePrompt's fundamentals and how-to coverage and works on the product's marketing. He read each source on this page in full or at the cited section, and counted the WildChat length figures himself from the public dataset.

Sources: Kojima et al., Large Language Models are Zero-Shot Reasoners (NeurIPS 2022, arXiv:2205.11916); Zhou et al., Large Language Models Are Human-Level Prompt Engineers (arXiv:2211.01910, 2022); Bsharat et al., Principled Instructions Are All You Need (arXiv:2312.16171, 2023); Li et al., Large Language Models Understand and Can be Enhanced by Emotional Stimuli (arXiv:2307.11760, 2023); Mizrahi et al., State of What Art? A Call for Multi-Prompt LLM Evaluation (TACL 2024, arXiv:2401.00595); Sclar et al., Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design (ICLR 2024, arXiv:2310.11324); He et al., Does Prompt Formatting Have Any Impact on LLM Performance? (arXiv:2411.10541, 2024); Lu et al., Fantastically Ordered Prompts and Where to Find Them (ACL 2022, arXiv:2104.08786); Zhao et al., Calibrate Before Use (ICML 2021, arXiv:2102.09690); Liu et al., Lost in the Middle (TACL 2023, arXiv:2307.03172); Anthropic, Claude documentation: prompting best practices; Yang et al., What Prompts Don't Say (arXiv:2505.13360, 2025); Wei et al., Chain-of-Thought Prompting Elicits Reasoning in Large Language Models (arXiv:2201.11903, 2022); Min et al., Rethinking the Role of Demonstrations (EMNLP 2022, arXiv:2202.12837); Google, Gemini for Workspace Prompting Guide 101 (October 2024); Levy, Jacoby and Goldberg, Same Task, More Tokens (ACL 2024, arXiv:2402.14848); Shi et al., Large Language Models Can Be Easily Distracted by Irrelevant Context (ICML 2023, arXiv:2302.00093); Zhao et al., WildChat: 1M ChatGPT Interaction Logs in the Wild (ICLR 2024, arXiv:2405.01470) and the WildChat-1M dataset (Allen Institute for AI, ODC-BY 1.0); Zheng et al., LMSYS-Chat-1M (arXiv:2309.11998, 2023); Chatterji et al., How People Use ChatGPT (NBER Working Paper 34255, 2025); Tamkin et al., Clio: Privacy-Preserving Insights into Real-World AI Use (arXiv:2412.13678, 2024); Yang et al., Large Language Models as Optimizers (ICLR 2024, arXiv:2309.03409); Cheng et al., Black-Box Prompt Optimization (ACL 2024, arXiv:2311.04155); Schulhoff et al., The Prompt Report (arXiv:2406.06608, 2024); Sarkar et al., Conversational User-AI Intervention: A Study on Prompt Rewriting for Improved LLM Response Generation (arXiv:2503.16789, 2025).

Related reading: what a good prompt looks like, why ChatGPT gives vague answers and how to rewrite a ChatGPT prompt automatically.