Quick answer
Chain-of-thought (CoT) prompting is asking an AI to "think step by step" before answering. Adding that one phrase makes the model produce a visible reasoning chain — and dramatically improves accuracy on math, logic, and multi-step problems. CoT is the foundation that modern reasoning models (o1, o3, extended thinking) automate.
In 2022, Google researchers noticed something strange. If you ask a language model a math question and add "let's think step by step", the answer is far more often correct. That observation became chain-of-thought prompting — one of the simplest yet most powerful tricks in AI use.
A concrete example
Bad prompt: "A train leaves Chicago at 60 mph. Another leaves New York at 80 mph. They are 800 miles apart. When do they meet?" — AI just guesses an answer, often wrong.
Better prompt: "A train leaves Chicago at 60 mph. Another leaves New York at 80 mph. They are 800 miles apart. Let's think step by step." — AI lays out: combined speed 140 mph, distance 800 miles, time = 800/140 ≈ 5.71 hours. Far more often correct.
Why does CoT work?
Because language models generate one token at a time, each new token can use all previous tokens as context. By spelling out steps explicitly, the model has more "scratchpad" to think with. Without CoT, the model has to leap from question to answer in a single mental jump, often hallucinating along the way.
- Math problems — major accuracy gains
- Logic puzzles — far fewer mistakes
- Multi-step planning — clearer outcomes
- Coding — better debugging and refactoring
- Decision-making — model evaluates options vs picking first one
For everyday questions, CoT often makes responses unnecessarily long and slow. Use it deliberately for hard problems, not for "what is the capital of France".
How does this relate to reasoning models?
Reasoning models like o3 and Claude's "extended thinking" mode AUTOMATE chain-of-thought. They are trained to internally generate long reasoning chains for hard problems without you having to prompt for it. You get the accuracy benefit without doing the prompt engineering.
Related reading
Bottom line
Three words — "think step by step" — make AI dramatically smarter on hard problems. It is the simplest, most effective prompt-engineering trick. Reasoning models bake this in by default; for chat models, you have to add it yourself.
