Chain-of-Thought Prompting

Chain-of-Thought (CoT) Prompting is a technique that has large language models explicitly write out intermediate reasoning steps before giving a final answer. Systematically confirmed by Wei et al. (2022) in a Google paper: on math problems, logical reasoning, commonsense reasoning, CoT dramatically improves model accuracy.

Why does CoT work? From the LLM mechanism: the model generates each token by predicting the next most reasonable token from what's already generated. When models output final answers directly, they need to "compress" the entire reasoning process into a few tokens — this often fails for complex problems. When models are asked to reason step-by-step, each intermediate step becomes an "anchor" for subsequent reasoning, allowing it to continue on a correct foundation rather than guessing from scratch.

The simplest Zero-Shot CoT just adds one sentence at the end: "Think step by step." This seemingly ordinary approach was found in 2022 research to improve GPT-3's accuracy on math problems from 17.7% to 78.7% — over 4× improvement, from just this phrase.

Differences between Zero-Shot CoT and Few-Shot CoT, and when to use each.

Zero-Shot CoT: directly add "think step by step" or similar instruction to the prompt without providing any reasoning examples. Simplest implementation; effective for most tasks requiring reasoning.

Few-Shot CoT: don't just say "think step by step" — also provide several demonstration examples with complete reasoning processes (problem + step-by-step reasoning + answer), letting the model learn the "format and depth of reasoning." For tasks requiring specific reasoning formats, or where Zero-Shot CoT isn't sufficient, Few-Shot CoT typically further improves accuracy.

Selection guidance: try Zero-Shot CoT first. If the output reasoning is too shallow, skips too many steps, or accuracy is still insufficient, switch to Few-Shot CoT with 2-3 examples of complete reasoning processes.

Important detail: in Few-Shot CoT demonstrations, reasoning step quality matters more than quantity. Two deep reasoning demonstrations typically outperform five shallow ones. Steps should show "how to decompose the problem," "how to identify key information," "how to eliminate wrong options" — not just "list the steps."

Several advanced Chain-of-Thought variants suited to different scenarios.

Tree of Thoughts (ToT): lets the model simultaneously explore multiple different reasoning paths (like tree branches), then evaluates which paths are most promising and continues the best one. Suited for problems with multiple possible solutions; better at finding optimal solutions than standard CoT, but higher computational cost.

Self-Consistency CoT: has the model solve the same problem multiple times with CoT (using different random sampling), then takes the most frequently appearing answer as the final answer. More reliable than single CoT; suited for high-importance tasks where errors aren't allowed. Implementation: set Temperature > 0 in API, call 5-10 times, count the most common answer.

Step-Back Prompting: before solving directly, first have the model step back to think "what's the principle behind this problem?" "what knowledge is typically needed to solve this type?" Then use these principles to solve the specific problem. Research shows this can further improve complex problem accuracy by 10-20%.

Claude 4's Extended Thinking is the built-in version of these techniques: lets the model do longer internal reasoning in a "thinking space" before generating the final answer. For Claude API developers, Extended Thinking is currently the easiest high-quality CoT implementation to use.

How to apply CoT in actual Claude usage, and several common misuse patterns.

Using CoT in claude.ai: most direct approach is adding "please think step by step, then give the final answer" to the end of your prompt. For complex problems, be more explicit: "please first list your reasoning steps, confirm each step is correct, then give the final conclusion."

Using CoT in the API: besides adding CoT instructions to System Prompt or User Message, Claude 4 series Extended Thinking is a more direct approach — set thinking: {type: "enabled", budget_tokens: 5000} in API calls; Claude performs CoT reasoning internally and outputs only the final conclusion (or optionally also the thinking process).

Common misuse patterns:

Using CoT on tasks that don't need reasoning — translation, summarization, format conversion don't need CoT; adding it just increases latency and cost.

Few-Shot demonstration reasoning steps too shallow — "first addition, then multiplication" descriptions have limited help; need to write out the details and decision process of reasoning.

Treating CoT as a universal solution — CoT has significant effects on tasks "requiring reasoning," limited effect on tasks "requiring knowledge" (factual queries); for tasks "requiring creativity" (creative writing), CoT may make output overly structured.

What this means for your usage: if you're doing tasks requiring precise reasoning (complex analysis, math calculations, logical judgment), adding "think step by step" typically improves Claude's answer accuracy by 20-50% at almost no cost. This is one of the lowest-effort, highest-impact prompting techniques.