Advanced prompting techniques

What the five rungs are, and what a higher rung means

All five techniques in this lesson do one thing: they add something to the prompt that was not there. Zero shot means you simply ask. Few shot means you also include a handful of correct input and output pairs. Chain of thought means you ask for the steps before the answer. Structured output means you fix the shape of the answer so that a program can read it. A system prompt means rules that apply to every message and are not repeated each time.

Here is the point most tutorials lose: this ladder is not a quality scale. A higher rung does not give a better answer, it makes a bigger prompt. Every rung you climb adds three things: tokens, latency, and one more thing that can be wrong. An example that contradicts your own rules is worse than no example.

So the working rule is this: start on the first rung and climb only when an output has shown you where it fell short. Four out of five prompts in real work stay on rung one, and that is not a sign of a beginner.

Giving examples: how many is enough, and when it backfires

The term few shot comes from the 2020 OpenAI paper titled "Language Models are Few-Shot Learners", and the idea is simple: instead of describing the task, show a handful of examples of it. For work whose shape is easier to see than to describe, this rung genuinely pays: classification, tone, and any format with fussy details.

A real example. Say you want to sort incoming customer messages into four buckets. Zero shot, the prompt says "put this message in one of these four buckets" and the model gets most of them right and improvises on the borderline ones. Few shot, you choose those borderline ones yourself:

Message: Hi, order 123 arrived yesterday but the box was dented. The item inside is fine.
Bucket: complaint

Message: Could you tell me when order 123 arrives?
Bucket: order status

Message: Hello. I want three more of the last one.
Bucket: new order

The value of those three lines is in the thing you could not write down: that "dented box, item fine" still counts as a complaint is your decision, not a general rule. Examples carry exactly those borderline decisions.

On how many, there is no magic number and anyone who gives you one is guessing. OpenAI own guidance for reasoning models says to try zero shot first and add examples only if needed, and warns on the same page that examples must align closely with the instructions, because a discrepancy between the two produces poor results.

Three places where examples backfire: when they all come from one bucket and the model copies that distribution; when one example contradicts one of your own rules, in which case the model believes the example rather than the rule; and when the examples are lifted from real client data, because that data now travels with every single run.

Where chain of thought came from, and why it is needed less now

January 2022, a paper from Google researchers appeared under the title "Chain-of-Thought Prompting Elicits Reasoning in Large Language Models". Its claim was that if your examples also show the intermediate steps to the answer, sufficiently large models do better on arithmetic and logic problems. That paper is where "think step by step" entered every tutorial.

Nearly five years later, the vendors own guidance says something else. The OpenAI page on getting the best out of reasoning models says to keep prompts simple and direct and to avoid chain of thought prompts, because these models reason internally and telling them to think step by step is unnecessary. Anthropic documentation on extended thinking says to prefer general instructions over prescriptive steps: a prompt like "think thoroughly" often produces better reasoning than a hand written step by step plan.

So the technique did not become wrong; it moved. It went inside the model. Two places still want it: smaller and older models with no internal reasoning, and any time you want to see the steps so you can argue with them. The second is a good reason, and the reason is inspection rather than answer quality.

One warning you will rarely read: the steps a model prints are not a report of what happened inside it. That text was generated the same way the rest of the text was. An answer can be right with irrelevant steps, or the steps can be convincing and the answer wrong. Treat it as an explanation offered to you, not a stamped receipt. That is our position, and it follows from the mechanism described in the lesson on large language models.

Take the date at the top of this page seriously. Vendor guidance is exactly the kind of thing that turns over fast here, and this passage is exactly the kind of claim that may be half wrong a year from now.

Structured output: when the answer has to enter a program

This is the one rung that nearly always pays for itself in real work, and the reason is not text quality: an answer with a fixed shape can be checked by machine. As long as the output is a fluent paragraph, the only way to evaluate it is to read it. When the output is a record with named keys, five lines of code tell you which record is broken.

There are two ways to get it and the difference matters. The first is to ask for the shape in the prompt and hope. The second is to use the mode the vendors built for this, which constrains the output to a schema; Anthropic and Google both keep separate documentation for it. If you work through an API, take the second. If you work in a chat window, the first is your only option and then checking the output is your only guarantee.

A trap specific to Persian and Arabic writers that you will not find in any English documentation: if your rules say "a number between 1 and 3" using Persian or Arabic-Indic digits, the model may well answer with those digits, and the output stops being valid JSON, because JSON accepts Latin digits only. The fix is one sentence: say in the rule that digits must be Latin. We put that into the recipe below.

And a habit worth more than the technique: never let a structured output directly perform something that is hard to undo. Sending an email to a customer, filing a return, changing a price. Put a person between the output and the action, at least until you have run it over a hundred real records.

System prompts: rules you do not repeat every time

The system prompt is where the standing rules of your work sit, applying to every message in that conversation. In an API it is a separate parameter and Anthropic keeps a dedicated documentation page for it. In chat products it is the thing you meet under names like custom instructions or project instructions.

What belongs there: who you are and who you write for, the constraints that never change, and output conventions such as never wanting a bulleted list. What does not belong there: the details of one specific task, anything that changes per request, and obviously no passwords or keys.

A point missing from most tutorials: a system prompt is not a security boundary. It is a rule the model usually follows, not a wall nothing crosses. Text you paste in your next message, the contents of a web page or a customer message, can carry an instruction inside it that contradicts your rules. That is the thing named properly in the block further down this page.

Where not to put these techniques

Four situations where climbing the ladder is only a cost. First, a one off: something you do once is not worth examples, a schema and a system prompt; ask, read the answer, done. Second, when the problem is missing information: if the right answer needs data the model does not have, no rung conjures it and the fix is supplying the data.

Third, when the output is for a person rather than a program. Structured output there only adds friction and makes the text stiffer. Fourth and biggest: when you have no way to check the result. More technique makes a wrong answer look more confident and more orderly, which makes it more dangerous. If you cannot say what a correct output looks like, solve that first.

One summary worth repeating: of these five rungs, the one with the highest return in day to day work is structured output, because it is the only rung that makes quality measurable. The others change the quality and leave the measuring where it was.