Part 11 — Making LLMs useful

This part is the highest-value-per-hour section of the guide for most people. It ends with Project 3: the Help Desk answerer you've been building toward since Chapter 4.

Chapter 44 — Prompting that actually works

Why prompting works at all

Everything you write is context that shapes p(next token). A vague prompt leaves the distribution wide; a precise one narrows it toward the answers you want. Prompting is not magic words — it's conditioning a probability distribution.

The seven techniques that matter

1. Be specific about the task, format, and audience.

Weak:    "Summarise this."

Strong:  "Summarise the document below for a non-technical executive.
          Write exactly 3 bullet points, each under 20 words.
          Focus on financial impact. Do not mention implementation details."

2. Give examples (few-shot). Showing beats telling, especially for format.

Classify the sentiment.

Review: "Arrived broken and support ignored me."     Sentiment: negative
Review: "Works exactly as described. Very happy."     Sentiment: positive
Review: "It's fine. Nothing special."                 Sentiment: neutral

Review: "Shipping was slow but the product is great." Sentiment:

Rules: 2–5 examples is usually plenty; keep the format perfectly consistent; include the tricky/edge cases; put them near the start of the prompt.

3. Ask for reasoning before the answer (chain of thought).

Weak:    "A shop sells pens at 3 for 2 EUR. How much for 17 pens?"

Strong:  "A shop sells pens at 3 for 2 EUR. How much for 17 pens?
          Think step by step, showing your working, then give the final
          answer on a line starting with 'ANSWER:'."

Why it works: each generated token is another forward pass. Reasoning tokens give the model computation space — it literally cannot do multi-step work in one token. (Newer "reasoning models" do this internally by default; for others, ask for it.)

4. Assign a role, when it genuinely narrows the task.

"You are an experienced technical writer reviewing API documentation for
 clarity and completeness."

Useful because it conditions on a style and standard of work. Don't over-believe it: "You are a world-class expert" does not make it correct.

5. Structure your prompt with delimiters.

### INSTRUCTIONS
Answer using only the context. Cite the document ID for each claim.
If the answer is not in the context, reply "Not found in the handbook."

### CONTEXT
[DOC 1] Employees receive 25 days of paid holiday per calendar year.
[DOC 2] Parental leave is 26 weeks at full pay.

### QUESTION
How much holiday do I get?

### ANSWER

Clear sections stop the model confusing instructions with data — and they're the first line of defence against prompt injection (Chapter 49).

6. Tell it what to do, not only what not to do.

Weak:    "Don't be verbose. Don't use jargon. Don't make things up."
Strong:  "Answer in 2 sentences using plain language. If you are unsure,
          say 'I'm not certain' and explain what information is missing."

7. Give it an escape hatch. Without permission to fail, a model will invent something.

"If the context does not contain the answer, reply exactly:
 'Not stated in the provided documents.'"

The prompt skeleton to start from

[ROLE / PERSONA]           who the model is being, if it helps
[TASK]                     what to do, precisely
[CONTEXT / DATA]           the material to work from, clearly delimited
[EXAMPLES]                 2-5, consistent format          (optional)
[CONSTRAINTS]              length, tone, what to avoid, escape hatch
[OUTPUT FORMAT]            exact structure, ideally with a template
[THE INPUT]                the actual thing to process    <-- last

Two reasons the input goes last: it's the most recent thing (models attend strongly to the end — Chapter 45), and it keeps everything before it stable, which enables prompt caching (Chapter 46).

Practice (30 min)

Take one real task you'd like an LLM to do. Write three prompts:

A one-line lazy version.
A structured version using the skeleton above.
The structured version plus 3 few-shot examples.

Run each 5 times and score the outputs yourself out of 5. Record the results in notes.md.

Almost everyone is surprised by how much version 2 beats version 1 — and by how often version 3 beats version 2 for format-sensitive tasks.

Common confusions

"Prompt engineering is dying because models are smarter." Partly true for simple tasks. But specifying the task clearly, structuring context, and giving escape hatches will always matter — that's just communication.
"Longer prompts are better." No. Relevant prompts are better. Irrelevant context actively hurts (Chapter 45) and costs money.
"There are magic phrases." Mostly folklore. Clarity, examples, structure, and reasoning space are what reliably work.

Check yourself

Why does "think step by step" help?
Why put the user's input at the end?
What is an "escape hatch" and why does it reduce hallucination?