Recipe

Decide how to adapt a language model to a task

Use when: an off-the-shelf model doesn't yet handle a task well, and you need to decide which adaptation technique to reach for next

Goal: the cheapest technique that genuinely satisfies the task's real requirements, climbed no further than necessary

Taught in The adaptation ladder as a decision chain

1. 1

   Write the task as a well-structured prompt: instructions, context, examples, and constraints. Test it against a realistic range of real inputs, not just the first one that happened to work.

   Example

   A one-off internal summarizer is a good fit. A single well-written prompt with two examples already produces summaries an editor is happy to send as-is, across a week of real articles.

   • a well-crafted prompt gives good output across real inputs, call after call — stop here; use it as it is.
   • outputs are inconsistent, factually wrong, or need to be machine-readable — go to Step 2.

   Why: Prompting costs nothing beyond writing and testing it. There's no new system to build and nothing new to maintain, so climb further only once it clearly falls short.

2. 2

   If the content is usually right but the FORMAT wanders -- a missing field, invalid JSON, an inconsistent structure -- add a JSON schema. Layer constrained decoding or validation on top, instead of reaching for something bigger.

   Example

   An order-extraction pipeline gets the caller's name and item right almost every time. But about one response in five is missing a field or isn't valid JSON at all.

   • schema plus validation makes the output well-formed and correct, every time — stop here.
   • the model still gets facts wrong, or needs information it was never trained on — go to Step 3.

   Why: A shaky format is a structure problem, not a knowledge or skill problem. Fixing the shape solves it directly, without touching what the model knows or can do.

3. 3

   If answers are wrong or outdated because the model lacks specific facts -- private data, recent events, anything past its knowledge cutoff -- wire in retrieval-augmented generation. That means: chunk, embed, search, inject, generate.

   Example

   A support assistant confidently describes a product line. It shipped six months after the model's training cutoff, and the assistant gets the specifics wrong every time.

   • retrieved context makes answers grounded and current, every time — stop here.
   • the task needs the model to DO something in the world, not only answer with retrieved text — go to Step 4.

   Why: A knowledge gap is fixed by handing the model the right text at request time, not by changing the model itself. Training new weights just to memorize facts a search index already holds is solving the problem the hard way.

4. 4

   If the task requires taking an action -- booking something, querying a live system, computing a precise value -- rather than producing more text, give the model callable tools and a tool loop. Assemble these into an agent if the task needs several steps.

   Example

   An assistant needs to actually cancel a specific subscription in a live billing system when a caller asks for it. It should not just explain how to.

   • tools cover the needed actions and the model chooses correct calls, every time — stop here.
   • even after real effort, none of the first four rungs produce reliable behavior — go to Step 5.

   Why: An action-taking task needs a way to affect the world. Prompting, structure, and retrieval can only ever describe what should happen -- they never make it happen.

5. 5

   Fine-tune the model itself if it still can't hold a consistent output style at real volume. The same applies if it can't generalize past what fits inside a prompt's examples. It also applies if a prompt long enough to cover every case makes latency too slow.

   Why: Fine-tuning changes the model's own weights. It's the biggest change on this ladder, and also the one with the most ongoing cost and upkeep -- exactly why it's the last rung reached for, not the first.