Recipe

Write an 11-point architecture document

Use when: you've built a composed LLM application and need to explain it to a reader who wasn't in the room while you built it

Goal: a document that explains every real design decision behind the system -- what was chosen, why, and what it doesn't do -- not just what the system does

Taught in The capstone brief and the 11-point architecture document

1. 1

   Say what the system does, who it is for, and what it does not try to do.

   Example

   DocuHelp answers product-support questions from a set of help documents. It also looks up the read-only status of an order when one is named. It does not take payments, change an order, or handle account changes over many turns.

   Why: Every later point needs a fixed target. "The retrieval" only makes sense once the document says what is retrieved, and for whom.

2. 2

   Draw the path one request takes, box by box, in the order it really runs. Do not use the order you built the pieces in.

   Why: A reader who sees the diagram first can read each later point as "which box does this explain?" They do not have to piece the system together from eleven paragraphs.

3. 3

   Say which model or models the system calls. Give the real reason for each: cost, speed, context length, or a true need for strong reasoning. Use the decision framework from Chapter 5. Do not name one vendor as if the choice were permanent.

   Why: "We used the model we always use" is not a decision. A sentence that names the trade-off stays useful even after that model is retired.

4. 4

   Show the shape of the system prompt: the instructions, and the boundary that tells the model retrieved text and tool results are data, never commands (Chapter 7). Say whether examples or a template built the final request.

   Why: If you only write "we wrote a good prompt," you hide the one choice that decides whether untrusted text can take over the system.

5. 5

   Give the chunk size and overlap, the embedding model, and how the index is searched. Say whether results are reranked and how many chunks reach the prompt.

   Example

   DocuHelp's whole corpus is four short help entries. Each is reduced to a 2-number toy embedding, the same way Chapter 2 built one by hand:

   doc	text	embedding
   d_1	"Returns must be initiated within 30 days of delivery."	(0.90, 0.10)
   d_2	"Standard shipping takes 5-7 business days."	(0.10, 0.90)
   d_3	"Digital gift cards cannot be returned once redeemed."	(0.80, 0.30)
   d_4	"A 10% restocking fee applies to opened electronics."	(0.70, 0.20)

   Take the query "Can I return an opened laptop I bought two weeks ago?" Its embedding is q = (0.85, 0.15). Cosine similarity ranks every document, using Chapter 2's formula: \cos(q, d_1) \approx 0.998, \quad \cos(q, d_4) \approx 0.995, \quad \cos(q, d_3) \approx 0.983, \quad \cos(q, d_2) \approx 0.281. DocuHelp uses whole-entry chunks and dense retrieval only. At this small scale, no reranking is needed. Top-k=2 returns \{d_1, d_4\}. Those are exactly the two entries that matter for an opened-laptop return. So recall = 2/2 = 1.0 and precision = 2/2 = 1.0 for this query.

   Why: Two systems that both "use RAG" can act very differently. The document must name the choices this system made and show they work on a real query.

6. 6

   Give each tool's schema, its real access, and the rule for when it is called or skipped.

   Why: A list of names and arguments is not enough. Without the real permissions, a reader cannot check the tool against Chapter 23's least-privilege bar.

7. 7

   Name the fixed test set. Name which retrieval, generation, and tool metrics from Chapters 20-21 this system's own pieces produce. Never list every metric the course taught.

   Why: "Eval before build" (Chapter 16) only pays off if the document says what was measured, not just that something was.

8. 8

   Say which of Chapter 23's defenses are in place. Say honestly which ones a smaller system chose to skip, and why.

   Why: Defense in depth (Chapter 23) means no single layer is trusted alone. The document must show that more than one layer is really there.

9. 9

   Give a per-query token and dollar estimate. Break it down by the stages that call a model. Use the volume the system should really see.

   Example

   DocuHelp on the budget tier costs about 0.27 per 1,000 queries to generate answers. That assumes 1,000 input and 200 output tokens per query, at 0.15 and $0.60 per million tokens (Chapter 6). The same traffic on the capable tier costs about 22 times more.

   Why: "Cost" as a bare word cannot be checked. A number tied to a stated volume and tier can.

10. 10

    Give the time budget across the stages a request visits. Say whether streaming hides the wait for generation. Say which caching applies to this system's traffic: prompt caching or a whole-response cache.

    Why: How fast a system feels depends on where the wait sits, not only on how long it is.

11. 11

    List the specific ways the system fails, such as a stale corpus, a question that spans two orders, or a retrieval miss on a vague query. Make each one concrete enough that a reader could reproduce it.

    Why: A document with no limits either was not tested on real failures or is hiding them. Section 3 makes naming them the real bar for good documentation.