System Prompt Design
How to write effective system prompts that hit the Goldilocks zone.
Why System Prompts Matter
System prompts are a critical piece of context engineering. They are large, structured, and carefully engineered artifacts. As LLMs evolve, system prompts evolve with them — they are an iterative engineering effort, not a one-time writing exercise.
The Goldilocks Zone
Effective system prompts balance specificity and flexibility. Too specific and you treat the LLM as a deterministic state machine. Too vague and you provide insufficient signal for consistent behavior.
Problems with Overly Specific Prompts
- ▸Hardcoded logic — prescribing exact steps for every scenario forces the model down predetermined paths
- ▸Exhaustive enumeration — listing every possible case is impossible and becomes a maintenance nightmare
- ▸Brittleness — every new edge case requires a prompt update
- ▸Wasted capability — if everything is predetermined, you don't need an autonomous agent
Problems with Overly Vague Prompts
- ▸Insufficient signal — the model can't produce consistent behavior without clear guidance
- ▸False assumptions — the prompt assumes the model knows company norms, brand voice, or policies
- ▸Undefined boundaries — vague instructions like 'escalate if needed' give no actionable criteria
- ▸No reasoning framework — different runs produce wildly different approaches to the same problem
What Good Looks Like
A well-designed prompt provides clear identity and scope, empowers rather than constrains, offers a reasoning framework instead of a flowchart, and establishes clear boundaries and heuristics.
| Principle | Description |
|---|---|
| Identity & scope | State clearly what the agent is and isn't. Customer support, not marketing. Orders, not sales. |
| Empower, don't prescribe | Define goals (resolve efficiently) instead of dictating tools for every scenario |
| Reasoning framework | Provide a multi-step approach that works across scenarios, not rigid branching logic |
| Clear boundaries | Heuristics like 'choose the simplest solution' cover many edge cases with one rule |
The best prompts teach principles instead of rules. They are efficient because they don't waste words. Each guideline covers many scenarios, with no overlapping or contradictory instructions.