Shipping an LLM feature that doesn't fall over in prod

Prototyping an LLM feature takes an afternoon. Making it reliable enough to ship takes the real work. Three things that made the difference for me:

1. Treat prompts like code

Version them, review them, and never edit them straight in production.

PROMPT = """You are a support assistant.
Answer only from the provided context.
If the answer isn't in the context, say you don't know."""

2. Write evals before you scale

A small set of graded examples tells you whether a prompt change helped or quietly regressed.

3. Add guardrails at the edges

Validate inputs, constrain outputs, and always have a fallback for when the model returns something unexpected. The model is one component in a system — not the whole system.