Match the mechanism to the uncertainty
Use retrieval when the model needs access to changing or private knowledge. Use tools when the application must perform a controlled operation. Use structured outputs when downstream code needs a predictable shape.
They can work together, but each adds latency, failure modes, and evaluation work.
Avoid the universal assistant
A broad prompt that can search, calculate, mutate, and explain everything is hard to secure and harder to test. Narrow capabilities create clearer contracts and better user expectations.
The design question is not which technique is most fashionable. It is which uncertainty belongs to the model and which belongs to deterministic application code.
A decision tree
Ask what the model needs to know and what the application needs to do. If the answer requires current private information, retrieve it. If the answer requires changing state, call a tool. If another program must consume the result, enforce a schema.
flowchart TD
A[Task] --> B{Needs private or changing knowledge?}
B -->|Yes| C[Retrieve relevant context]
B -->|No| D[Use prompt context]
C --> E{Needs a side effect?}
D --> E
E -->|Yes| F[Call typed tool]
E -->|No| G[Generate response]
F --> H[Validate result]
G --> I[Validate output shape]
H --> J[Return or review]
I --> J
Retrieval is not a truth guarantee
Chunking, ranking, and citations determine whether the model receives useful evidence. Store document identity and version with each chunk, and show the source to the user when the answer affects a decision. If retrieval returns weak evidence, the system should say that it does not know rather than fill the gap.
Tools need application contracts
Tool descriptions help the model choose, but they do not enforce safety. The server validates arguments, authorizes the actor, checks current state, and returns a bounded result. The model is a caller of the API, not the owner of the API's rules.