LLM Stacks API

Treat context as a designed input

An LLM application combines instructions, the user's request, and any supporting material. Decide which material belongs in each request and where it came from. More context is useful only when it improves the task; unrelated documents can make evaluation harder and increase processing work. Keep the information needed to reproduce a result without retaining unnecessary sensitive content.

For a document assistant, define how passages are selected, how their freshness is checked, and how the answer points back to evidence. Test questions that the available documents cannot answer. The workflow should be able to acknowledge missing support instead of turning every retrieval result into an apparently certain conclusion.

Put limits around generated actions

The OWASP prompt injection guidance explains that malicious instructions can arrive through direct input or external material, and that retrieval does not eliminate the issue. Design the application so a document cannot grant new permissions merely by containing persuasive instructions.

Give each available tool a narrow purpose and a defined input schema. Check arguments before execution, enforce the user's permissions outside the model, and require an appropriate review step for consequential changes. Keep a distinction between drafting an action and carrying it out. A generated explanation of an action is not evidence that the action succeeded.

Evaluate output as part of the product

If the application needs structured fields, validate required values and handle rejected output deliberately. Do not assume a syntactically valid object is factually correct. For prose, use task-specific criteria such as evidence support, completeness, and whether uncertainty is expressed when information is missing.

Compare changes on the same examples. Preserve prompt versions, model identifiers, retrieval settings, and the evaluation procedure. Include ordinary requests, unusual phrasing, conflicting documents, and attempts to expand the task's scope. A memorable successful demo should not carry the entire decision to release.

Plan for interruption and fallback

Decide what users see when generation stops, a provider is unavailable, or a tool returns an error. Where a fallback model is used, test its capabilities and permissions separately. The interface should make a partial draft distinguishable from a completed operation and give users a useful next step.