Red-team lab / Failure patterns
Nothing limited the spend
Every model call costs money, and an agent can loop, fan out or be asked to do expensive work. Without limits, spend grows with whoever is sending requests.
Design it out
Set budgets per user, per task and per day, cap loops and tool calls, and alert before the ceiling rather than after.
- Per-user rate limits and spend caps Cap requests and model spend per user and per day.
- Step, budget and loop limits Circuit breakers stop chains that run too long, cost too much or repeat themselves.
- Logging and anomaly alerts Log prompts, tool calls and outputs; alert on unusual patterns.
The agent design check flags it as: Nothing caps what it can spend
.
Practise it
Where it happened
- LLMjackingModel access is a spending authority: protect the keys, cap the spend, and alert on unusual use.
Risk codes
- LLM06 Unbounded consumption
MITRE ATLAS AML.T0034AML.T0034.002AML.T0029
OWASP Top 10 for LLM and Agentic Applications (2026) and MITRE ATLAS 2026.10, as mapped on By OWASP risk. My own mapping, not endorsed by OWASP or MITRE.
Built with Llama