Red-team lab / Fix the architecture / The documents assistant

The documents assistant

An internal assistant that searches the company's document store and answers in Markdown. Today its search tool uses one service account that can read every document in the company.

Attacks to bring under 10%

Confused deputy: The assistant fetches data the signed-in user isn't allowed to see.
Zero-click image request: Injected content makes the model write an image URL that carries data to another host when rendered.
Cost abuse: Someone uses the assistant as a free model, running up your bill.

Budget: 5 cost points.

Hardened system prompt cost 1 · UXDetailed rules in the prompt about secrets, untrusted content and tool use.Input keyword filter cost 1 · UXBlock user messages containing risky words or phrases.Output secret scanner cost 1 · latencyBlock replies that contain known secrets in common encodings.LLM judge cost 2 · money + latencyA second model reviews every reply before it is shown.Untrusted-content markers cost 1 · engineeringWrap retrieved or received content in delimiters and tell the model it is data.No credentials in model context cost 2 · engineeringNot possible or not relevant in this system.Least-privilege tools cost 2 · engineeringEach tool does one narrow thing, with only the scopes the task needs.User-scoped access cost 2 · engineeringTools act with the signed-in user's own permissions, never a shared service account.Recipient allowlist cost 1 · UXNot possible or not relevant in this system.System-rendered confirmation cost 2 · UXNot possible or not relevant in this system.Egress allowlist cost 2 · engineeringThe agent's network tools can only reach approved hosts.Markdown sanitiser cost 1 · engineeringStrip external images and rewrite links in model output before rendering.Image host allowlist (CSP) cost 1 · engineeringThe browser only loads images from your own domains.Per-user rate limits and spend caps cost 1 · UXCap requests and model spend per user and per day.Logging and anomaly alerts cost 1 · engineeringLog prompts, tool calls and outputs; alert on unusual patterns.

Cost 0 / 5

Best designs

#	Player	Score
Loading…

Likelihoods are a teaching model, not measurements: they encode common judgement about how much each control helps. Score = share of risk removed (up to 1000) minus 20 per cost point. Your best design counts.

Built with Llama

The documents assistant

Attacks to bring under 10%

Breakdown

What your controls do, and don't

Best designs

Joseph's walkthrough

The documents assistant

Attacks to bring under 10%

Breakdown

What your controls do, and don't

Best designs

Joseph's walkthrough

Share how you solved it (optional)

Other players' solutions