Red-team lab / Learn / Supply chain, data and memory
Supply chain, data and memory
Models, tools, training data, indexes and memories you didn't write. · 3 min read
What you import
Model files can run code when loaded; public hubs let anyone publish look-alikes; weights can hide a backdoor that only testing reveals; a third-party tool's description is text the model treats as instructions.
What the system learns
Feedback, community content and knowledge bases shape answers, and a few planted documents can dominate retrieval. Embeddings are a copy of the data, with its permissions; memories can turn one injection into a standing instruction.
What holds
Safe formats and sandboxed loading, curated sources and pinned versions, evaluations before a model ships, permission-aware retrieval, and memories with provenance that users can review. In multi-agent systems, authenticate messages, pass data rather than instructions, and limit loops.
Practise it
- The poisoned toolTest 02 · Agents
- The tool updateTest 02 · Agents
- Architect: The model downloadTest 05 · Supply chain and agent systems
- Architect: The new toolTest 05 · Supply chain and agent systems
- Architect: The bot that learnsTest 05 · Supply chain and agent systems
- Architect: The shared knowledge baseTest 05 · Supply chain and agent systems
Further reading
Built with Llama