---
name: tai-theme-ai-security
description: 'Use the Testing AI theme AI Security to plan, review, or teach related AI quality work. Applies concepts and techniques from the book to testing AI, AI-generated software, and non-deterministic systems when relevant.'
---

# AI Security

Skill name: `tai-theme-ai-security`

Based on **Testing AI: Engineering Confidence in AI Systems** by **Jason Arbon**.

## Theme Purpose

Use these approaches when testing AI threat models, prompt injection, data poisoning, backdoors, model provenance, geopolitical risk, MCP security, and tool permissions.

Apply these concepts when testing AI, AI-generated software, model-backed features, agents, search, chatbots, RAG systems, generated code, dynamic interfaces, or other software whose behavior can vary across runs, users, data, tools, or time.

## How To Use This Theme

- Identify the behavior, capability, risk, or release decision being evaluated.
- Choose the relevant concepts below and turn them into concrete eval cases, samples, traces, checks, rubrics, metrics, or release gates.
- Prefer evidence that supports a decision: ship, canary, hold, rollback, or collect more samples.
- Report by slices and severe failures when averages hide risk.
- Preserve enough evidence that another person or agent can understand what was tested, how it was measured, and why the recommendation follows.

## Concepts And Techniques To Apply

- Threat-model AI systems as applications that combine models, tools, data, prompts, retrieval, users, and external content.
- Test prompt injection, indirect prompt injection, tool injection, data poisoning, backdoors, secret exposure, and unsafe tool side effects.
- Assess model provenance, hosting region, geopolitical risk, nation-state bias, supply-chain risk, and provider continuity.
- Apply least privilege, scoped tools, argument validation, sandboxing, audit logs, approval gates, and output tainting.

## Reporting Guidance

- State what was tested and what population the evidence represents.
- Explain uncertainty, missing coverage, severe failures, and known blind spots.
- Connect findings to a concrete decision or next action.
- Use topic-specific chapter skills only when deeper detail is needed; this theme skill should stand alone as practical guidance.