---
name: tai-theme-bias-representation-and-culture
description: 'Use the Testing AI theme Bias, Representation, and Culture to plan, review, or teach related AI quality work. Applies concepts and techniques from the book to testing AI, AI-generated software, and non-deterministic systems when relevant.'
---

# Bias, Representation, and Culture

Skill name: `tai-theme-bias-representation-and-culture`

Based on **Testing AI: Engineering Confidence in AI Systems** by **Jason Arbon**.

## Theme Purpose

Use these approaches when testing bias taxonomies, cultural and language bias, socioeconomic and accessibility bias, counterfactuals, raters, deployment bias, survivorship bias, and feedback loops.

Apply these concepts when testing AI, AI-generated software, model-backed features, agents, search, chatbots, RAG systems, generated code, dynamic interfaces, or other software whose behavior can vary across runs, users, data, tools, or time.

## How To Use This Theme

- Identify the behavior, capability, risk, or release decision being evaluated.
- Choose the relevant concepts below and turn them into concrete eval cases, samples, traces, checks, rubrics, metrics, or release gates.
- Prefer evidence that supports a decision: ship, canary, hold, rollback, or collect more samples.
- Report by slices and severe failures when averages hide risk.
- Preserve enough evidence that another person or agent can understand what was tested, how it was measured, and why the recommendation follows.

## Concepts And Techniques To Apply

- Define bias types explicitly: cultural, language, socioeconomic, accessibility, representation, deployment, and feedback-loop bias.
- Use slices, counterfactuals, raters, disagreement analysis, severity, and harm framing.
- Look for survivorship bias by asking which users, prompts, traces, failures, abandoned sessions, blocked requests, or dropped cases never reached the eval set.
- Check English and Western overrepresentation, multilingual quality, dialect variation, cultural assumptions, and domain-specific harms.
- Report both measured disparity and likely user harm, including confidence intervals where useful.

## Reporting Guidance

- State what was tested and what population the evidence represents.
- Explain uncertainty, missing coverage, severe failures, and known blind spots.
- Connect findings to a concrete decision or next action.
- Use topic-specific chapter skills only when deeper detail is needed; this theme skill should stand alone as practical guidance.