---
name: tai-ch098-anti-patterns-hiring-yesterday-s-tester-for-tomorrow-s
description: 'Apply chapter 98 of Testing AI, Anti-Patterns: Hiring Yesterday''s Tester for Tomorrow''s Systems, as a workflow for evaluating AI and non-deterministic systems. Use for test planning, eval design, quality review, release evidence, examples, or coaching related to anti-patterns: hiring yesterday''s tester for tomorrow''s systems.'
---

# Anti-Patterns: Hiring Yesterday's Tester for Tomorrow's Systems

Skill name: `tai-ch098-anti-patterns-hiring-yesterday-s-tester-for-tomorrow-s`

Based on **Testing AI: Engineering Confidence in AI Systems** by **Jason Arbon**.

## Purpose

AI quality teams need people who can build evidence systems, not just execute inherited test
rituals.

## Use This Workflow

- Identify the AI behavior or release decision being evaluated.
- Define realistic cases, slices, unacceptable outcomes, and evidence needed for confidence.
- Choose measurements that match the risk: rubric scores, samples, intervals, traces, human review, deterministic checks, or production monitors.
- Report uncertainty, severe failures, and decision impact instead of only a pass/fail result.

## Key Guidance

Many teams respond to AI risk by adding more traditional QA capacity. That can help for
deterministic product surfaces, but it does not solve the core AI quality problem. Tomorrow's
systems require people who can combine testing intuition with statistics, coding, product
judgment, AI tooling, data sense, and safety thinking.

## Apply The Approach

Create representative cases, score them with explicit criteria, review severe failures separately, report uncertainty, and connect the evidence to a concrete decision.

## Expert Notes

At expert level, staff AI quality as a hybrid discipline: quality engineering, data evaluation,
AI tooling, risk analysis, automation, and product judgment. This is a leverage role, not a
checkbox role.