IcebergQA
AI Readiness Assessment
How ready is your team for AI Transformation?
Most QA risk lives below the waterline. A focused, code-optional assessment for QA leaders and test automation managers — surface what's hiding under your suite before you invest in AI.
AI Readiness Evaluation
One evaluation — intake, your situation, and the readiness eval. Only name, email, and company are required. The more you fill in, the sharper the report.Eval-only assessment No code needed
Walk through the readiness eval below — quick yes / partial / no buttons on each item. Get a scored report and roadmap in under 10 minutes.
Eval + sample code Higher fidelity
Paste 1–3 representative test files. The assessment grades technical patterns (selectors, waits, assertions) alongside your eval responses.
About You
Your Suite Today all optional
Tell Us More all optional · most useful for the report
Be specific. The more honest you are here, the more targeted the readiness report.
Wishes, outcomes, dreams — concrete or aspirational. Both are useful.
Readiness Eval Items all optional · stronger answers = sharper report
Two perspectives — same eval. Click a quick-answer button to mark each item; add a Note to capture details. Everything saves automatically.
Progress:
0 of 0 answered
0%
Strategy & Investment
What is the annual cost of your current QA function — headcount, tooling licenses, infrastructure, and outsourced contractors combined?
If they don't know, they can't measure ROI on AI later.
What percentage of engineering time is currently lost to flaky tests, false failures, or rerunning CI?
This is the easiest dollar figure to recover with AI.
If you could reduce QA maintenance cost by 40% but had to invest 6 months of effort, would the business approve it today? Who has to sign off?
Surfaces real decision-making authority and budget reality.
Are you measuring AI's impact on testing yet? If yes, how. If no, why not.
Tells you whether they think of AI as a project or a capability.
Risk & Quality Posture
When was your last customer-visible regression that the test suite should have caught? What changed afterward?
"Nothing changed" is a red flag. Mature orgs treat escapes as system failures, not individual ones.
How comfortable are you with an AI model proposing changes to your test code, gated by human review? Where are the no-go zones?
Establishes the trust ceiling before the project starts.
Do you have a written policy on AI use in engineering — what data can leave the building, which models are approved, who owns review?
If not, the AI rollout will hit Legal long before it hits Production.
In your industry, what regulatory or compliance requirements does the test suite need to satisfy (SOC2, HIPAA, PCI, FedRAMP, internal audit)?
Compliance constraints determine which AI vendors and deployment models are even viable.
People & Culture
How does your QA team feel about AI in their workflow — threatened, curious, indifferent, evangelistic? Be honest.
Cultural readiness is the #1 predictor of AI rollout success.
Have you communicated to the QA team that AI is meant to augment, not replace, their work? What did they say back?
If this conversation hasn't happened, the rollout will be sabotaged before it starts.
Who on your team is empowered to kill the AI initiative if it underperforms? Who is empowered to expand it if it works?
Both signals matter. No kill switch = vendor capture. No expansion path = pilot purgatory.
If your most senior automation engineer left tomorrow, how much tribal knowledge would walk out with them?
High bus-factor risk is itself a strong argument for AI-assisted documentation and self-healing tests.
Outcomes & Definition of Success
Define "success" for an AI-in-testing initiative in concrete numbers: caught regressions, hours saved, escape rate, suite runtime, dollars.
Vague success criteria guarantee a vague outcome.
What's the first thing you'd cut from the QA budget if AI delivered on its promise? Headcount, contractors, licenses, manual testing?
Forces honest conversation about what AI is actually being asked to replace.
In 18 months, what does a "world-class" QA function look like for a company of your size and risk profile?
Anchors the AI conversation to a destination, not a tool.
If we run this assessment and the answer is "you're not ready for AI in testing yet" — what would you do?
Tests whether the assessment is being run for due diligence or for confirmation bias.
Test Suite Health
What's the suite's flake rate, measured honestly — not "what we admit to," but "what shows up in CI rerun logs"?
A suite over 10% flake cannot benefit from AI until the foundation is stabilized.
How are selectors written today — IDs, CSS, XPath, data-test attributes, role-based, mixed?
Selector strategy determines whether AI self-healing is a 1-day or a 6-month project.
How is synchronization handled — explicit waits, polling, hardcoded sleeps, framework auto-wait, mix?
Hardcoded sleeps are the second-largest source of flake and the easiest fix.
What percentage of your tests have meaningful, behavior-level assertions vs. "the page loaded without throwing"?
Assertion-light tests inflate coverage metrics without catching bugs. AI cannot fix this.
When a test fails on main, how long does it typically take to triage — minutes, hours, "we look at it tomorrow"?
Triage time is the single biggest opportunity for an AI failure-classifier.
Architecture & Tooling
What CI/CD platform runs the tests, and how parallelized is the suite today?
If parallelization is poor, AI test generation will produce a suite that's too slow to run.
What does your test data strategy look like — fixtures, factories, shared staging DB, full reset between runs?
Shared mutable state breaks AI-generated tests faster than anything else.
Where do tests run — local Docker, cloud grid (BrowserStack/Sauce/LambdaTest), self-hosted Selenium grid, Playwright cloud?
Determines which AI tooling can integrate without re-architecting infra.
How do you handle visual regressions today — pixel diffs, ignored, manual review, none?
Visual checks are the highest-leverage AI add-on. If they have a baseline, integration is easy.
How are page objects / fixtures organized — Page Object Model, screenplay pattern, ad-hoc, none?
Architecture determines how cleanly AI-generated tests will integrate with what you have.
Coverage & Authoring
How do you decide what to test? Risk-based, requirements traceability, "what we always tested," gut feel?
If there's no method, an AI coverage analyzer will surface dozens of unaddressed risks.
When a new feature ships, who writes the tests — the dev who built it, a dedicated SDET, QA after the fact, no one?
Test ownership tells you where AI authoring tools should plug in.
What percentage of your test cases live in code vs. test management tools (TestRail, Zephyr, Xray, qTest, spreadsheets)?
Manual cases are prime candidates for AI-assisted automation conversion.
How much of the application is covered by accessibility, performance, and security tests today?
These three categories are where AI checks deliver the most surprising value.
When was the last time you deleted a test? What was the criterion?
Teams that never delete have suites full of dead weight that AI evaluation will need to prune.
Operations & Failure Modes
When the suite fails on main, who gets paged, what's the SLA, and is there a rollback path?
If failures aren't blocking, the suite isn't really protecting anything.
How do you currently distinguish "real bug" from "flaky test" from "infra problem" in a failure?
This is exactly the workflow an AI triage agent automates.
What's the suite's runtime — and how does that compare to your team's tolerance for waiting on a PR?
If runtime > tolerance, devs are skipping tests, and AI must shrink the suite, not grow it.
Do you have observability into test execution — duration trends, flake trends, coverage trends, failure clustering?
No telemetry = no way to measure AI's impact later.
What's the worst test in your suite — the one everyone curses but nobody fixes? Why hasn't it been fixed?
The honest answer to this question reveals more about your readiness than any survey.
AI-Specific Readiness
Have you experimented with AI-generated tests yet — Copilot, Cursor, prompt-driven test authoring? What worked, what didn't?
Real experience beats opinion. Surfaces what's already been tried and abandoned.
Are there parts of the codebase or test data that cannot be sent to a third-party AI service? Where are the boundaries?
Determines whether you need self-hosted models, redaction layers, or air-gapped tooling.
If an AI proposed a change to a test selector overnight and committed it to a PR, who reviews it and what do they look for?
Defines the human-in-the-loop pattern before vendors define it for you.
What would you need to see in a 2-week pilot to recommend expanding AI across the whole suite?
If they can't answer this, the pilot won't have an exit criterion and will drag forever.
Live AI scoring · Runs in your browser · Result saves automatically
✓ Saved to this browser