Beyond Test Execution

When Development Accelerated, QA Hit a Wall

In September 2026, our engineering team introduced Claude into its development workflow. Development output increased approximately threefold, developer testing coverage improved, and the cycles of bugs and rework decreased. But the amount of work requiring QA validation increased dramatically.

As the Lead Quality Assurance Engineer at a small company, I couldn’t realistically solve that challenge by adding enough people to match the increased output. The change also exposed weaknesses in how work had historically moved from Development to QA.

Development cards didn’t always capture everything that had changed. Additional work, scope adjustments, design decisions, or effects on related functionality could go undocumented. By the time I tested a card, the developer had often moved on, making it difficult to reconstruct the full scope of validation.

AI-assisted development began producing much more detailed explanations of code changes and implementation decisions. That transparency was valuable, but it also revealed testing scenarios we had previously overlooked.

Investigating the Historical Problem

Before deciding how to respond, I used Claude to analyze our historical QA and development processes. I looked at TFS development work items from 2025 through September 2026, associated source-code changes, existing documentation, and time-tracking records. I wanted to understand the relationship between what had been documented and what had actually been implemented.

The findings were uncomfortable, but useful:

• Some development changes hadn’t been documented well enough to receive adequate test coverage.
• In some cases, I had tested something different from the implementation because the work item hadn’t clearly communicated the intended functionality.
• I had spent time testing functionality that wasn’t affected by a change.
• Some testing approaches couldn’t meaningfully validate the behavior being investigated.

In one investigation of previously completed work, we identified six missing scenarios. Further investigation confirmed a potential code defect affecting those scenarios. That reinforced a fundamental lesson: you cannot adequately test what you don’t know has changed.

This wasn’t simply an execution-speed problem. It involved information quality, traceability, communication, and deciding what actually needed validation. As coverage improved, some cards expanded from approximately six scenarios to twenty. In one workload assessment, the estimated testing effort rose from around 40–50 hours to 147 hours. Those were estimates, not hours I had already worked, but they made the limits of the existing approach clear.

The Turning Point

My initial recommendation was to increase QA staffing. Leadership saw it differently: AI was changing how companies operated, and adding headcount wasn’t necessarily the preferred response.

That was difficult to hear. I questioned whether AI could be trusted with work that had traditionally depended on human judgment. I also worried about my career. Would building these systems eventually eliminate my own position?

Shortly afterward, I went camping with my nieces. For four days—and during roughly twenty hours of driving—I kept turning the problem over. Eventually, I stopped asking only what AI might take away and started asking what I could build with it.

I remember reaching a somewhat defiant conclusion: “Alright, let’s automate myself right out of a job.”

But the idea quickly became something more constructive. If I could build a repeatable, reliable QA process using AI, perhaps I could adapt that approach to different products, teams, and organizations. The workload challenge became an opportunity to rethink how quality engineering could work.

Building the First QA Skills

I started by creating specialized Claude skills for test planning and test finalization. I tried the initial workflow against thirteen development cards from an upcoming release. Engineering hadn’t yet introduced its improved AI-generated handoff summaries, so the system had to work with the information already available on the cards.

The experiment immediately showed me where the workflow needed attention. The plans required significant review, and the rules around assumptions, reporting, and escalation weren’t strong enough. Creating instructions was only the beginning; I had to examine what those instructions actually caused the system to do.

Then I watched a demonstration of Playwright CLI and Claude coordinating multiple agents. It prompted a new question: what would testing look like if agents could work on different tasks concurrently?

I developed a manual testing skill and incorporated emulators. Soon, I could test and finalize the same cards through an AI-assisted workflow that had previously required extensive manual effort. I caught up on a release that had fallen behind during my vacation, then continued refining the skills over subsequent releases. I also shared discoveries with Engineering so we could improve our development and testing practices together.

Early Results

One of the biggest breakthroughs came when I completed testing on approximately half a release in a single day after its first deployment to UAT. Previously, I might have spent that time working through only a few cards. Some cards that had historically taken around an hour could be completed in twenty minutes or less.

These were early, observed examples—not controlled comparisons or a guarantee for every release. Still, they showed that the opportunity extended beyond executing test steps faster. The workflow reduced time spent:

• Requesting SQL queries from developers.
• Finding API endpoints and request bodies.
• Clarifying implementation details and identifying affected application areas.
• Writing extensive testing documentation during execution.
• Preparing reproducible defect reports.

The process continued finding valid failures. In some cases, the investigation surfaced code context that helped developers understand where a failure originated.

It also surfaced questions that needed human judgment. For example, outdated terminology remained in one area after a feature had been renamed elsewhere. Finding that inconsistency didn’t automatically determine its classification. I still needed to discuss with Engineering whether it was a defect, a scope change, or additional work.

Establishing Guardrails and Human Oversight

A large part of this initiative has been learning how AI makes mistakes and preventing those mistakes from silently changing the testing process.

I’ve encountered misclicks, failed interface interactions, redundant scenarios, and instructions interpreted too literally. The system has sometimes assumed an industry-standard behavior instead of checking our actual requirements, producing a false failure. It has also requested developer help before investigating available information, or attempted to adjust instructions when actual behavior differed from expectations.

I’ve established rules requiring the system to:

• Report unexpected results and stop when a step can’t be executed as instructed.
• Ask questions rather than invent assumptions.
• Request my approval before changing a test plan.
• Never silently rewrite expected results to make a test pass.
• Distinguish passed, failed, and unexecuted tests.
• Provide reproducible failure steps, screenshots, and supporting evidence.
• Retain relevant API responses and SQL query information when those methods are used.
• Request human review and confirmation of failures.

The workflow can run headlessly with multiple agents, but I supervise it through agent communications, execution reports, questions, summaries, and action items. Headless execution doesn’t mean unsupervised decision-making.

AI can execute the work, but it cannot silently redefine what success means.

Human and AI Testing Together

I’m developing a risk-based approach to deciding which work needs independent human verification. Currently, customer-critical functionality where a failure could cause a P1 incident receives manual validation alongside AI-assisted testing.

One example involved estimating and signature processes that customers rely on for essential business operations. I independently reproduced failures Claude had identified, then conducted exploratory testing and found additional issues.

I shared those findings with Claude, which dispatched agents to investigate and reproduce the new failures. The system then requested permission to add the scenarios to the test plan so they could be validated again after Engineering completed the rework.

That interaction showed how human investigation and AI execution could strengthen one another. I remained responsible for understanding business risk, exploring unexpected behavior, and deciding which discoveries should become permanent coverage.

Building Toward a Living Test Library

My work now extends beyond individual releases. I’m restructuring test cases into smaller, atomic scenarios and maintaining structured records in formats such as JSON and YAML.

The longer-term goal is a living test library connected to application functionality. Instead of recreating complete regression plans whenever code changes, I want to identify and reuse relevant tests. New functionality would introduce coverage, while existing tests would evolve with the application.

I’m also collaborating with Engineering to establish Playwright infrastructure for more extensive automated testing. The architecture I’m working toward connects requirements and acceptance criteria, application functionality, source-code changes, atomic test cases, execution results, reusable documentation, and automated regression coverage.

That library, change-based test selection, and expanded Playwright automation are still in development. They are goals I’m building toward, not completed implementations.

Questions I’m Still Exploring

The early results are encouraging, but I don’t consider the system finished. I’m continuing to investigate:

• Which features need independent human verification, and how risk should determine automation and oversight.
• Where manual testing provides the most value in an increasingly automated development lifecycle.
• How to keep coverage accurate as the application evolves.
• How to securely handle passwords, credentials, secrets, and sensitive data in AI-assisted workflows.
• How to maintain reusable documentation and automation without excessive maintenance overhead.
• How to make the approach transferable across engineering environments.

These questions are part of the engineering work. Faster execution alone isn’t enough if the process becomes difficult to trust, secure, or maintain.

What I’ve Learned

The biggest change hasn’t simply been learning to make AI execute tests. It’s been learning to look at the entire quality process differently.

I’ve had to question established practices, investigate uncomfortable findings, learn unfamiliar technologies, build new workflows, and repeatedly challenge the results produced by my own tools.

AI-assisted testing hasn’t removed the need for human judgment. It has changed where that judgment can provide the greatest value. Instead of spending every available hour on repetitive execution and documentation, I can increasingly focus on understanding risk, investigating failures, improving coverage, and helping Engineering make better decisions.

What began as an attempt to solve an unsustainable workload has become a broader interest in scalable, maintainable, and adaptable quality engineering. I’m still learning, experimenting, and building toward what comes next.

Where the initiative stands

This account uses generalized descriptions. It includes no employer source code, internal links, customer data, credentials, proprietary queries, or actual internal work-item identifiers.