Commitments around AI-driven savings in testing often get written into the contract during the deal phase, before anyone has verified that the delivery team’s own testing capability can support them. Those savings rarely show up, not because the AI tooling failed, but because the commitment was made against a foundation nobody assessed. By the time the gap surfaces, it is a delivery problem and a client conversation nobody wants to have, instead of something caught before the ink dried.
Ask any QA leader in an Indian enterprise whether AI should be part of the testing strategy, and the answer will always be an enthusiastic yes. But ask how much of their day-to-day testing uses AI, and the answer is likely to be very different.
The disparity between how enterprises perceive AI in testing versus how much they use AI in testing today is backed by industry data. According to the Leapwork and SD Times survey of 300-plus QA professionals, 88 per cent of respondents said that AI was a priority for their organisation’s testing strategy, yet only 12.6 per cent said they were currently using it for testing daily. As per the World Quality Report of 2025, only 15 per cent of respondents have achieved organisation-wide AI implementation, while 43 per cent remain in the experimental phase and 30 per cent of them are operating within limited use cases.
This huge disparity leads to the question of how senior leaders of companies can continue to insist that testing can be done by AI agents to replace human testers. However, the root cause of this gap is not AI.
Why Does This Hit Indian Enterprises and IT Services Harder?
Indian IT services teams routinely take over projects that never had basic testing processes in place to begin with, let alone a usable set of test cases. There is no clean automation layer to add AI on top of, because there was never a manual functioning process to automate in the first place. AI gets asked to accelerate testing on an application that was never properly tested to start with, and the gap between what was promised and what gets delivered traces straight back to that inheritance, not to anything the AI did or did not do.
Most SOWs only account for business-as-usual delivery and do not consider testing transformation in scope. Even though the team recognises there are inherent gaps in the existing processes, tools and frameworks, there is no budget or resourcing consideration made, knowingly or unknowingly. A few customers wrongly assume that the transformation is a free service the service provider would bring in. On the other hand, service providers often refrain from raising a change request, and the gaps get carried forward instead of closed.
What Is Stopping QA Teams from Adopting AI?
Most QA processes are not designed for AI-assisted testing workflows. Instead, they were designed for human-driven activities like manually creating, reviewing and maintaining test-related assets. Trying to force AI into existing workflows without redesigning them for AI-generated output does more harm than good.
Rather than solving the real problem with automated testing, most organisations use test automation to generate and run more tests faster. Large and unstable test automation systems are now too expensive to maintain. Minor changes to the UI, test environment problems, inadequate test data and test dependencies cause many false failures that QA engineers must spend time investigating and repairing.
Another challenge is the quality of user stories that are used to drive testing and development. AI is rarely used to improve the quality of user stories before development begins. Instead, QA teams use ambiguous requirements and end up designing test cases with poor coverage and quality.
The AI testing tool itself is blind and relies on how the AI was trained and the input that the human provides. A generic prompt, for example, will generate generic tests that pass a syntax check but fail to incorporate the critical logic that makes the application under test different from all others. In many cases, teams end up with more test cases, not better test coverage.
Many organisations today fail to thoroughly review AI-generated test cases in the same manner that they would review test cases written by test engineers. The large volume of test cases generated along with an organisation’s confidence in AI may lead to directly using these test cases generated by AI into test execution. Although the intention was to save time, the time spent fixing failures encountered during test execution far outweighs the time spent reviewing AI-generated test cases.
What Does a Passed AI Pilot Not Prove?
The success rate of a pilot reflects more the things it has not tested than the things it has proved. The application defects and weaknesses that have been built into the test suite remain unknown because the pilot was not motivated to test for them. More importantly, the application’s poorly written user stories rarely get exposed to a pilot. This is because the people executing the pilot already understand the application and the requirements well enough to interpret any ambiguous requirement.
The AI-generated test cases that were created to support the pilot would likely not pass a real code review. But the code review activity itself might be skipped as many pilots do not follow a defined process.
Often a single application is chosen for a pilot, and even within that single application, just a single feature is chosen to test. In one engagement that I reviewed for an e-commerce client, the pilot covered a single checkout flow out of twenty that ran in production.
There are rarely any measures in place to test the successful completion of a pilot. These would typically include some form of measurement for the quality of the output generated by the AI, the amount of rework required to fix items generated by the AI, and the test coverage achieved. Without these metrics, it may not be possible to validate the outcomes of a pilot.
By the numbers
Key figures from this story- 88%
- QA professionals who say AI is a testing priority
- 12.6%
- who actually use AI for testing daily
- 15%
- with organisation-wide AI implementation
After the pilot has successfully proven out the demo for that one app or feature, it gets roped into the larger roll-out plans for the wider organisation. Suddenly everything that was not tested by the pilot is tested and fails. The previously stable but never-tested test suite breaks under real load. The gap in the requirements for test cases that the one experienced tester knew to fill in for the pilot, but no one else knew to implement for subsequent test runs through AI, gets discovered as incorrect test coverage. The AI had succeeded in proving out a demo, not an AI success story.
The Real AI Testing Strategy Starts Before the AI
AI does not automatically fix a weak testing process. It will only expose the weaknesses that already exist and, in some cases, magnify them at scale. Indian enterprises and IT services companies need to move beyond treating AI adoption as a scorecard of completed pilots, purchased tools and impressive leadership presentations.
Before scaling out AI in testing, one must first fix what it will be running on. This means fixing the poor requirements that lead to problems downstream including weak test design and poor automation. A solid QA process that has been updated to use the AI tools in testing has to be designed and implemented. This process should include review steps for all test cases and automation code generated by AI.
Someone needs to own approving the test cases and automation code generated by AI. The test automation framework must be designed in such a way that it is maintainable and reliable. Measurements need to be more than just tool adoption and pilot completion. Defining metrics such as test coverage, rework ratio, defect detection and maintenance effort will help to measure the true quality of the AI-generated test cases and automation code.
AI in testing can scale from pilot mode to something that can have a measurable and significant impact once the right foundation is in place.
The author is a software engineering leader with 25-plus years of experience in QA, DevOps, automation and AI. Views are personal.
