Functional testing of enterprise applications starts with understanding artifacts such as requirements, use cases, or business rules, and includes various phases: test design, test data creation, test automation, test execution, and test maintenance. Our work primarily focuses on improving productivity and quality of testers during all these phases.

This talk primarily focuses on the test automation phase. In industrial practice, test cases often start out as steps described in natural language. These are essentially directions a human tester needs to follow to interact with an application, exercising a given scenario. Since tests need to be executed repeatedly, such manual tests then have to go through test automation to create scripts or programs out of them. Test automation can be expensive in programmer time. We describe a technique to automate test automation. The input to our technique is a sequence of steps written in natural language, and the output is a sequence of procedure calls with accompanying parameters that can drive the application without human intervention. The technique is based on looking at the natural language test steps as consisting of segments that describe actions on targets, except that there can be ambiguity in identifying segments, in identifying the action in a segment, as well as in the specification of the target of the action. The technique resolves this ambiguity by backtracking, until it can synthesize a successful sequence of calls.

We present an evaluation of our technique on professionally created manual test cases for two open-source web applications as well as a proprietary enterprise application. Our technique could automate over 82% of the steps contained in these test cases with no human intervention, indicating that the technique can reduce the cost of test automation quite effectively.

Suresh Thummalapenta is a member of the Programming Technologies and Software Engineering department at IBM Research - India. His research interests include Automated Software Engineering with emphasis on Software Testing and Mining Software Engineering Data. His main goal is to develop techniques and tools that can help deliver high-quality software more productively. His current research interests include developing new techniques that combine natural language processing and program analysis for improving productivity and quality in the delivery of testing services.

Suresh received his PhD degree in Computer Science from North Carolina State University in 2010. His adviser was Tao Xie, who leads Automated Software Engineering group at North Carolina State University. He received M.S from North Carolina State University and B.Tech from Andhra University, both in computer science. Prior to joining PhD program, Suresh worked as a software developer and test engineer in Tata Consultancy Services, India, for five years.