Improving Test Coverage of Ajax Applications

Dietrich Kappe, Monday, October 9, 2006 @ 8:00 am

It has been recognized for some time that Ajax applications are a different animal when it comes to testing. In the world of pre-AJAX web applications, the state of the art involved recording HTTP transactions and scraping the resultant HTML pages to check for correct results. That was fine as long as the application was the typical form based, rectangular HTML webapp. With Ajax, all sorts of other considerations come into place, like whether a particular drag and drop action works, not just if the recorded HTTP Request that it's supposed to generate results in the right HTML page or XML message.

The shortcomings of the protocol and screen scraping approach have lead to more sophisticated solutions such as Selenium, a browser based testing solution. With Selenium, all that Javascript can execute and all of those DOM changes can happen and your test scripts can excercise the application through the much more fine grained issuing of interface events. Problem solved, right?

Well, not really. If you're implementing a document based rather than a form based application, getting good test coverage of an Ajax application can be problematic. In the case of a form based application, you have a limited number of paths, so path based, statement based or condition based code coverage measures can all work fine. Now imaging trying to do path based code coverage for Microsoft Word or Excel; the number of interface choices at each point can mushroom like crazy. If the number of UI event choices at each point are no more than 10, then the number of paths after n operations is 10^n.

Fortunatley, this problem of code coverage has been faced once before -- during the move from form-based and command-line based applications to GUI applications. Lots of good research has been done on this topic. Take for instance Coverage criteria for GUI testing by Atif M. Memon, Mary Lou Soffa and Martha E. Pollack (September 2001, ACM SIGSOFT Software Engineering Notes , Proceedings of the 8th European software engineering conference held jointly with 9th ACM SIGSOFT international symposium on Foundations of software engineering ESEC/FSE-9, Volume 26 Issue 5). They propose an approach of dividing up the user interface into a number of components that can be tested independently. They point out that large GUI's are haeirarchicacal, made up of dialogs, panes, etc.

Once an application has been divided into components, all possible events available for a component are identified and an event flow graph is constructed that reflects all possible compositions of events. Then inter-component interactions are identified and an "integration tree" is constructed, i.e. a tree rooted at the main or starting point of the application and connecting to components that are activated via events from another component (restricted-focus, unrestricted-focus and termination events in the parlace of the paper). The paper goes on to describe metrics and testing approaches based on this application decomposition. It also points out how statement-based code coverage can be very misleading.

There are other approaches to testing GUI's, but I hope this one has given you a notion that there is more to testing an Ajax application than turning a Use Case into a test scenario, or that a statement-based code coverage tool will let you sleep well at night over the adequacy of your functional tests.

Technorati : ajax, code coverage, testing

agile ajax

Comments: 1 so far

• Leave a comment
• Comment feed for this post
• TrackBack URL

1. 

   This is interesting. At ArtOfTest, we actually are proposing to segment your application UI into segements called testregions that you inject in your source code. You can then automate against these segments. They offer isolation of these segements from other changes in your application and allow you to be more structured and focused in your testing. Check out TestRegions at http://www.artoftest.com

   Comment by John, Thursday, April 5, 2007 @ 12:18 pm

Leave a comment