We have just seen that service-oriented architecture brings many benefits, but it also brings unique testing challenges for the quality assurance (QA) team. So testers should be well prepared and adopt best practices to build an approach that will help them effectively address those challenges. Lessons learned from several large-scale SOA implementation projects have been leveraged to define key elements that can make a QA team successful with testing SOA applications.

First, from the very beginning of a new SOA project, it is very important to put together a collaborative cross-functional test team that understands the business tasks and processes to be tested; the different aspects of service-level agreements; and the underlying technologies being used. In that team, business analysts should be providing clear inputs with regard to test objectives, and they should review and approve the proposed test strategy and test plans. Test architects are the main stakeholders of the test plans being defined by the team. They should take into account the service-level agreements that describe the expected level of quality of service, performance, fault tolerance, etc. Testers, performance engineers and developers should be trained on the SOA technologies and tools, and they should be ready to effectively turn test plans into executable test cases.

This collaborative test team, once put in place, should rely on the well-known mantra "test early, test often." It is far more effective to find and fix defects close to where they were introduced than it is to find them later on, when it becomes harder to spot the cause of problems—and when fixing them has a much greater impact on the application code and design. In a service-oriented architecture, where there is less control over the entire solution being built, and where services will be reused in different applications and in many different ways, it is recommended to test each service individually first. For example, will services that are reused still perform well with an additional load? Validating the functions and the performance of services being built, as well as the ones being reused, would reduce the number of issues found during the integration phase.

The composing of services is usually done by designing the workflow using business process models, most of the time using the state machine paradigm. Sometimes, these models are turned into executable code by using the Business Process Execution Language (BPEL). In all cases, understanding those interaction models is helpful in creating integration test scenarios that involve the right services together, with the correct combination.

Of course, the test team will test the complete solution. But it should also validate that the new application meets all the business objectives for which it is being implemented. Obviously, there is no benefit for an organization to adopt a brand-new, bug-free, well-performing application that no one will actually use because it does not meet one of the business objectives. With an SOA approach, testing, like all the other development activities, should be business driven.

Certainly, continuous testing along the entire software delivery cycle is resource intensive—unless you automate the creation and execution of most of the tests. Effective test automation for SOA applications is a challenge in itself that calls for a rigorous approach. Test automation should be considered a task on its own, and not just an activity that testers perform in their spare time or after the fact, when everyone realizes the project is behind schedule. Instead, a test automation project needs to be run just like any software development project: with a detailed plan that includes requirements, objectives and resource allocation.

One of the problems that often occurs with test automation is the application user interface changes frequently during development—sometimes from one build to another. This means that automated test cases are broken and need to be updated frequently. In an SOA, if most of the tests have been created and automated based on the service layer, and therefore are independent of the user interface changes, this will offer better maintainability and will save cost and time.

Another way to optimize the test automation activity is to reuse test assets as much as possible. For example, a good practice is to create libraries of executable test steps that can be put together to build more complex test cases. A data-driven testing approach could also be applied where the same test scenario is reused several times with different sets of data.

Moreover, effective test automation calls for tools that can ease the creation and maintenance of both functional and nonfunctional tests in relation to the development platform used on a project, specifically with the change and test management tools. And, in the context of a service-oriented architecture, the test tools should address the test of GUI-less services and support the underlying open standards and technologies.

IBM Rational Performance Tester Extension for SOA Quality software is for performance engineers who need to conduct load and performance testing of services. It extends IBM Rational Tester for SOA Quality and IBM Rational Performance Tester software.

Rational Tester for SOA Quality simplifies the creation of tests for GUI-less services by automating the generation of Web service test client or by recording the interaction between a service consumer and a service provider. It comes with a graphical test editor that enables both high-level and deeper detail views. No programming knowledge is necessary to create, modify and execute functional tests. Moreover, the software leverages the business modeling work developers have already done, and it automatically generates test cases based on the behavior of business processes defined in BPEL. Rational Tester for SOA Quality can automatically detect variable data during test recording and prepare the test for data-driven testing. And it allows users to easily create customized data sets to be used by the tests during execution.

Rational Performance Tester Extension for SOA Quality provides a flexible workload modeling capability, enabling automated generation of service performance tests. This modeling helps ensure that the performance test accurately mirrors the user base, including different groups of service consumers, as well as the activities and usage patterns of each of the groups. Through the modeling exercises, this tool can provide more effective real-world scenarios.

Now you can easily pinpoint the performance bottlenecks of your SOA application. To help find the root cause of the bottlenecks, Rational Performance Tester Extension for SOA Quality allows you to import IBM Tivoli® Monitoring response time breakdown data.

Knowing which new challenges that key SOA characteristics bring to testing activities is helpful for you and your team. Some of these challenges will require adjustments to your current quality management and testing processes, and they will demand that you implement new tools specifically designed to deal with new assets under test. IBM Rational Tester for SOA Quality and IBM Rational Performance Tester Extension for SOA Quality address those needs and are effective solutions to tackle the challenging tasks of testing service-oriented applications.