Performance Regression Testing Using Japex and WSTest, and Future Tips

The initializeDriver() method in the TestServiceBaseDriver.java class creates a service port and initializes it with an endpoint address.

The prepare() method populates variables, objects, and data structures and readies them for next phase.

The run() method makes web service calls to methods on the server using the service port created during the initialization phase.

For more details about writing a driver class, see the Japex manual.

Write Configuration Files

Japex configuration file defines test suites. A test suite can have one or more drivers and/or test cases. Each driver can have one or more test cases. The configuration file can also have zero or more global parameters. The global parameters appear before drivers and the drivers appear before test cases in the configuration file.

You can use parameters in the configuration file to configure the performance tests. The parameters can be of three different scopes: global, driver, or test case. The global parameters apply to all drivers and test cases. The driver parameters apply to a specific driver and test cases. The test case parameters apply to specific test case.

The Japex configuration file for the sample application is TestServiceXMLDriver-for-build-comparison.xml. The file is located in the configs directory. The test suite is named WSpex. The file defines a number of global parameters.

The parameters that begin with the keyword japex are reserved for Japex. The japex.numberOfThreads parameter defines the number of simulated users. The parameters japex.warmupTime and japex.runTime are test case parameters. They set the warm up and run duration for all test cases. The parameters japex.warmupsPderDriver and japex.runsPerDriver define the number of warm ups and runs that should be executed for each driver.

The last parameter, japex.reportsDirectory, specifies the location of the report files.

A few more configuration files are created in the configs\entities directory. The files TestServiceXMLDriver-for-build-one.xml and TestServiceXMLDriver-for-build-two.xml define drivers for two different builds of the application server.

These files define which driver class to use. They also specify the port number and endpoint URL. In the sample, the driver class is TestServiceXMLDriver. The two different builds of the application server should have their instances listening on different port numbers.

The configuration file, classpath-as.xml, defines the class path that Japex can use to configure and run the tests. The file all-tests.xml defines all the test cases and any applicable input parameters. These files are also in the configs\entities directory.

For more details about writing a configuration file, see the Japex manual.

Generate Portable Artifacts and Compile the Client

The way to generate portable artifacts and compile all the Java classes for the client is similar to that for server. The ant build task does this automatically.

Running the Sample

After running the script to set up your environment, you can run the sample as follows:

1. Start both GlassFish instances by entering the following command for each instance:
   
     %AS_HOME%/bin/asadmin start-domain domain1
   
   
2. Build the web service, the client, and needed artifacts, and package them into a WAR file by entering the following command in the techtip\Sample directory:
   
     ant build
   
   
3. Deploy the application by entering the following command:
   
     ant deploy
   
   
4. Run the application by entering the following command:
   
     ant run
   
   
5. You should see output that looks something like this:
   
      run:
        [java] Reading configuration file 
   'configs/TestServiceXMLDriver-for-build-comparison.xml' ... 
        [java] Estimated warmup time + run time is 1 minutes 
        [java]   TestServiceXMLDriver-for-build-one using 1 
   thread(s) on 1 cpu(s) 
        [java]     Run 1: echoVoid,177.661,echoInteger,148.
   959,echoFloat,140.094,echoString,162.918,echoDate,84.997,
   echoStruct,166.121,echoSynthetic4K,72.235,echoSynthetic8K,
   122.979,echoSynthetic12K,86.514,echoArray40,28.863,echoArr
   ay80,18.589 
   ...
   
       [java] Generating reports ...
   
   

   The application generates reports in the reports directory. You can explore specific reports by viewing the index.html page in the directory for the report of interest. Each report is placed in a separate directory below the reports directory. Each specific report directory has a name that represents the time stamp of when the report was created.

   Each report shows a summary of the run as well as details of each test case, including graphs comparing the results.

   For example, here is part of a graph that compares transactions-per-second for a test run against two builds.

   Result Summary

    
   

   You can learn more about the reports on the Japex site.

6. After running the application, you can undeploy it by entering the following command:
   
     ant undeploy
   
   
7. Delete all classes and the WAR file generated for the application by entering the following command:
   
     ant clean
   
   

About the Authors

Singh and Bharath Mundlapudi are staff members of the Java Performance Engineering group at Sun Microsystems.

Back to Top 

Over the years, the Enterprise Tech Tips have covered a wide variety of Java Technology topics. Here's your opportunity to tell us what topics you would like to see covered in future issues of the Tech Tips. Send your topic requests to Enterprise_TechTips@sun.com. Your input will help us make the Tech Tips better serve your needs.

Back to Top