This plugin generates trend reports for the Testability Explorer, an open source program which uses byte-code analysis to look for testability pitfalls in Java code.
Testability Explorer is an open-source tool that identifies hard-to-test Java code. Testability Explorer provides a repeatable objective metric of "testability." This metric becomes a key component of engineering a social change within an organization of developers. The Testability Explorer report provides actionable information to developers which can be used as measure of progress towards a goal and a guide to refactoring towards a more testable code-base. Further information can be found here.
The following features are provided by this plug-in:
- Configuration of the XML report files to scan after a build.
- Configurable threshold value at which a build will become unstable if your code's testability is beyond this number.
- Trend reports
- Overall testability cost for your project.
- Number of excellent, good and poor testable classes in your project.
- Detailed information how the testability cost is aggregated per class.
The current release is available in the download section. This plug-in is developed and maintained by Reik Schatz. Please use the Hudson mailing lists or the Testability Explorer mailing list to ask questions, create feature request or bug reports.
Use the Hudson plugin browser or Download and install the plugin. Restart Hudson, then go to your project and click on
Configure in the left menu.
Publish Testability Explorer Report. Enter the correct path to your Testability Explorer report files. The path will be relative to your project root. The project root is usually where your build.xml (Ant) or pom.xml (Maven) file is located. Let's say as part of your build, you create the Testability Explorer report in
reports/testability/result.xml, then the Report File Pattern for Hudson would have to be
reports/testability/*.xml if you want to use a Wildcard.
The Overall Threshold is a number value representing an overall testability cost. By setting this threshold you define that your project code base, must not have a higher overall testability cost than this number. If the testability cost is higher, the build will become unstable. As a rule of thumb, a good testability is below 100 but aim for a score below 50. The Hudson core project has more than 500 classes and a great testability score of just 22!
The Per-Class Threshold is a number value representing a testability cost on a per class basis. By setting this threshold you define that not a single class in your project may have a higher testability cost than this value. If the testability cost of any class is higher, the build will become unstable.
For each individual build you can review the testability scores. Click on a single build in the left column then on the Testability Explorer Report icon in the screen center. This will bring up the report. You can see a summary, which will show you the number of classes in your project and how many of them a excellent, good and poor testable. The total is the overall testability score for your project.
In the lower section you will find the 20 least testable classes in your project. You can click on a class to see an aggregation of what is causing the cost on a method level. Then for each method, you can go into detail and see how the cost is computed.
In the main project page, you have a testability trend graph to the right side. This is how your overall testability is evolving during the different builds. The Testability Explorer Report icon in the center and the left menu, will bring you to a testability trend graph on the class level. You can see how many excellent, good or poor testable classes you have and how this number is changing over time.
- First version