{jenkins-plugin-info:pluginId=clif-performance-testing} |
Enables running performance tests defined with CLIF from Jenkins, and getting a variety of automatically generated performance reports and charts. Useful for: automated performance testing, performance testing in continuous integration, providing a simple web user interface for CLIF, monitoring QoS or applications QoE and possibly send alerts in case of bad responsiveness. |
CLIF is an open (source), extendible performance testing software supporting a variety of load injection protocols: TCP, UDP, HTTP(S), FTP, DNS, LDAP(S), SIP, RTP, GIT, SVN, JDBC, JMS...
CLIF also embeds a probe framework to monitor arbitrary resources available on the network (via SNMP, JMX...) or on computing nodes (CPU, RAM, disks, network...)
CLIF comes with a variety of integrations and user interfaces: command-line, simple standalone GUI, full-fledged GUI integrated to Eclipse, Maven plug-in and... Jenkins plug-in.
Thanks to this plug-in, you can import CLIF test projects (bundled scenarios, test plans and test data sets), typically written using CLIF's main GUI, as Jenkins jobs. Then, a detailed performance report is generated for each test run, including a global statistical analysis as well as graphs (scatter chart, moving statistics, histograms and quantile distribution analysis) about load injectors' responses times and throughput. The CLIF performance report also provides statistics and charts about probes measurements.
A performance trend chart is also maintained through test executions.
Prior to running or importing any CLIF test plans, you need to install at least one CLIF runtime, typically a 2.3.3+ "CLIF server" distribution, available for download from here.
Once you have unzipped this CLIF distribution, make sure you have a Java 8+ execution environment installed.
To check your CLIF installation, run this command line:
Example on Windows 7 (cmd.exe) |
|
---|---|
|
Example on Linux |
|
---|---|
|
In case Java can't be found, or the default Java environment is not appropriate, change your PATH environment variable so that the bin directory of the right Java runtime is found first.
Once the CLIF Performance Testing plug-in is installed in Jenkins, you can add the installed CLIF runtime to the Jenkins tools configuration page (Manage Jenkins>Global Tool Configuration).
Go to the Clif section, click on button Clif installations... and fill the fields:
Click on Save or Apply buttons at the page bottom.
Note: you may declare as many CLIF runtimes as you need, including several instances of the same runtime (same CLIF home directory) but with different names and Java properties.
The CLIF Performance Testing plug-in provides two ways of creating a CLIF test job:
As mentioned at the top of this page, CLIF's main GUI is based on Eclipse. CLIF scenarios, test plans and possible data sets are grouped together in a CLIF project. It is straightforward to use a source code management (Svn, CVS, Git...) client provided by Eclipse to push this project to an SCM repository.
Once your CLIF project is available on an SCM repository, create a new Freestyle project (New Item... > Freestyle Project). Go to the Source Code Management section and provide all necessary details about your project repository.
Then, go to the Build section, and add an Invoke Clif build step:
Finally, you may be interested in adding a post-build action to automatically generate on each CLIF job successful completion. Refer to section below dedicated to CLIF reports configuration.
Another convenient way to proceed is to export the CLIF project as a zip archive from CLIF's Eclipse-based main GUI, and then to import it from Jenkins using wizard Import a Clif zip.
Once you have selected your CLIF project's zip file (beware of automatic refresh that tends to discard your file selection), the wizard makes you choose the CLIF runtime for the new job(s), as well as the test plans to import. One job will be created per imported test plan. However, note that all these jobs will share a common workspace, that will become visible as soon as you run the created jobs.
Click on Validate button.
Note: the import wizard also adds a Publish Clif performance report post-build action. Refer to next section to configure CLIF reports.
With the Advanced settings of Publish Clif performance report post-build action, you may set:
Beyond the detailed CLIF performance report generated for each CLIF job successful run, a Clif performance Trend report is also maintained to see the response time and error trends for each request type.