Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 12 Next »

This plugin integrates Micro Focus products with Jenkins. Using the plugin, you can create and use virtual services, run Performance Center or LoadRunner performance tests, run UFT functional tests, run tests in your lab, and run tests on mobile devices. This plugin also lets you upload test results to ALM. In addition, ALM Octane users can track and trigger Jenkins pipelines from within the user interface. 

Jira

Submit issues and feedback through JIRA.

This plugin is open source. If you identify any issues or want to submit enhancement requests, please use JIRA. The component for this plugin is the hp-application-automation-tools-plugin component. Your feedback and involvement will allow us to stabilize and enhance the capabilities of the plugin. The latest, early access, beta versions of this plugin are available here

 

 

Content Security Policy

Starting with version 1.641 (or 1.625.3), Jenkins introduced the Content-Security-Policy header. This causes some of the integration links, such as links to reports, to become inoperable. For details, see Configuring Content Security Policy and Jenkins Security Advisory . For suggested workarounds until the issue is resolved, see Content Security Policy Header.

What's New in version 5.4

This version provides the following enhancements:

ALM Octane

  • Pipeline Multibranch support (requires ALM Octane 12.55.31 or later, Jenkins 2.7 or later, Git 3.8.0 or later, and Pipeline Multibranch Plugin 2.16 or later)
  • Discover and run UFT tests stored in a Subversion (SVN) repository (ALM Octane 12.55.25 or later)
  • Auto-categorize tests as StormRunner Functional tests (ALM Octane 12.55.17 or later)
  • Track code coverage reports from Jenkins. This requires a post-build step, as described in the ALM Octane Help Center. (ALM Octane 12.55.13 or later)
  • Suspend/resume CI server connection. (ALM Octane 12.55.13 or later)

StormRunner Functional (SRF)

  • Ability to test the connection from within the plugin
  • Bug and stabilization fixes

Service Virtualization (SV)

  • Improved structure of EXPORT command output to keep project structure
  • EXPORT command is able to write output into SV project archive (.vproja)
  • Several bug fixes and minor improvements

Mobile Center (MC)

  • Ability to select a specific workspace for a test when working with a server with multiple workspaces.
  • Bug fix: App is uploaded even if instrumentation failed during the app upload.

Supported Integrations

This plugin supports the following Micro Focus product versions:

  • ALM Lab Management 12.xx
  • ALM Octane 12.53.20 or later (12.55.4 or later required for direct UFT integration and for PC integration using pipelines)
  • Application Lifecycle Management 12.xx
  • LoadRunner 12.xx
  • Mobile Center 2.0-2.70
  • Performance Center 12.xx (12.53 or higher required for trend reports)
  • Service Virtualization 3.80 - 4.20
  • StormRunner Functional 1.40
  • StormRunner Load 2.8
  • UFT Pro (LeanFT) 14.03
  • Unified Functional Testing (UFT) 12.xx and 14.03

 

ALM Octane integration
Connecting your Jenkins server to ALM Octane enables you to track and analyze builds, automated test results, and SCM data as part of your product and release quality analysis. For more details, see the ALM Octane Help Center.

If you have these Jenkins plugins, ALM Octane supports them:

  • MultiJob Plugin 1.15 or later
  • Maven Plugin 2.5 or later
  • Parameterized Trigger Plugin 2.25 or later
  • Matrix Job Plugin 1.4 or later
  • Pipeline Plugin (enabling Pipeline as Code). ALM Octane creates a single pipeline step for each Jenkins job. Steps and stages inside a job are not recognized.

Notes:

  • If you are working with Quality Center 10.00 or earlier, and QuickTest Professional 9.x or 10.x, use the Quality Center Plugin.
  • If you are working with ALM Octane, make sure that you work with version 5.1 or later of this plugin, and that you do not have the HPE ALM Octane CI plugin installed. For more details, see Upgrade from the HPE ALM Octane CI plugin to this plugin.
  • For ALM Octane-UFT integration, make sure that you work with version 5.2 or later of this plugin and with UFT 12.50 or later. 
  • For ALM Octane to collect Performance Center test results from pipeline runs, make sure that you work with version 5.2 or later of this plugin.
  • ALM Octane can display a pipeline's structure only if it is built using one of the following:
    • The built-in Jenkins upstream/downstream mechanism
    • The Multijob plugin
    • The Parameterized plugin

Prerequisites

  1. Java version 6 or higher. To verify your Java version, go to http://www.java.com/en/download/installed.jsp.
  2. ALM/Quality Center client installed the machine that will run the tests. To check if you have a client installed, follow the instructions on this page:   http://<your_server>:8080/qcbin/TDConnectivity_index.html
  3. For running UFT tests from ALM install the ALMClient in common registration mode by accessing the following link from an Internet Explorer browser on the UFT machine: http://<almserver>:8080/qcbin/start_a.jsp?Common=true
  4. For running file system scenarios with LoadRunner or UFT you need to set up a node in Jenkins. For details, see Create an execution node.
  5. For building and tracking pipelines on ALM Octane:
    • JQuery Plugin 1.7.2-1 or later (Required to enable the integration)
    • A plugin that enables publishing test results. For example, JUnit Plugin 1.10 or later, NUnit plugin, and so on (Required to enable ALM Octane to collect your automated test results.)
  6. For ALM Octane integration with UFT:
    • Jenkins Git plugin (version 2.4.4 or later)

 


User Guide

You can run client side-or server-side (Default or Functional) test sets and build verification suites from Application Lifecycle Management (ALM) or functional tests from the file system. You can create and configure ALM Octane pipelines from the ALM Octane user interface, or on Jenkins.

Create an execution node

  1. Creating an execution node is only required when running scenarios from LoadRunner or UFT, that are stored on the file system. You only need to set an execution node if the Jenkins master machine is not the executing machine. If you are running LoadRunner/UFT on the master machine, you do not need to set and select a node.
  2. Go to the Jenkins Server home page.
  3. Click the Manage Jenkins link in the left pane. 
  4. In the Manage Jenkins Page click Manage Nodes.
  5. Click New Node.
  6. Enter a Node name, select Permanent Agent, and then click OK.
  7. In the next screen, enter information in the required fields. Enter the full path of the Remote Root directory, to which the tests results will be saved.
  8. Provide one or more labels for the node. Separate the values with a space. Through these labels, you will be able to identify the nodes used in the job.
  9. In the Usage area, select Only build jobs with label expressions matching this node.
  10. Click Save.
  11. Connect the execution node to the Jenkins server as described below.

Connect an execution node to the Jenkins server

  1. On the computer that you defined as an execution node, open a browser and go to the Jenkins Server home page.
  2. Open Manage Jenkins > Manage Nodes.
  3. If there is a warning mark adjacent to the node you want to connect, click the node's link. You are prompted to connect the agent to Jenkins.
  4. Click Launch to download the agent and then run the downloaded agent. Alternatively, use one of the other methods suggested on the screen.

Configuring the connection to your ALM Server

  1. Go to the Jenkins Server home page.
  2. Click the Manage Jenkins link in the left pane. 
  3. In the Manage Jenkins Page click Configure System.
  4. In the Configuration tab, scroll down to the Application Lifecycle Management section.
  5. Click Add ALM server, if no server is currently configured.
  6. Specify a meaningful server name and URL. When specifying an ALM Server URL, use the full string: http:/myserver.mydomain/qcbin.

Upgrade from the HPE ALM Octane CI plugin to this plugin

If you already set up your CI server to work with ALM Octane using the HPE ALM Octane CI plugin, perform the following steps to start using this plugin instead:

  1. Go to the Jenkins Server home page.
  2. Click the Manage Plugins link in the left pane. 
  3. Select the Installed tab and uninstall the HPE ALM Octane CI plugin.
  4. Restart Jenkins.
  5. Install the Jenkins plugin, version 5.1 or later.
  6. Restart Jenkins.
  7. Configure the Connection to your ALM Octane Server.
  8. To enable the plugin to work with existing ALM Octane pipelines:
    In the ALM Octane CI configuration section, click Show plugin instance ID. In the Instance ID box, enter the Instance ID used by your pipelines in ALM Octane (ALM Octane: Settings > DevOps > CI serversInstance ID column.

Configuring the connection to your ALM Octane Server

You must configure this connection if you want to do the following:

  • Create ALM Octane pipelines to track your Jenkins server activity in ALM Octane
  • Configure a testing tool connection in ALM Octane to set up ALM Octane-UFT integration.

Before you configure the connection

  1. Ask the ALM Octane shared space admin for an API access Client ID and Client secret. The plugin uses these for authentication when
    communicating with ALM Octane. The access keys must be assigned the CI/CD Integration role in all relevant workspaces. For details, see Set up API access for integration.
  2. If your network requires a proxy to connect to the Internet, set up proxy configuration.
    1. Go to Manage Jenkins > Manage Plugins > Advanced and define the necessary HTTP Proxy Configuration details:
      • If the ALM Octane server is located outside of the Jenkins server's private network, define the proxy server's details to enable Jenkins to communicate with ALM Octane.
      • If the ALM Octane server and the Jenkins server are both in the same private network (common if you are using ALM Octane on premises), instruct Jenkins not to use the proxy to reach ALM Octane: Add ALM Octane's URL to the No Proxy Host list (using ALM Octane's Fully Qualified Domain Name (FDQN)).
        Note: In this case, you may still need the proxy server details defined to enable Jenkins to access other external resources, such as Git.
    2. Test that Jenkins can now reach ALM Octane:
      Click the Advanced button beneath the HTTP Proxy Configuration details.
      Enter ALM Octane's URL (using its FQDN) and click Test URL.
    3. Restart the Jenkins server so the plugin uses the new proxy settings.
  3. Decide which Jenkins user ALM Octane will use to execute jobs on the server.
    Caution: We strongly recommend setting this user’s permissions to the minimum required for this integration.
    The Jenkins user must have the following permissions:
    • For working with ALM Octane pipelines: Job Build.
    • For ALM Octane-UFT integration: Job Create, Delete, and Read.
      If your Git repository requires credentials, Credentials create and update permissions are also required.

Configure the connection

  1. Go to the Jenkins Server home page.
  2. Click the Manage Jenkins link in the left pane.
  3. Click Configure System.
  4. Scroll down to the ALM Octane CI section.
  5. Follow the instructions on the screen to complete the configuration:
    • For Location, enter the URL of the ALM Octane server, using its fully
      qualified domain name (FQDN). Use the following format (port number is optional):
      http://<ALM Octane hostname / IP address> {:<port number>}/ui/?p=<shared space ID> 
      For example, in this URL, the shared space ID is 1002: http://myServer.myCompany.com:8081/ui/?p=1002 
      Tip: You can copy the URL from the address bar of the browser in which you opened ALM Octane.
    • When entering the Client ID and Client secret, make sure to enter the exact text without leading or trailing spaces.
    • For Jenkins user, enter the Jenkins user ALM Octane will use to execute jobs on the server.
      Make sure this user exists in Jenkins. 
      If you do not specify a Jenkins user, ALM Octane uses Anonymous, and is limited to Anonymous’s permissions.
  6. Click Test Connection to validate the configuration.
  7. Click Submit to save your changes.

After the connection is set up you can do any of the following

Create and configure ALM Octane pipelines on Jenkins

Before you can create an ALM Octane pipeline, you must Configure the Connection to your ALM Octane Server.
To create an ALM Octane pipeline in Jenkins:

  1. Go to the Jenkins Server home page.
  2. Click on the job that you want to use as the root job for your pipeline.
    Note: Select a job that is not already part on an existing ALM Octane pipeline.
  3. In the menu in the left pane, click ALM Octane Pipelines.
  4. Click Create Pipeline.
  5. Enter the details for the pipeline:
    1. Provide a name for the pipeline. The Workspace list displays all of the workspaces on the ALM Octane server you configured in the ALM Octane CI section.
    2. Select a workspace for your pipeline.
    3. The Release list displays the releases available in the workspace you selected.
      Optionally, select a release to associate with your pipeline.
  6. Click Create.
    Caution: If you delete a pipeline, all its labels and configuration information are lost.


To edit the pipeline and configure pipeline steps:

  1. Go to the Jenkins Server home page.
  2. Click on a job that is included in an existing ALM Octane pipeline.
  3. In the menu in the left pane, click ALM Octane Pipelines. The page for editing pipelines opens.
  4. You can edit the pipeline’s name, workspace, or release.
  5. For a job that runs tests, configure test and environment details. For details, see Create and configure pipelines.
    Note:
    • A Jenkins job may be included in a pipeline more than once. When you configure a Jenkins job, this affects all of the relevant steps in the ALM Octane pipeline.
    • To dynamically set environment tags during the build run, based on build parameter values, configure the pipeline in ALM Octane.
  6. Click Apply.


For details on tracking your pipelines in ALM Octane, see Run and analyze pipelines.

 


Running Test Sets from ALM

 Set up a job

  1. Go to the Jenkins Server home page.
  2. Click the New Job link or select an existing job.
  3. Enter a Job name (for a new job).
  4. Select Build a free-style software project and click OK.
  5. In the Project Configuration section scroll down to the Build section.
  6. Expand the Add build step drop-down and select Execute functional tests from ALM.
  7. Select one of the ALM servers that you configured in the previous step.
  8. Enter the server credentials, project and domain. Note: If you are using the ALM scheduler, it will run under the Jenkins agent user. For example, if Jenkins is running as a System user, the scheduler will run the tests as a System user. This will not affect test execution. 
  9. Add the test set folders or specific test sets that you want to include, using the ALM path. To add a specific test, add the test name after the test set path.To add multiple entries, click the down arrow on the right of the field and enter each item on a separate line. See the example below.

  10. Optionally, indicate a timeout in seconds after which the job will fail.
  11. Click Advanced to indicate a Run mode (local, remote, or planned host) If you specify a remote host mode, specify a host name. This must be a machine with a valid installation of the testing tool.
  12. Click Apply to save your changes and continue with more build steps. Click Save when you are finished adding build steps.

     

    Root\testfolder1\testset_a
    Root\testfolder1\testset_b
    Root\testlab_folder
    Root\testlab_folder\testset_a\test-name

Set up the Post Build actions

  1. In the Post-build Actions section, expand the Add post-build action drop-down and select Publish HPE test result.
    Run the job
  2. Run or schedule the job as you would with any standard Jenkins job. 

Review the results

  1. From the dashboard, click on the job.
  2. Click the Console link to view the ALM information.
  3. Copy the ALM link to your Internet Explorer browser and view the Test Set results from within ALM.

 


Running Server Side Tests Using HPE ALM Lab Management

If you have Lab Management activated in ALM, you can run server-side tests from functional test sets and build verification suites. After setting up the test sets and build verification suites, you can configure a Jenkins build step to execute your tests.

Set up a job

  1. Go to the Jenkins Server home page.
  2. Click the New Job link or select an existing job.
  3. Enter a Job name (for a new job).
  4. Select Build a free-style software project and click OK.
  5. In the Project Configuration section scroll down to the Build section.
  6. Expand the Add build step drop-down and select Execute HPE tests using HPE ALM Lab Management.
  7. Select one of the ALM servers that you configured in the previous step.
  8. Enter the server credentials, project and domain. 
  9. Optional: If you're using proxy, you can configure the proxy settings.
  10. Select a Run Type from the drop down menu (functional test set or build verification suite).
  11. Enter the ID of your run entity (either the test set ID or the build verification suite ID).
  12. Optional: Enter a description of the build step.
  13. Enter a duration (in minutes) for the timeslot. The minimum time is 30 minutes.
  14. Optional: If you have defined an AUT environment configuration in ALM, you can enter the ID here in order to execute your timeslot with specific AUT parameters.
    If you have CDA configured in ALM and want to implement it for this time slot, select the Use CDA for provisioning and deployment checkbox and enter your CDA details.
    Set up the Post Build actions
  15. In the Post-build Actions section, expand the Add post-build action drop-down and select Publish HPE test result.

Run the job

Run or schedule the job as you would with any standard Jenkins job. 

Review the results

  1. From the dashboard, click on the job.
  2. Click the Console link to view the ALM information.
  3. Copy the ALM link to your Internet Explorer browser and view the Test Set results from within ALM.

Note: For environments with Internet Explorer 9 and ALM version 12.20 or later: The report, normally accessible through a link in the console log, will not be available.


Creating/Updating the AUT Environment Configuration Using HPE ALM Lab Management

If you have Lab Management activated in ALM, you can create/update an AUT Environment Configuration for an existing AUT Environment in ALM.
 Set up a job

  1. Go to the Jenkins Server home page.
  2. Click the New Job link or select an existing job.
  3. Enter a Job name (for a new job).
  4. Select Build a free-style software project and click OK.
  5. In the Project Configuration section scroll down to the Build section.
  6. Expand the Add build step drop-down and select Execute AUT Environment preparation using HPE ALM Lab Management.
  7. Select one of the ALM servers that you configured in the Configure the connection to your ALM server step.
  8. Enter the server credentials, project name, and domain. 
  9. Enter the ID of the environment for which you want to create/update a configuration.
  10. Select one of the following options to Indicate whether or not to create a new AUT Environment Configuration or update an existing one.
    1. For Create a new configuration named, enter a name for the new configuration.
    2. For Use an existing configuration with ID, enter the ID of your AUT Environment Configuration in ALM.
  11. Optional: Enter a path for a JSON file that contains values for the AUT Environment parameters for the relevant configuration.
  12. Optional: Enter a name of a build environment parameter in order to save the ID of the created/updated configuration for future use.
  13. Add the AUT Environment parameters that you want to update for the created/updated configuration. For each parameter:
    1. Select the type of the parameter from the drop-down menu (Manual, Environment, From JSON).
    2. Enter the full path of the parameter as it appears in ALM.
    3. Enter the value you want to assign to this parameter.

Run the job

Run or schedule the job as you would with any standard Jenkins job. 

 


Uploading Test Results to ALM

You can upload tests running in JUnit, NUnit, or TestNG frameworks to ALM as part as the run job or as a separate job. ALM will create tests and test sets (of External type) corresponding to the executed tests.

 Set up a job

  1. Go to the Jenkins Server home page.
  2. Click the New Job link or select an existing job.
  3. Enter a Job name (for a new job).
  4. Select Build a free-style software project and click OK.

Set up the Post Build actions

  1. In the Project Configuration section, scroll down to the Post-build Actions section.
  2. Expand the Add post-build action drop-down and select Upload test result to ALM.
  3. Select one of the ALM servers that you configured in the Configure the connection to your ALM server step.
  4. Enter the server credentials, project name, and domain. 
  5. Select a testing framework from the drop-down list: JUnit, NUnit, or TestNG.
  6. Optional: Enter the name of the Testing Tool name, to be used in the corresponding entity in ALM.
  7. Enter the ALM Test folder path, in which to store the external tests. Do not include the Root test folder (Subject).
  8. Enter the ALM Test Set folder path, in which to store the external test sets. Do not include the Root test sets folder.
  9. Enter the Testing result file condition, relative to the root path of the job. For example, use **/junitResult.xml for Junit Plugin results.
  10. Optional: Enter the Jenkins server URL.

Run the job

  1. Run or schedule the job as you would with any standard Jenkins job. 

Review the results

  1. From the dashboard, click on the job.
  2. Click the Console link to view the ALM information.
  3. Copy the ALM link to Internet Explorer and view the tests, test sets, and test runs that were created within ALM.

 


Running Functional Tests

For details about functional testing using Jenkins, see the following ADM Help Center topics:


Uploading Apps to Mobile Center

This add-in provides a standalone builder for uploading apps to Mobile Center.

For details, see the Jenkins Integration section in the Mobile Center Help.

 


Running LoadRunner Scenarios from the File System

For additional information, see the online help or blog post about continuous integration with LoadRunner.

 Set up a job

  1. Go to the Jenkins Server home page.
  2. Click the New Job link or select an existing job.
  3. Enter a Job name (for a new job).
  4. Select Build a free-style software project and click OK.
  5. In the Project Configuration section, select Restrict where this project can be run, and select the appropriate node. For details, see Create an execution node.
  6. Scroll down to the Build section.
  7. Expand the Add build step drop-down and select Execute HPE tests from file system.
  8. Click the LoadRunner-Specific Settings button and specify the following settings:
    1. Controller Polling Interval. The interval in seconds by which to poll the Controller for the scenario status. The default is 30 seconds.
    2. Scenario Execution Timeout. The total time in seconds to allot for the execution of all listed scenarios. After this time, the Controller process will be terminated.
    3. Errors to Ignore. You can specify which errors to ignore during the run. For example: Error: CPU usage for this load generator has exceeded 80%. Enter each error string on a separate line.
  9. In the Tests box, enter a test with its full absolute path, or a folder or MTB containing one or more tests or LoadRunner scenarios. To specify multiple entries, click the down arrow on the right of the field and enter each test path on a separate line. Make sure that the paths are accessible from all machines in the local network.
  10. Indicate a timeout in seconds after which the job will fail in the Timeout box (optional). 
  11. Click Apply to save your changes and continue with more build steps. Click Save when you are finished adding build steps.

Set up the Post Build Actions

  1. In the Post-build Actions section, expand the Add post-build action drop-down and select Publish HPE test result.
  2. Select an archiving option for the test run results:
    • Archive HPE test reports for failed tests: Only save test result reports for failed tests.
    • Always archive HPE test reports: Always save test result reports.
    • Always archive and publish HPE test reports: Always save and publish test result reports.
    • Do not archive HPE test reports: Never save the test results.


Run the job
Run or schedule the job as you would with any standard Jenkins job.


Review the results in the Run Results Viewer  

  1. Extract the files from the archive file. When you when you extract the zip file contents, it creates the following folders and files:
    • LRR folder- The Controller raw result folder, viewable in LoadRunner Analysis.
    • LRA folder- The Analysis session folder, viewable in LoadRunner Analysis.
    • HTML folder- HTML reports in two subfolders, IE and Netscape, in their respective formats.
    • SLA.xml - An XML report showing the status of the SLA in the test.
  2. Open the Run Results Viewer and select File > Open.
  3. In the Open dialog box, select` Results XML file and navigate to the Results.xml file in the folder that you extracted.
  4. Click Open and view the results.

Note: The pass and fail criteria for LoadRunner scenario tests are based on the LoadRunner Service Level Agreement (SLA). If you do not define an SLA, the scenario will fail. For details, see the LoadRunner Help Center.


Review the results in Jenkins  

You can also view the results from the Jenkins interface, in the following modes:

  • Per job/project
  • Per build/run

Per Job / Project (cross build / run results)

Click Project performance report in the left side Job menu. This lets you review the various results from the job, per scenario. You compare the job results using an SLA.

Per build / run 

  1. Click Performance report to examine the performance summary report with its different results.
  2. Click Transaction summary to open a detailed view of the transactions statistics taken directly from LoadRunner Analysis report.
  3. Click Simple Junit style SLA result
    1. Click on “Test result”
    2. Click on “All tests” to see the overview.
    3. Select a scenario by clicking on its name.
    4. Click on History in the left panel to compare the results with those from previous runs.

Note: These reports might require you to perform a Security header override. For details, see Content Security Policy Header.


Running Performance Tests using HPE Performance Center

Set up a job

  1. Go to the Jenkins Server home page.
  2. Click the New Job link or select an existing job.
  3. Enter a Job name (for a new job).
  4. Select Build a free-style software project and click OK.
  5. In the Project Configuration section scroll down to the Build section.
  6. Expand the Add build step drop-down and select Execute HPE tests using HPE Performance Center.
  7. Optional: Enter a description of the build step.
  8. Enter the hostname or IP address of a PC server.
    Example: If the Performance Center server URL is http://MY_SERVER/loadtest, enter MY_SERVER.
  9. If you are running the Jenkins job over a secured Performance Center server, select the Use HTTPS protocol option.
  10. Enter the server credentials, project and domain.
  11. Enter the Test ID. You can get the ID from My Performance Center > Test Management > Test Lab > Performance Test Set view. If the column is not visible, you can select it by clicking the Select Columns button .
  12. Select an option for adding the Test Instance ID:
    • Automatically select existing or create new if none exists (Performance Center 12.55 or later). If you select this option, Performance Center creates a test instance or locates the existing test instance.
    • Manual selection. Enter the Test Instance ID (available from My Performance Center > Test Management > Test Lab > Performance Test Set view).
  13. Choose whether to use a Local Proxy.
  14. Choose a Post Run Action: Collate Results/Collate and Analyze/Do Not Collate.
  15. Select a trend report option:
    • Do Not Trend. No trend report is created.
    • Use trend report associated with the test (Performance Center 12.55 or later). If Auto Trending is selected in the Load Test, select this option to automatically publish trend results.
    • Add run to trend report with ID. If you select this option, enter the trend report ID (supported for version 12.53 or later).
  16. Enter a duration for the Ad-Hoc timeslot. The minimum time is 30 minutes.
  17. Choose whether to use VUDs licenses.
  18. Choose whether to consider the SLA status for determining the build-step status. If you do not enable the Set step status according to SLA option, the build-step will be labeled as Passed as long as no failures occurred.

Set up the Post Build actions

  1. In the Post-build Actions section, expand the Add post-build action drop-down and select Publish HPE test result.
  2. Select a report archive mode.

Run the job

  1. Run or schedule the job as you would with any standard Jenkins job.

Review the results

  1. From the dashboard, click on the job.
  2. Click the Console link to view the PC run information.
  3. When the Collate and Analyze option is set as the post run action, the Performance Center HTML report can be viewed and downloaded directly from Jenkins via the Build Artifacts or Test Result options.

 

Configuring Trending Report Charts on Jenkins

  1. Install the plot plugin from https://wiki.jenkins.io/display/JENKINS/Plot+Plugin.
  2. Open your job configuration, and add a new post build action: Plot build data.
  3. Click Add Plot.
  4. Configure the following settings and leave the other settings blank or unselected (see the Data Series Files table below for setting details):
    • Plot group. Enter a meaningful name for the group. For example, Performance Trending.
    • Plot title. Enter a name that is related to the measurement. For example, Average Transaction Response Time.
    • Number of builds to include. Enter the number of builds to include. We recommend no more than 10 builds.
    • Plot y-axis label. Enter the appropriate measurement unit. For example, Seconds. See the recommended unit for each measurement in the table below.
    • Plot style. Select the plot style. Line or Line 3D are the most suitable for trending.
    • Data series file. Enter one of the measurement types listed in the table below.
      (Tip: We recommend using the bolded measurements for trending because they compare like-for-like measurements to provide an indication of the application’s performance and workload.)
    • Load data from csv file. Select this check box.
    • Make sure Include all columns is selected.
    • Display original csv above plot. Select this check box.
  5. Repeat steps 1-4 to add more plots.
  6. Save the configuration.

    Plot configuration example:
     
    Data Series Files

Type

Data Series File

Comment

Unit

TRT (Transaction Response Time)

pct_minimum_trt.csv

Minimum transaction response time

Seconds

 

pct_maximum_trt.csv

Maximum transaction response time

Seconds

 

pct_average_trt.csv

Average transaction response time

Seconds

 

pct_median_trt.csv

Median transaction response time

Seconds

 

pct_percentile_90_trt.csv

90th percentile of transaction response time

Seconds

 

pct_stddeviation_trt.csv

Standard deviation transaction response time

Seconds

 

pct_count1_trt.csv

Number of occurrences of the transaction

Count

TPS (Transaction per seconds)

pct_minimum_tps.csv

Minimum transaction per second

Count

 

pct_maximum_tps.csv

Maximum transactions per second

Count

 

pct_average_tps.csv

Average transactions per second

Count

 

pct_median_tps.csv

Median transactions per second

Count

 

pct_sum1_tps.csv

Total amount of transaction per second for a given transaction

Count

TRS (Transaction Summary)

pct_count1_trs.csv

Total amount of occurrences for a given transaction

Count

UDP (User defined data point)

pct_minimum_udp.csv

Minimum value for a user defined data point

Unit

 

pct_maximum_udp.csv

Maximum value for a user defined data point

Unit

 

pct_average_udp.csv

Average value for a user defined data point

Unit

 

pct_median_udp.csv

Median value for a user defined data point

Unit

 

pct_stddeviation_udp.csv

Standard deviation value for user defined data point

Unit

 

pct_count1_udp.csv

Number of occurrences for a given user defined data point

Count

 

pct_sum1_udp.csv

Sum of the values reported in a given user defined data point

Unit

VU (Running Vusers)

pct_maximum_vu.csv

Maximum number of running Vusers in the scenario

Count

 

pct_average_vu.csv

Average number of running Vusers in the scenario

Count

WEB

pct_minimum_web.csv

Minimum value of web statistics (# of connections, throughput, hits per second, etc.)

Unit

 

pct_maximum_web.csv

Maximum value of web statistics

Unit

 

pct_average_web.csv

Average value of web statistics

Unit

 

pct_median_web.csv

Median value of web statistics

Unit

 

pct_sum1_web.csv

Total value of web statistics

Unit

Note: If you get a file does not exist error (“<file_name.csv> doesn’t match anything”), you can ignore this because the file will be created during the job execution.
  
 


Running tests with Service Virtualization

Before you begin setting up a job in Jenkins, make sure you prepare your virtual services and other prerequisites as described in the Service Virtualization Help Center.

Configure the connection to your Service Virtualization Server

  1. Install the "hpe Application Automation Tools" plugin. (Manage Jenkins > Manage Plugins)
  2. Go to the Jenkins Server home page.
  3. Click the Manage Jenkins link in the left pane.
  4. In the Manage Jenkins Page click Configure System.
  5. In the Configuration tab, scroll down to the Service Virtualization section.
  6. Click Add SV server.
  7. Specify Service Virtualization Server details. The Server URL must be in the following format:
    <scheme>://<Service Virtualization Server IP or hostname>:<port number>/management
    For example:https://agata:6085/management

 

For details on how to use a virtual service in a Jenkins test or how to automate the update of simulation models, see the Continuous Integration section in the Service Virtualization Help Center.

 


Support for Pipelines

Generate pipeline code

To set up a pipeline test job for your HPE tool:

    1. From Jenkins Dashboard. click New Job or select an existing one.
    2. On the page that opens, enter a job name (for a new job), click Build a Pipeline project, and click OK.
    3. In the Project Configuration page, scroll down to the Pipeline section.
    4. Enter the stage and node arguments into the Script area. For example,

      stage('RunUFTTestFromFS'){ // The stage name
          node('Test'){ //  The name of the node in which to run the test.
  1. Prepare the code for your testing tool:
    1. Click the Pipeline Syntax link.
    2. In the Snippet Generator drop down, select the desired step, for example, uftScenarioLoad: Run UFT scenario.
    3. Fill in the fields as required. Fields marked in red are mandatory. Note: For fields that take multiple values, such as in the Tests field, separate multiple entries with a line break.
    4. If relevant, select one of the #Report archive modes.
    5. Click Generate Pipeline Script. Copy the code to the clipboard.
  2. Return to the Project Configuration page, and paste the generated Groovy script into the Script field in the Pipeline section.
  3. Repeat the above steps to add other commands to your script.
  4. Save the script and run or schedule the job as you would with any standard Jenkins job.
  5. After the test run, click the Console link on the dashboard to see a link to your results. Copy the link to your browser (Internet Explorer required to view ALM test sets in ALM).
Supported Pipeline job types

The available Pipeline job types are loadRunnerTest, uftScenarioLoad, runFromAlmBuilder, sseBuild, sseBuildAndPublish, pcBuild, svChangeModeStep, svDeployStep, svExportStep, and svUndeployStep.

Product

Pipeline step name

Description

LoadRunner

loadRunnerTest

Run LoadRunner performance tests from a file system scenario file

UFT

uftScenarioLoad

Run a UFT scenario from file system scenario

ALM

runFromAlmBuilder

Execute functional tests from ALM

ALM Lab Management

sseBuild

Execute tests using ALM Lab Management

ALM Lab Management

sseBuildAndPublish

Execute tests using ALM Lab Management and Publish tests result

Performance Center

pcBuild

Execute tests using Performance Center

Service Virtualization

svChangeModeStep

Change the mode of a Virtual Service

Service Virtualization

svDeployStep

Deploy a Virtual Service

Service Virtualization

svExportStep

Export a Virtual Service

Service Virtualization

svUndeployStep

Undeploy a Virtual Service

Pipeline jobs are not supported for Mobile Center uploads, ALM test uploader, and ALM AUT job types.

Report archive modes

The available archive modes are:

  • Archive test report for failed tests
  • Always archive test reports
  • Always archive and publish test reports (LR only)
  • Do not archive test reports

 


Enabling Non-English Languages

In order to allow the add-in to support non-English languages, make the following changes to your Jenkins Master and nodes configuration:

Master configuration
  1. Open the Jenkins.xml file in the Jenkins installation folder, for example: c:\Jenkins\Jenkins.xml.
  2. Go to the <service> -> <arguments> section and add the -Dsun.jnu.encoding=UTF-8 -Dfile.encoding=UTF-8 flags after the -jar flag.

For example:

 

<service>
<id>jenkins</id>
<name>Jenkins</name>
<description>This service runs Jenkins continuous integration system.</description>
<env name="JENKINS_HOME" value="%BASE%"/>
<!--
This example assumes that you have java in your PATH.
To run Jenkins with a specific version of Java, specify a full path to the desired java.exe.
-->
<executable>%BASE%\jre\bin\java</executable>
<arguments>-Xrs -Xmx256m -Dhudson.lifecycle=hudson.lifecycle.WindowsServiceLifecycle -jar -Dsun.jnu.encoding=UTF-8 -Dfile.encoding=UTF-8 "%BASE%\jenkins.war" --httpPort=8080 --webroot="%BASE%\war"</arguments>
<!--
The *interactive* flag causes the empty black Java window to be displayed.
<interactive />
-->
<logmode>rotate</logmode>
<onfailure action="restart" />
</service>

 

Slave configuration
  1. Open the jenkins-slave.xml file on the Jenkins slave (node) machine in the folder that you designated.
  2. Go to the <service> -> <arguments> section and add the -Dsun.jnu.encoding=UTF-8 -Dfile.encoding=UTF-8 flags after the -jar flag.

 

<service>
<id>jenkinsslave-c__jkns</id>
<name>jenkinsslave-c__jkns</name>
<description>This service runs a slave for Jenkins continuous integration system.</description>
<!--
This example assumes that you have java in your PATH.
To run Jenkins with a specific version of Java, specify a full path to the desired java.exe.
-->
<executable>C:\Program Files (x86)\Java\jre1.8.0_91\bin\java.exe</executable>
<arguments>-Xrs -jar "%BASE%\slave.jar" -Dsun.jnu.encoding=UTF-8 -Dfile.encoding=UTF-8 -jnlpUrl http://xxx.xxx.xxx.xxx:8080/jenkins/computer/VM112233/slave-agent.jnlp -secret xyzabc</arguments>
<!--
The *interactive" flag causes the empty black Java window to be displayed.
<interactive />
-->
<logmode>rotate</logmode>
<onfailure action="restart" />
</service>

Configuration for Java Web Start clients
  1. On the Jenkins master machine, go to Manage Jenkins -> Manage Nodes. Click the relevant node or slave. and select Slave -> Configure -> Advanced.
  2. Add the following to the JVM options text box – “-Dsun.jnu.encoding=UTF-8 -Dfile.encoding=UTF-8”.
  3. Restart your Jenkins master, node, and slave.
  4. Verify that your changes took effect:
    1. For the Master: Go to Manage Jenkins -> System information -> “file.encoding”, “sun.jnu.encoding”. The value should be “UTF-8”.
    2. For the node/slave: On the master machine, go to Manage Jenkins -> Manage Nodes.
      Click on the relevant node or slave and select System information-> “file.encoding”, “sun.jnu.encoding”. The value should be “UTF-8”.
      Workaround: If you find that the slave machine still does not support localized encoding, launch the slave from the command line as shown in the image below:

     For example:

 

java -Dsun.jnu.encoding=UTF-8 -Dfile.encoding=UTF-8 -jar slave.jar -jnlpUrl http://xxx.xxx.xxx.xxx:8080/jenkins/computer/VM112233/slave-agent.jnlp -secret xyzabc

 

     


Tips and Troubleshooting

Integration Issues
  • If your job includes UFT, QTP, or Service Test tests running on a remote ALM/QC machine (run mode = Run Remotely), you should manually stop the test execution.
  • When UFT is installed on the slave machine (node), the LoadRunner test job will fail in the Analyze Result stage. Workaround: Add the path of the LoadRunner bin folder (%LR_PATH%/bin) to the PATH environment variable.
Content Security Policy Header

Starting with version 1.641 (or 1.625.3), Jenkins introduced the Content-Security-Policy header. This prevents some of the links that appear in the integration to become inoperable. For example, the links to the LoadRunner Performance and UFT HTML reports will not work.
Workarounds:

  • View the reports locally on the slave machine, in your Jenkins results folder under the relevant build ID, or on the master machine in the Jenkins builds folder.
    • For UFT HTML reports, go to the %jenkins% \jobs\<job_name>\builds\<job_count>\archive\UFTReport folder. For example, C:\Program Files (x86)\Jenkins\jobs\Uft-htmlReport\builds\80\archive\UFTReport\GuiTest2\run_results.html.
    • For LoadRunner Performance reports, go to the %jenkins% \jobs\<job_name>\builds\<job_count>\PerformanceReport folder. For example, C:\Program Files (x86)\Jenkins\jobs\Plug_Debug\builds\5\PerformanceReport\index.html.
  • Click Manage Jenkins > Script Console, and run one of the following scripts from the console:
    • For UFT HTML and Performance reports, type the following line and press Run.

       

      System.setProperty("hudson.model.DirectoryBrowserSupport.CSP", "")
      
    • Another option for For LoadRunner Performance reports is the following:

       

      System.setProperty("hudson.model.DirectoryBrowserSupport.CSP", "sandbox; default-src 'none'; img-src 'self'; style-src 'self' 'unsafe-inline'; script-src 'self' 'unsafe-inline'; child-src 'self';")
    • An additional option for LoadRunner Performance reports that is more secure, but it disables the left pane menu and embedded Javascript:

       

      System.setProperty("hudson.model.DirectoryBrowserSupport.CSP", "sandbox; default-src 'none'; img-src 'self'; style-src 'self'; script-src 'self'; child-src 'self';")



For best performance, it is recommended to install the testing tools and ALM/QC as nodes on slave machines, and not directly on the Jenkins server.  For instructions on configuring nodes, see
https://wiki.jenkins.io/display/JENKINS/Step+by+step+guide+to+set+up+master+and+slave+machines+on+Windows.
On the Jenkins slave machine, make sure the the Jenkins Slave service is not logged in with the Local System account. To function properly, it should log in with an account the has administrator privileges.

.

Troubleshoot the Jenkins-ALM Octane Integration

If you are working with version 5.1 or later of this plugin, make sure you do not have the old HPE ALM Octane Plugin installed on your Jenkins server.

In addition, here are some suggested solutions for issues you might encounter:

  • Issue: Cannot run pipelines from ALM Octane. The Run button is disabled.
    Possible solutions:
    • The Jenkins server might be down, or it is not communicating with ALM Octane.
    • If this occurred after upgrading from the HPE ALM Octane Plugin: Make sure that in the plugin's ALM Octane CI configuration, you updated the Instance ID with the ID defined in ALM Octane for the CI server. For details, see Upgrade from the HPE ALM Octane CI plugin to this plugin
       
  • Issue: When creating a new pipeline in ALM Octane, the message "Failed to fetch Jenkins job" is displayed in the Job field after you select a CI server.
    Possible solutions:
    • This happens when a Jenkins job indirectly calls itself. Try creating the pipeline from Jenkins. In this case, the log files do not indicate any problem.
      Version 5.2 of this plugin introduced a fix for this issue.
    • See "Errors occur when ALM Octane requests something from Jenkins".
       
  • Issue: Errors occur when ALM Octane requests something from Jenkins.
    When running a pipeline from ALM Octane, the error message is displayed: "Run pipeline: an Error occurred while running the pipeline”
    When creating a new pipeline in ALM Octane, the message "Failed to fetch Jenkins jobs - Error 403 forbidden" is displayed in the Job field after you select a CI server.
    Solution: This is an issue with the permissions of the Jenkins user that ALM Octane uses to execute jobs on the server.
    In the Jenkins configuration, in the ALM Octane CI section define a Jenkins user. This user must have Job Build permissions.
    For details, see Configure the Connection to your ALM Octane Server.
     
  • Issue: When creating a new CI server in ALM Octane, the Jenkins server is not displayed in the list of connected servers even though Test Connection on Jenkins worked.
    Solution: Make sure that the Jenkins proxy settings are correct and that you restarted the Jenkins server after changing proxy settings. For details, see Configure the Connection to your ALM Octane Server.
     
  • Issue: Test results from the pipelines are not reflected in ALM Octane.
    Solution: Test results from Maven jobs are not reported to ALM Octane if the job is under a folder. Free style and pipeline jobs report results as expected.
    Version 5.2 of this plugin introduced a fix for this issue.

Using log files to troubleshoot
Jenkins sends various events and data to ALM Octane. For example: Start job, Finish job, SCM data, test results.
When an error occurs, check the communication between Jenkins and ALM Octane.
To help troubleshoot, you might want to check the log file on the Jenkins server for connectivity issues, security issues, and so on.

The log files are located in the following locations: 
Jenkins system logs: http://<Jenkins url>/log/all
Plugin logs: http://<Jenkins url>/userContent/nga/logs/nga.log

Understanding the plugin's log messages

  • At the end of a successful flow, Jenkins sends: done, left to send 0 events
    For example: 
    INFO - EventsClient: sending events [job\-name:STARTED] to 'http://localhost:8080'...
    INFO - EventsClient: sending events [job\-name:FINISHED] to 'https://mqast010pngx.saas.mf.com'...
    INFO - EventsClient: ... done, left to send 0 events
    INFO - TestDispatcher: There are pending test results, connecting to the MQM server
    INFO - TestDispatcher: Successfully pushed test results of build job-name#10
  • When the ALM Octane server is not available, log messages may look like this:
    ERROR - BridgeClient: connection to MQM Server temporary failed: authentication error com.hp.mqm.client.exception.AuthenticationException: Authentication failed: code=503; reason=Service Unavailable
  • When an error occurs in the Jenkins -> ALM Octane communication, log messages may look like this:
    ERROR - EventsClient: max number of retries reached
    TestDispatcher: There are pending test results, but we are in quiet period
Troubleshoot Creating and Configuring ALM Octane Pipelines
  • Issue: Error when trying to create a pipeline:

     

    Unable to create pipeline

     

    Solution: Make sure the workspace you selected is included in the list of workspaces defined for the access keys that you used to
    connect the HPE ALM Octane CI to ALM Octane.
    If you still encounter an error, select a different workspace, or ask an ALM Octane shared space admin to add the workspace to the list. For details, see Set up API access for integration.
  • Issue: Error when trying to configure a job:

     

    Unable to retrieve job configuration

     

    Solution: The pipeline containing this job might have been created in ALM Octane in a workspace that is not accessible from the HPE ALM Octane CI plugin. 
    Make sure that the workspace is included in the list of workspaces defined for the access keys that you used to connect the HPE ALM Octane CI to ALM Octane. 
    If you still encounter an error, ask an ALM Octane shared space admin to add the workspace to the list. For details, see Set up API access for integration.

 

 

  • No labels