This plugin allows the user to create Single and Pipeline Jobs to build and execute VectorCAST Projects. Coverage is displayed using the VectorCAST Coverage Plugin.

This plugin adds a new top-level menu item to the Jenkins sidebar. Select the VectorCAST menu item to create a new VectorCAST Job.

There are two build/execute job types and one diagnostic job type.

• Single Job
• Pipeline Job
• Diagnostic Job

Single Job creates a Single Jenkins Job to build/execute and (optionally) report on all environments in a VectorCAST Project.

Pipeline Job creates a Pipeline Jenkins Job to build/execute and (optionally) report on all environments in a VectorCAST Project in parallel. A Pipeline script (Jenkinsfile) is created to run a build/execute job for each environment and an overall job to combine the results.

⚠️ Pipeline Jobs require VectorCAST 2018 as a minimum version and VectorCAST 2019SP3 to perform parallel execution on a single VectorCAST Project.

There are two options for running tests:

• Using an SCM system (any that is supported by Jenkins).
  • Jenkins will check out the source code and tests into the workspace for each Jenkins Job from the user's repository.
  • The Pipeline Job will then combine the coverage and test results from all of the individual machines/nodes.
  • The VectorCAST Project should be specified as relative to the root of the checkout.
  • There is now an option to use the main Pipeline Job's Workspace as a dedicated single checkout directory. This checkout directory must be available to all executors across all nodes either by having all executors running on the same computer or having the main Pipeline Job's Workspace on a shared network drive.
• Using an existing drive/directory for the VectorCAST Project.
  • The VectorCAST Project should be specified as an absolute path that is available on all machines/nodes.
  • Each job can optionally clean up the working directory which will have no effect on the VectorCAST Project since it is located elsewhere.
  • The reports are generated into the workspace and archived as part of the Jenkins Job.

The user will be able to disable the use of Change Based Testing to perform a complete run of their VectorCAST Project. By default, Change Based Testing is enabled but this option can be disabled by unchecking the Use Change Based Testing box.

Users with Continuous Integration Licenses can access those licenses by checking the Use Continuous Integration License checkbox. If you do not have Continuous Integration Licenses, do not check this box as you will encounter licensing errors.

If the user wishes to call the Jenkins Job from another Pipeline Job, check the Use Parameterize the Jenkinsfile box. This will add parameters to the Pipeline Job that will be used by the VectorCAST Pipeline Job to locate an external VectorCAST Project (VCAST_PROJECT_DIR) and force the VectorCAST Jobs to be executed on a specific node (VCAST_FORCE_NODE_EXEC_NAME) instead of using the compiler as a node label.

Calling the build command will return Failed, Unstable, or Success corresponding to the results of the VectorCAST Pipeline Job. To allow the main Pipeline Job to continue on error, the user can surround the build command without a catchError block as demonstrated below.

catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE', catchInterruptions : false ) {
   build job: 'UnitTestingProject_vcast_pipeline', parameters: [string(name: 'VCAST_PROJECT_DIR', value: 'C:\\UnitTesting\\Project'), string(name: 'VCAST_FORCE_NODE_EXEC_NAME', value: 'MyTestNode')]
}

Additionally, if a shared artifact directory is specified, jobs for the VectorCAST Project will be executed independently but have a central location for storing the build artifacts. This option can be used to accelerate testing by use of VectorCAST's Change Based Testing feature.

For Pipeline Jobs, the plugin processes the build log to determine which tests have been skipped by VectorCAST's Change Based Testing feature. Any test cases that previously passed and were skipped on the current execution will be marked as skipped for JUnit and displayed in yellow on the test result trend chart and denoted as Skipped in the test results list.

Other Vector tool integrations are supported by this plugin.

• PC-lint Plus
• Squore
• TESTinsights

• For PC-lint Plus, the user must provide the command or script that executes PC-lint Plus analysis. It is important that the following format properties are included to ensure the results file is readable by the Warnings-NG plugin:

    -v // turn off verbosity
    -width(0) // don't insert line breaks (unlimited output width)
    -"format=%f(%l): %t %n: %m"
    -hs1 // The height of a message should be 1

• For Squore analysis, the user must provide the Squore command found on the last page of the Squore project's configuration/build page.
• For TESTinsights, if the user is using Git or Subversion for SCM, the plugin will attempt to have the SCM version linked to the TESTinsights project for team access and distributed change based testing. The user must provide the following:
  • TESTinsights URL - The URL to TESTinsights server and project (Use Copy Team Area Url).
  • TESTinsights Project - The Project Name in TESTinsights to push the results (leave blank to use the Jenkins Job Name).
  • TESTinsights Credential ID - The Credential ID from Jenkins for TESTinsights.
  • The proxy to push to TESTinsights server in the format proxy.address:port (optional)

When using a Pipeline Job, the sub jobs are created to run on specific node related to the compiler chosen for the environment. For example:

Make sure to set the labels on the relevant Jenkins nodes. Manage Jenkins -> Manage Nodes -> configure appropriate node and set the ‘Labels’ field. In this example the labels have been set to GNU_Native_5.1_C and GNU_Native_4.8_C

• VectorCAST must be installed and setup on each node.
• The environment variables VECTORCAST_DIR and VECTOR_LICENSE must be set.
• Jenkins must be installed and setup.
• BUILD_URL must be defined (in Jenkins->Manage Jenkins->Configure System and define 'Jenkins URL' and save the settings).

VectorCAST HTML reports for metrics were updated to use cascading style sheets (CSS) in the 2019 release and 2020 for top level project metrics. This was done to give users more flexibility in what and how metrics are displayed. To maintain single file HTML format, VectorCAST Reports used inline CSS. Inline CSS was disallowed under Jenkins more restrictive CSP.

"Jenkins 1.641 / Jenkins 1.625.3 introduce the Content-Security-Policy header to static files served by Jenkins (specifically, DirectoryBrowserSupport). This header is set to a very restrictive default set of permissions to protect Jenkins users from malicious HTML/JS files in workspaces, /userContent, or archived artifacts."

The result of this combination incorrectly formatted the VectorCAST reports.

Numerous options are available to correct this:

• Use the Jenkins Resource Root URL (Manage Jenkins > Configure System)
• Reconfigure VectorCAST Jobs to use external CSS (VCAST_RPTS_SELF_CONTAINED=FALSE) combined with enabling anonymous reads from the Manage Jenkins > Configure Global Security > Authorization
• Reconfigure the Jenkins Content Security Policy
• Download the archives and view reports locally

For more information on the Jenkins CSP, please see Configuring Content Security Policy

For more information on VectorCAST Reports and Jenkins Content Security Policy, please see the article VectorCAST Reports and Jenkins Content Security Policy

For non-pipeline jobs, JUnit publisher will fail any environments published without test results. If you have an environment with no test results, you will need to manually check the box "Do not fail the build on empty test results" in the Publish JUnit test result report configuration.

For customers using VectorCAST's requirements gateway, there's a potential for loss of requirements data when running test environments in parallel while using a shared requriments database.

• Added support in VectorCAST version 2023 Projects for environments that are not in a group

• Decoupled the two parameters used for parameterized pipeline
• Fixed skipped test analysis issue for overloaded C++ methods
• Fixed error in pipeline syntax "continue" changed to "return" for each loop
• Fixed single job error when not using CBT for missing incremental build rebuild report
• Added option to have pipeline job be marked as failed if statement or branch coverage decreases
• Added post-SCM step to run commands after SCM checkout before the project build/execute
• Fixed error identifying unbuilt migrated system test projects

• findbugs-related warnings in the output from maven are fixed in the VectorCAST/Execution plugin v0.74
• The Jenkinsfile base has been restructured and commented for pipeline jobs
• Added catchError to additional tools stage
• Additional Java tests incorporated into the plugin build
• The VectorCAST icon has been updated to match the Jenkins design aesthetic
• Previously, errors indicated that calls cannot run in the Jenkins sandbox. VectorCAST/Execution v0.74 fixed errors caused by the updated version of Java required for the Jenkins plugin.

• Fixed support for entering a Windows path for PC Lint Plus or Squore commands.
• Fixed support for Continuous Integration Licenses.
• Fixed support for System Test Environments with Squore.
• Fixed issue when TESTinsights project name is not given, it is named after the job.

• Support was removed for the deprecated VectorCAST Multi-Jobs. Users should create VectorCAST Pipeline Jobs in place of VectorCAST Multi-Jobs.
• Previously, VectorCAST Jobs were executed on the built-in node by default. Now, when creating new VectorCAST Job, the user must provide the name of the node that the job should be executed on. Jenkins advises against running builds on the built-in node. See Controller Isolation for more information.
• A new stage was added to VectorCAST Pipeline jobs that runs the system tests in serial after running all unit tests. This prevents multiple instances of VectorCAST from modifying the same instrumented source files at the same time.
• Support for additional external tools was added for VectorCAST jobs. See the Additional Tools options on the job creation page.
  • PC-lint Plus
  • Squore
  • TESTinsights
• Options were reorganized on the VectorCAST Pipeline Job creation page.
• The VectorCAST Execution Plugin version is now displayed on the VectorCAST Jobs page.
• Previously, custom reports that used the VCAST_MANAGE_PROJECT_DIRECTORY environment variable would fail to generate. Now, the environment variable is set when the reports are generated.

• Updated baseline pipeline script to reflect changes in the code coverage plugin
• Updated to skip CBT analysis on migrated system test projects
• Updated script to use import to detect DataAPI.

• Updated scripts to use io.open for all opens
• Updated parse_traceback to display all tracebacks

• Updated pipeline job to handle continuous integration licenses

• New HTML reports match current Jenkins CSP
• Encoding read for GBK characters
• Bug fixes for compatibility issues with VectorCAST/2021
• Bug fix for counting run system test cases as skipped
• Handle uppercase VectorCAST Project .vcm extension

• Add support for Python 3.9

• Fixed bug caught by updated to Python 3
• Fixed command capture for processing later
• Changed side panel display of VectorCAST to not be shown when anonymous user is using Jenkins

• Single jobs not displaying full report and rebuild report correctly in summary (VC2020)
• Capturing stdout for skipped analysis causing console output to be delayed
• Pipeline job fails unless user specific .vcm for VectorCAST Project
• Problem with CBT analysis when not using single checkout directory
• Using lowercase %workspace% causes single checkout directory error
• Updated managewait.py to overcome race condition causing script to hang on readline()
• Added skipped test analysis to Single Jobs
• Added detection of disabled environment to DataAPI generated results
• Added addition error detection

• Skipped tests not detected for cover environments
• Check for illegal characters in pipeline job names
• Add Option to not use CBT in Jenkins pipeline plugin
• VC2020 Manage reports have no formatting in Jenkins job summary
• Remove xUnit reporting
• Remove truncated output during the build
• "Could not acquire a read lock on the project's vcm file" should raise an error, but it does not
• Add option to active CI licenses
• Regression Scripts with identical ending directories show testcase as skipped
• Add option to make job parameterized for VectorCAST Project location and Force Node Execution rather than compiler
• Update error thrown when user creates absolute path with SCM snippet
• Need to add more error detection
• Expand JUnit test case name display to include file.subprogram
• Change reports to not use VCAST_RPTS_SELF_CONTAINED=FALSE

• Support Multi-Branch Pipelines
• Include plugin version in Pipeline Job Jenkins Script and console log
• Check for illegal characters in Pipeline Job names
• Update Single and Multi-Jobs to use same reporting as Pipeline Jobs
• Remove CombinedReport.html from previous Pipeline job build at beginning of new build
• Make aggregate coverage results consistent with VectorCAST Project aggregate coverage results
• Always use new VectorCAST report API for VectorCAST/2019 and later
• Fix potential file access race condition when generating xml
• Fix exception when building VectorCAST/Unit regression script environments
• Support for VectorCAST/vpython 3

• Introduce pipeline job

• Updated support for Jenkins Pipeline
• Updated python scripts to work with VectorCAST 2019 SP3
• Added fix for Ada environments not building properly

• Update support for Manage API and VCAST_RPTS_SELF_CONTAINED

• Update for VECTORCAST_DIR rollback

• Update for different drive than workspace

• Updates for duplicate results in multijob

• Revert changes from 0.43 and 0.44 to remove need for VECTORCAST_DIR. VECTORCAST_DIR will need to be defined when running Jenkins for now.

• Add support for new VCAST_RPTS_SELF_CONTAINED option added in VC19 SP2
• Add support for using Manage API to generate XML reports if available
• Correct variable expression on Linux platforms
• Updated scripts to handle compound only in compound case

• Updates for F+FC coverage and updating database pathing

• Updates for changed to aggregate coverage report metrics heading change

• Additional debug logging

• Updates for scripts not handling Ada operator "+"

• Update exception handling when using DataAPI

• Updated to support VectorCAST 2019 SP1

• Update to get complete display name for C++ functions
• Update to fix characters that need escaping in XML attributes

• Fix for disabled environments
• Fix for printing non UTF-8 compliant failure code

• Updates for using external CSS/images

• Removed xUnit dependency and converted to JUnit.

• Added better legacy support for VectorCAST installations that do not have their executables on the system PATH

• Removed all uses of the environment variable VECTORCAST_DIR. From now on it is assumed that VectorCAST executables are on the system PATH. Legacy support is still maintained for older versions of VectorCAST.
• Additional cleaning up of old files

• Updates for corner cases, verbose out issue, and cleaning up previous build's files
• Problem when function coverage enabled, but not function call
• Function coverage format incorrect in XML causing plugin to throw an error
• Added catch for additional licensing errors
• Added catch for all (E) Line: errors in the console log

• Fix for function and basis path coverage when using VectorCAST 2019

• Update to fix auto job updates (where path to VectorCAST Project was being removed)

• Update to make the management report generate for a cover project

• Fix for spurious newline characters in report title in XML for Jenkins with VectorCAST 2019

• Corrected missing " that may affect running multi-job on Linux
• Added support for generating reports using VectorCAST 2019

• Support overlapping version 17 VectorCAST Projects
• Updates to support long directory paths in VectorCAST Project reporting

• Support newer versions of xUnit plugin

• Support MultiJob plugin up to 0.29 and later, 0.30 onwards

• Don't create intermediate CSV file for bad test case management report

• Raise post-groovy alert for bad test case management report

• Improve support for unit without coverage, avoiding corrupt xml files

• Removed spurious " in Linux single job
• Corrected link from xUnit graph to report

• Correct regression with report naming for archived artifacts with shorter names
• Added environment variable (VCAST_VC_SCRIPTS) to provide optional source of vc_scripts

• Improve support for long VectorCAST Project names, environment names and compiler names

• Correct regression with windows variable names being used in Unix script

• Correct regression with missing space in commands for single job

• Option to set the name of the single job or multi-job (name is pre-pended to sub-job in the case of multi-jobs)
• Option to configure (at creation/update time) the node to run the single job or top-level multi-job on
• Allow license retries for single jobs
• Update summary/detailed text written by the groovy scripts

• Update to retry functionality to support jobs created with earlier plugin versions

• Update to store and use job details when auto-updating
• Added (optional) ability to retry a command if it fails due to unavailable license

• Added a job that can be used to update an existing multi-job

• Added support for new version of VectorCAST Manage that uses 2 levels instead of 4

• Improved groovy script to mark failing builds as failed rather than unstable

• Allow conversion script to accept report that has a missing or incomplete Function Coverage column

• Added --force option to use of --release-locks
• Added option to use either HTML or TEXT format for the build description

• Add execution report link to all test cases
• Added update to pulling in both the full report and incremental build report into the job build description
• Added update to pulling in both the full report and incremental build report into the job build description

• Always display the VectorCAST menu and leave permission checking/reporting to Jenkins

• Corrected processing checking if BUILD_URL has been set

• Corrected processing to support function and function call coverage

• Corrected typos in Diagnostics job and pattern for files to copy

• Add support for spaces in paths

• Add support to keep or clean the working directory

• Support added for multi-job with SCM and for calculating correctly aggregated coverage for the top-level display in the VectorCAST coverage plugin

• Initial release (no support for using SCM with multi-job)