Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Wiki Markup
{jenkins-plugin-info:testng-plugin|jiraComponent=testng|sourceDir=testng-plugin-plugin}
Excerpt

This plugin allows you to publish TestNG results generated using org.testng.reporters.XMLReporter.

TestNG result xml file contains more information than the junit report xml file . This plugin exposes those extra information in graph and table reports.

...

  • TestNG XML report pattern: This is a file name pattern that can be used to locate the TestNG XML report files (for example */target/testng-results.xml*). The path is an Ant-style pattern (e.g. fileset) or a list of files and folders separated by the characters ';:,'. TestNG must be configured to generate XML reports using _org.testng.reporters.XMLReporter _for this plug-in to function.
  • Escape Test Description string?: If checked, the plug-in escapes the description string associated with the test method while displaying test method details. Unchecking this allows you to use HTML tags to format the description. (enabled by default).
  • Escape exception messages?: If checked, the plug-in escapes the test method's exception messages. Unchecking this allows you to use HTML tags to format the exception message e.g. embed links in the text. (enabled by default)
  • Show Failed Builds?: If checked, the plug-in includes results from failed builds in the trend graph. (Disabled by default). 
    • If this is a maven build, it is better to configure the build step with -Dmaven.test.failure.ignore=true option. This results in build with test failures being marked as Unstable, thus distinguishing it from build that failed because of non test related issues 
    • Even when this option is selected, failed builds with no results and aborted builds will not be displayed in graphs
  • Mark build as unstable on Skipped config/test methods?: Marks the build unstable of skipped configuration or test methods are found in results. If build result is worse that UNSTABLE, this option has no effect
  • Mark build as failure on failed configuration?: Marks the build as failed if there are any configuration method failures
  • Thresholds for marking the build as Unstable/Failed: User can configure a Threshold Mode as number of tests or percentage of tests and then configure these values separately as well. This allows for scenarios where we want the builds to be marked successful even when there are some test failures or mark the build as failed even if there is a single test method failure.

Pipeline in Jenkinsfile

According to 

Jira
serverJenkins JIRA
serverIddd058dce-0c66-3b77-8b09-71b1d7728747
keyJENKINS-27121
 
Jira
serverJenkins JIRA
serverIddd058dce-0c66-3b77-8b09-71b1d7728747
keyJENKINS-46979
..

post {
        always {
            step([$class: 'Publisher', reportFilenamePattern: '**/testng-results.xml'])
        }
}

Open Issues

Jira Issues
anonymoustrue
urlhttps://issues.jenkins-ci.org/sr/jira.issueviews:searchrequest-xml/temp/SearchRequest.xml?jqlQuery=project%3DJenkins+AND+component%3Dtestng-plugin+AND+status+in+%28%22In+Progress%22%2C+Open%2C+Reopened%29+ORDER+BY+priority%2C+status%2C+createdDate+ASC

Change Log

Release 1.15 (June 28th, 2018)

  • Fixed: JENKINS-32746 - Change loop determination condition in order to check all loaded builds (Julien Herr)
  • Fixed: JENKINS-26971 - Looking at failed TestNG result causes an exception (Vladimir Sitnikov)
  • Added: JENKINS-51341 - Display suite name at main page and at method result page (Vladimir Sitnikov)
  • PR-35 - Show "failed configurations" earlier than "failed tests" (Vladimir Sitnikov)

Release 1.14 (July 5th, 2016)

  • Added: Update to new parent POM (jglick)
  • Added: JENKINS-27121 - Pipeline compatibility (jglick)
  • Fixed: General code quality fixes (faisal-hameed)
  • Added: Added some basic form validation

Release 1.13 (May 15th, 2016)

  • Fixed: JENKINS-34822 - TestNG plugin has incorrect default threshold values

Release 1.12 (May 10th, 2016)

  • Fixed: JENKINS-34711 - NullPointerException after Update to 1.11 of plugin

Release 1.11 (May 8th, 2016)

  • Added: Add ability to configure failed/skipped test thresholds (ChrisEverling) (see JENKINS-20985)

Release 1.10 (March 31st, 2015)

  • Upped compatible Jenkins version to v1.554.3
  • Added: MIT license
  • Added: Add property to fail build on config fail (damienbiggs)
  • Added: Make sure TestNGTestResultBuildAction caches counts and loads results lazily (Jesse Glick)

Release 1.9.1 (August 13th, 2014)

  • Make duration calculation for PackageResults more accurate
  • Retain white-space formatting in reporter output (steveims)

Release 1.9 (August 11th, 2014)

  • Fixed the duration values to be more accurate by accounting for tests executing in parallel
  • Fixed: JENKINS-24176 - Test result trend breaks lazy-loading (Jesse Glick)
  • Fix default value of testng.reportFilenamePattern (juherr)

Release 1.8 (July 20th, 2014)

  • Fixed: Added back package names in certain places where they were helpful (see JENKINS-19217)

...

  • Fixed: JENKINS-11236 - TestNG plugin links to wrong document (kohsuke)
  • Fixed: JENKINS-11045 - Hold TestNG Results in memory using weak references
  • Performance improvements (kutzi)
  • Improve Load time for graphs (kutzi) + trivial cleanup

Release 0.28 (September 20th, 2011)

  • Bumped up supported Jenkins version to v1.403 or later
  • Fixed: JENKINS-9918 - Limit the number of rows in method execution order table

Release 0.27 (September 11th, 2011)

  • Added: JENKINS-10882 - Add duration in the XML API for test-result, package and classes Nicolas De Loof
  • Added: JENKINS-10018 - New graph for test method details page

Release 0.26 (July 6th, 2011)

  • Added: JENKINS-8926 - Add group name to class test methods page
  • Added: JENKINS-9969 - Store/Display user provided custom test name

Release 0.25 (June 24th, 2011)

  • Fixed: JENKINS-10000 - Config methods for classes with no test methods are not displayed
  • Added: JENKINS-10001 - Store/Display group names (if any) for test methods
  • Added: JENKINS-10098 - Add test/suite names to class results page

Release 0.24 (June 13th, 2011)

  • Fixed: JENKINS-9742 - Test Report exposes data via the Remote APIs now

Release 0.23 (June 8th, 2011)

  • Fixed: JENKINS-9835 - Remove isRelativePath option
  • Fixed: JENKINS-9836 - Start storing testng-results.xml with build artifacts
  • Moved help information from configuration options to help.html
  • Fixed: JENKINS-9839 - Fix graph refresh after a build is deleted
  • Added: JENKINS-9837 - capture parameters used to run the test method
  • Changed to start appending '_x' where 1<=x<=n to test method names in urls
    to disambiguate multiple tests with same name from same class. This stops
    using the UUID that was being associated with all the test methods.

(NOTE: There is a slight break in backward compatibility in this release.
URLs for older data for test methods will change. Also, significant code changes and
features have been introduced in this version. Older builds might face some issues,
though the chances are very low.)

Release 0.22 (May 29th, 2011)

  • A whole lot of code cleanup. Removed a lot of unnecessary classes. No new features.

Release 0.21 (May 20th, 2011)

  • Have checkboxes for escaping test description and exception message selected by default
  • Changes to how exceptions are displayed. No need to repeat message, highlight exception class etc
  • Removed code supposed to make this work as Maven plugin. Was all unnecessary code.
  • Fixed NPE because of GroupTestRun

Release 0.20 (May 9th, 2011)

  • New: JENKINS-9605 - Add option to not HTML-sanitize localized messages of exceptions

Release 0.19 (May 2nd, 2011)

  • Fixed: JENKINS-9083 - fix urls in results trend chart (wasn't completely fixed in v0.18)

...