Skip to end of metadata
Go to start of metadata

Plugin Information

View TestNG Results on the plugin site for more information.

This plugin is up for adoption. Want to help improve this plugin? Click here to learn more!

This plugin allows you to publish TestNG results generated using org.testng.reporters.XMLReporter.TestNG result xml file contains more information than the junit report xml file . This plugin exposes those extra information in graph and table reports.

This plugin makes it possible to import TestNG XML reports from each build into Jenkins.

The data is parsed using the output generated using org.testng.reporters.XMLReporter. The results are displayed with a trend graph and all details about which tests that failed are also presented.

Features

The page layouts are intentionally designed to look very similar to Jenkins's JUnit plugin.

  • In the project/build page you can see a summary of passed/fail tests as well as passed/fail configuration methods. It also enlists the failed test cases with a direct link to view the test failure.

  • In the Class result summary page test methods are separated from configuration methods

  • In the Package result summary you can see the order which test methods were ran during the regression run.

  • Different graphs will show you the pass/failed trend of job's previous results.

  • Able to hide/expand tables that display package and class results

Usage

Once you have configured your build to run TestNG based test cases and also generate the TestNG specific result XML, using this plugin is very simple. In Post-Build Actions, simply enable Publish TestNG Results. This option allows you to configure the following properties:

  • TestNG XML report pattern: This is a file name pattern that can be used to locate the TestNG XML report files (for example */target/testng-results.xml*). The path is an Ant-style pattern (e.g. fileset) or a list of files and folders separated by the characters ';:,'. TestNG must be configured to generate XML reports using _org.testng.reporters.XMLReporter _for this plug-in to function.
  • Escape Test Description string?: If checked, the plug-in escapes the description string associated with the test method while displaying test method details. Unchecking this allows you to use HTML tags to format the description. (enabled by default).
  • Escape exception messages?: If checked, the plug-in escapes the test method's exception messages. Unchecking this allows you to use HTML tags to format the exception message e.g. embed links in the text. (enabled by default)
  • Show Failed Builds?: If checked, the plug-in includes results from failed builds in the trend graph. (Disabled by default). 
    • If this is a maven build, it is better to configure the build step with -Dmaven.test.failure.ignore=true option. This results in build with test failures being marked as Unstable, thus distinguishing it from build that failed because of non test related issues 
    • Even when this option is selected, failed builds with no results and aborted builds will not be displayed in graphs
  • Mark build as unstable on Skipped config/test methods?: Marks the build unstable of skipped configuration or test methods are found in results. If build result is worse that UNSTABLE, this option has no effect
  • Mark build as failure on failed configuration?: Marks the build as failed if there are any configuration method failures
  • Thresholds for marking the build as Unstable/Failed: User can configure a Threshold Mode as number of tests or percentage of tests and then configure these values separately as well. This allows for scenarios where we want the builds to be marked successful even when there are some test failures or mark the build as failed even if there is a single test method failure.

Pipeline in Jenkinsfile

According to  JENKINS-27121 - Getting issue details... STATUS   JENKINS-46979 - Getting issue details... STATUS ..

post {
        always {
            step([$class: 'Publisher', reportFilenamePattern: '**/testng-results.xml'])
        }
}

Open Issues

T Key Summary Assignee Reporter P Status Resolution Created Updated Due
Loading...
Refresh

Change Log

Release 1.15 (June 28th, 2018)

  • Fixed: JENKINS-32746 - Change loop determination condition in order to check all loaded builds (Julien Herr)
  • Fixed: JENKINS-26971 - Looking at failed TestNG result causes an exception (Vladimir Sitnikov)
  • Added: JENKINS-51341 - Display suite name at main page and at method result page (Vladimir Sitnikov)
  • PR-35 - Show "failed configurations" earlier than "failed tests" (Vladimir Sitnikov)

Release 1.14 (July 5th, 2016)

  • Added: Update to new parent POM (jglick)
  • Added: JENKINS-27121 - Pipeline compatibility (jglick)
  • Fixed: General code quality fixes (faisal-hameed)
  • Added: Added some basic form validation

Release 1.13 (May 15th, 2016)

  • Fixed: JENKINS-34822 - TestNG plugin has incorrect default threshold values

Release 1.12 (May 10th, 2016)

  • Fixed: JENKINS-34711 - NullPointerException after Update to 1.11 of plugin

Release 1.11 (May 8th, 2016)

  • Added: Add ability to configure failed/skipped test thresholds (ChrisEverling) (see JENKINS-20985)

Release 1.10 (March 31st, 2015)

  • Upped compatible Jenkins version to v1.554.3
  • Added: MIT license
  • Added: Add property to fail build on config fail (damienbiggs)
  • Added: Make sure TestNGTestResultBuildAction caches counts and loads results lazily (Jesse Glick)

Release 1.9.1 (August 13th, 2014)

  • Make duration calculation for PackageResults more accurate
  • Retain white-space formatting in reporter output (steveims)

Release 1.9 (August 11th, 2014)

  • Fixed the duration values to be more accurate by accounting for tests executing in parallel
  • Fixed: JENKINS-24176 - Test result trend breaks lazy-loading (Jesse Glick)
  • Fix default value of testng.reportFilenamePattern (juherr)

Release 1.8 (July 20th, 2014)

  • Fixed: Added back package names in certain places where they were helpful (see JENKINS-19217)

Release 1.7 (June 28th, 2014)

  • Fixed: JENKINS-23285 - All links on trend graph map are wrong on "testngreports" subpage
  • Fixed: JENKINS-19217 - Change class names to not contain package names on result pages

Release 1.6 (Feb 9th, 2014)

  • Fixed: JENKINS-19353 - Exception error message newlines are escaped
  • Fixed: JENKINS-20968 - Doesn't resolve parameters in "TestNG XML report pattern" field
  • Moved some configuration into Advanced section and cleaned up help sections
  • Added: JENKINS-20438 - Make it an option to show failed builds in trend graph
  • Added: New option to mark build as unstable on finding skipped config or test methods

Release 1.5 (Aug 15th, 2013)

  • Fixed: JENKINS-19157 - Make st:bind work when Jenkins has jQuery plugin installed
  • Added: failure details to failed configuration methods on TestNG results page

Release 1.4 (Aug 6th, 2013)

  • Bumped up supported Jenkins version to v1.526 or later
  • Integrated TestNG plugin results into email-ext plugin

Release 1.3 (Jul 22nd, 2013)

  • Added: JENKINS-11899 - Print all URLs as HTML anchors (when escaping description and exception message)
  • Added: JENKINS-12275 - TestNG plugin should contribute to Dashboard View plugin
  • Fixed: JENKINS-18750 - NPE in testng graph
  • Fixed: JENKINS-17564 - Reporter output being escaped incorrectly

Release 1.2 (Jun 30th, 2013)

  • Added: Error details to failed test list
  • Fixed: JENKINS-18498 - hudson.tasks.test.AggregatedTestResultAction$1 cannot be cast to hudson.tasks.test.TestResult
  • Fixed: JENKINS-16832 - Do not show failed builds in trend graph
  • Fixed: JENKINS-18206 - Total duration of a test class should include the times for the configuration methods

Release 1.1 (Feb 1st, 2013)

  • Fixed: JENKINS-16592 - Invalid URL in testngreport on tests execution history chart

Release 1.0 (Jan 21st, 2013)

  • Added: Implement AbstractTestResultAction
  • Removed: deprecated class BuildIndividualReport (deprecated since v0.23)
  • Removed: deprecated fields isRelativePath, results, fullName (were deprecated since v0.23 or earlier)
  • Changed: duration in results to be seconds instead of milliseconds (as we now implement AbstractTestResultAction)
  • Removed: Commented out Age column from reports as it wasn't symmetrical to Junit reports. Needs re-implementation
  • Fixed: JENKINS-16374 - Allow concurrent builds for Projects using TestNG plugin to proceed

Release 0.33 (Jan 06th, 2013)

  • Upped compatible Jenkins version to v1.447
  • Fixed: JENKINS-15119 - Duration sorting is incorrect in TestNG report
  • Fixed: JENKINS-12046 - TestNG shouldn't look for result files if build was aborted (again!)
  • Added: JENKINS-10832 - make testng reporter-output field viewable from testng-plugin generated reports (Dan Alvizu)

Release 0.32 (February 14th, 2012)

  • Fixed: JENKINS-12648 - Results should be reported for failed/aborted builds as well (Reverted fix for JENKINS-12046 in v0.30)

Release 0.31 (January 31st, 2012)

  • Fixed: JENKINS-12187 - Ignore testng XMLs (if any) from previous builds
  • Fixed: JENKINS-12494 - NPE while parsing testng xml with empty full stacktrace and no short stacktrace (Benoit Guerin)
  • Fixed: JENKINS-12567 - skipped tests and configurations shouldn't mark the build as unstable (Christoph Kutzinski)

Release 0.30 (January 22nd, 2012)

  • Fixed: JENKINS-12046 - TestNG shouldn't look for result files if build was aborted
  • Fixed: JENKINS-11158 - Don't fail build if result files cannot be found
  • Added: More logging around what the plugin is doing
  • Improved TestNG trend graph accuracy. It now shows all builds, even ones for which there were no TestNG results

Release 0.29 (December 4th, 2011)

  • Fixed: JENKINS-11236 - TestNG plugin links to wrong document (kohsuke)
  • Fixed: JENKINS-11045 - Hold TestNG Results in memory using weak references
  • Performance improvements (kutzi)
  • Improve Load time for graphs (kutzi) + trivial cleanup

Release 0.28 (September 20th, 2011)

  • Bumped up supported Jenkins version to v1.403 or later
  • Fixed: JENKINS-9918 - Limit the number of rows in method execution order table

Release 0.27 (September 11th, 2011)

  • Added: JENKINS-10882 - Add duration in the XML API for test-result, package and classes Nicolas De Loof
  • Added: JENKINS-10018 - New graph for test method details page

Release 0.26 (July 6th, 2011)

  • Added: JENKINS-8926 - Add group name to class test methods page
  • Added: JENKINS-9969 - Store/Display user provided custom test name

Release 0.25 (June 24th, 2011)

  • Fixed: JENKINS-10000 - Config methods for classes with no test methods are not displayed
  • Added: JENKINS-10001 - Store/Display group names (if any) for test methods
  • Added: JENKINS-10098 - Add test/suite names to class results page

Release 0.24 (June 13th, 2011)

  • Fixed: JENKINS-9742 - Test Report exposes data via the Remote APIs now

Release 0.23 (June 8th, 2011)

  • Fixed: JENKINS-9835 - Remove isRelativePath option
  • Fixed: JENKINS-9836 - Start storing testng-results.xml with build artifacts
  • Moved help information from configuration options to help.html
  • Fixed: JENKINS-9839 - Fix graph refresh after a build is deleted
  • Added: JENKINS-9837 - capture parameters used to run the test method
  • Changed to start appending '_x' where 1<=x<=n to test method names in urls
    to disambiguate multiple tests with same name from same class. This stops
    using the UUID that was being associated with all the test methods.

(NOTE: There is a slight break in backward compatibility in this release.
URLs for older data for test methods will change. Also, significant code changes and
features have been introduced in this version. Older builds might face some issues,
though the chances are very low.)

Release 0.22 (May 29th, 2011)

  • A whole lot of code cleanup. Removed a lot of unnecessary classes. No new features.

Release 0.21 (May 20th, 2011)

  • Have checkboxes for escaping test description and exception message selected by default
  • Changes to how exceptions are displayed. No need to repeat message, highlight exception class etc
  • Removed code supposed to make this work as Maven plugin. Was all unnecessary code.
  • Fixed NPE because of GroupTestRun

Release 0.20 (May 9th, 2011)

  • New: JENKINS-9605 - Add option to not HTML-sanitize localized messages of exceptions

Release 0.19 (May 2nd, 2011)

  • Fixed: JENKINS-9083 - fix urls in results trend chart (wasn't completely fixed in v0.18)

Release 0.18 (April 28th, 2011)

  • Fixed: JENKINS-9083 - handle jenkins context path in URLs properly
  • Updated pom dependency to jenkins 1.399

Release 0.17 (April 24th, 2011)

  • Fixed JENKINS-9299 : test results trend chart not showing
  • Updated pom dependency to jenkins 1.396

Release 0.16 (March 25th, 2011)

  • Fixed the breakage for new jenkins build because we were using ChartUtil class methods which are deprecated in hudson since 1.320
  • Bumped the dependency revision to jenkins 1.395. This plugin now only works with jenkins. Shoot me an email if you are still using Hudson and I will try to release a version for hudson as well.

Release 0.15 (March 24th, 2011)

  • Fixed JENKINS-9002 : cssClass is not set properly for ClassResult view in testng-plugin
  • Fixed JENKINS-8976 : testng plugin is not able to differentiate between multiple test-methods run as part of the same <class> in data provider mode
  • Fixed JENKINS-9157 : Package results should now aggregate up the duration totals correctly

Release 0.14 (February 18th, 2011)

  • Fixed JENKINS-8828 : exception.message is not displayed correctly for failed tests
  • Fixed JENKINS-7916 : FormatUtil method is using TimeUnit.toMinutes which is not defined in Java 1.5

Release 0.12 (February 12th, 2011)

  • Fixed : The plugin wasen't showing all the @Test methods which ran as part of the same class. For example you can have multiple @Test methods in the same class with couple of @Before and @After methods.Release 0.11 (February 8th, 2011)
  • Fixed : diff value was display as --3 instead of 3 and -3 instead of +3
  • Fixed : Checked in the missing sidebar icon files

Release  0.7 (October 25, 2010)

  • Fixed: Plugin fails to find testng-results.xml file when project is configured to checkout sources from SVN SCM.

Release  0.5 (September 24, 2010)

  • Fixed: parser test cases were failing with IOException
  • Fixed: Updated colors for Trend Graph to match JUnit graph and bar colors
  • Fixed: Better display of content on the test method result page
  • Fixed: Open/Close js behavior on Package results page
  • Fixed: Removed a lot of dead code and unnecessary files
  • Fixed: Trend Graph is shown for Projects and not for Builds
  • Changed: TestNG Results link at Project page doesn't take you to latest build results anymore
  • New: Trend graph has tooltips and links to different builds

Release  0.4 (August 17, 2010)

  • New: Initial release

Release  0.1 (August 12, 2010)

  • First Revision

46 Comments

  1. The current version available is 0.7. Can I find the release notes for that version somewhere?
    I am using version 0.5 and since I have no test environment available so far, I need to have an indication whether I take the risk of updating the plugin or not.

    BTW, I finally found the readme in the source code. Which looks pretty good. How about just copying the content of the file on this page. It would be great.

  2. Unknown User (robert.mark.mills@gmail.com)

    I tried out the 0.7 version, and the plugin doesn't appear to display properly to me. There is merely a check box for "TestNG Results" under the project configuration. I don't see the additional fields specified in the hudson.plugins.testng.Publisher/config.jelly file (and documented in the wiki page above). Perhaps the plugin is not built correctly?

    Thanks,

    Mark

    1. Unknown User (farshid.ghods@gmail.com)

      Mark,

      are you still facing the issue after upgrading to version 0.8 ?

  3. I tried to use 0.7 but it seems to have some problems I couldn't resolve. The latest test result and a graph are shown but class/package test results are missing. Clicking on the 'TestNG Results' link returns a 404 response because no testngresults directory has been created by the plugin.

    I have an ant task that runs my tests:

    <testng outputdir="Test/Reports" reporter="org.testng.reporters.XMLReporter" classpathref="test.classpath">
    <xmlfileset dir="Test" includes="testng.xml"/>
    </testng>

    which are written to the project directory into Test/Reports. For the Hudson job I configured testng-plugin to publish TestNG results and set the report pattern to 'Test/Reports/**/*.xml'. On the console output there is only a line stating that the xml file was parsed. In the system log I see a bunch of warnings:

    Nov 12 11:03:43 v-100003-svr01 com.hudson78: 12.11.2010 11:03:43 hudson.plugins.testng.parser.ResultPullParserHelper parseToTagIfFound
    Nov 12 11:03:43 v-100003-svr01 com.hudson78: WARNUNG: next() threw exception : already reached end of XML input (position: END_DOCUMENT seen ...</testsuite>\n... @10:1)
    Nov 12 11:03:43 v-100003-svr01 com.hudson78: 12.11.2010 11:03:43 hudson.plugins.testng.parser.ResultPullParserHelper parseToTagIfFound
    Nov 12 11:03:43 v-100003-svr01 com.hudson78: WARNUNG: next() threw exception : already reached end of XML input (position: END_DOCUMENT seen ...</testsuite>\n... @11:1)
    Nov 12 11:03:43 v-100003-svr01 com.hudson78: 12.11.2010 11:03:43 hudson.plugins.testng.parser.ResultPullParserHelper parseToTagIfFound
    Nov 12 11:03:43 v-100003-svr01 com.hudson78: WARNUNG: next() threw exception : already reached end of XML input (position: END_DOCUMENT seen ...</testsuite>\n... @20:1)
    Nov 12 11:03:43 v-100003-svr01 com.hudson78: 12.11.2010 11:03:43 hudson.plugins.testng.parser.ResultPullParserHelper parseToTagIfFound
    ...

    Is that a known bug or what could be the cause?

    1. Unknown User (farshid.ghods@gmail.com)

      that means the testng.xml file does not have a </testsuite> xml node. my assumption was that any testng.xml file will have a testsuite . I could be wrong . I will check with testng specs later.

      can you share couple of those files here ?

      1. With version 0.8 I had no more of those errors.

        But what is still a problem:

        • Looking at a job the link from the side menu, the central "TestNG Result" link and the trend links are correctly pointing at http://<server-address>/hudson/job/<jobname>/[<build>/]testngreports
        Now when I click on the "TestNG Result" link I get to a page where all links lack the "/hudson" part so all are resulting in a 404-error.

        • I have a test class that contains 5 methods. 4 of them are annotated with @Test(groups = { "group1" }) and one with @Test(groups = { "group2" }). Both groups are executed by TestNG and looking at the report files results for all 5 methods are written out correctly.
        On the package result page of the testng-plugin the class is listed twice under the topic "All Classes" with the correct duration and total method number for each. But the column "(diff)" after "Total" shows a "--3" for group2 which seems wrong as for that group nothing changed. The testng-plugin seems to mix both groups up. Also if I click on the class link I get the same page with the 4 methods of group1 no matter which of both links I choose though it should show either the methods of group1 or group2.

  4. Unknown User (scottsevertson)

    Plugin version 0.7 has a minor bug: The javascript method to show/hide results tables is defined with inconsistent case. Some define/use "showOrHideTable", some "showorHideTable". This may be resolved by editing the .jelly template files to use consistent case.

    1. Unknown User (farshid.ghods@gmail.com)

      Hi Scott,

      are you still facing the issue after upgrading to 0.8 ?

  5. Unknown User (scottsevertson)

    Also, version 0.7 appears to be missing an icon: Requests to /build/plugin/testng-plugin/icons/report.png results in a 404. To resolve, we placed an icon in Hudson's plugins directory, under plugins/testng-plugin/icons/report.png. We used Hudson's built-in clipboard icon, found in the Hudson install directory, under images/48x48/clipboard.gif, which we converted to a .png and renamed.

    1. Unknown User (farshid.ghods@gmail.com)

      sorry for the very late response. I just checked in the missing icon.

      I tried to release the newer version but still dont know how to release the plugin from github. working on it now :)

      1. Scott,

        I just released the 0.12 version with the icon fix.

        1. I can't get the plugin work with neither latest hudson nor latest jenkins. After installing the plugin and a program restart the plugin does not appear in the list of installed plugins.

          1. hmm.

            I didn't change its dependency list of its plugin id.

            I just created a new jenkins 1.397 and installed testng 0.12 and it worked fine.

            can you try removing it and then install it after restart

            1. Thanks for the tip. Manually removing the plugin and reinstalling it did the trick.

              Now if you could add the group name to the heading of the test methods (at least if a class contains more than one group) everything would be perfect :)

              1. Johann,

                sure.

                If you get a chance can you please file a bug here http://issues.jenkins-ci.org/browse/JENKINS/component/15801 and attach a teng-result.xml file which contains your tests with groups

                -Farshid

  6. Hi,

    I've installed newest version 0.14  and there is some issue with parametrized test. The reports doesn't take into consideration parameters and doesn't distinguish tests. 

    Each test is reported separately but link to details points always to the same test. Do you have such functionality on your  road-map?

    Regards,

    Pawel

    1. I think you are referring to this issue : http://issues.jenkins-ci.org/browse/JENKINS-8976

      I already checked in the fix but didn't get time to do more testing before releasing the new version.

  7. Does this work with the Maven job type?

    I've installed the plugin, checked the checkbox, next build, but cannot see any difference in the test results!?

    1. we based this plugin on the code from javancss plugin. So, even though the code to support maven builds is present, I haven't yet tested it.

      1. Possible that, because JUnit reports are active per default for Maven jobs, TestNG plugins results get overwritten or something like that?

      2. I've taken a look into the source and seen that the plugin only reacts to the org.codehaus.mojo.testng-plugin - which I've never heard of before.

        Maybe all that has to been changed, is that it acts on the surefire plugin?

        1. Christoph, which source are you talking about?

          This testng report plugin is not related to surefire plugin in any way. All we do is look for testng-results.xml file that is generated using org.testng.reporters.XmlReporter.

          1. I'm talking about this https://github.com/jenkinsci/testng-plugin-plugin/blob/master/src/main/java/hudson/plugins/testng/MavenPublisher.java

            As  the surefire plugin is perfectly capable of running TestNG tests, I wonder why this dependency to the TestNG report plugin is needed

            1. I can't promise when, but I'll go ahead and look into this soon. I'll probably just end up using testng project source itself :)

              1. This issue should be taken care of. I have removed all Maven related code. It wasn't needed in the first place.

  8. I can't get the plugin to work with matrix tasks. Is there a way to have an overview of all configuations that run? Currently i have to select specific configuraion to get to the results.

  9. Is there a way to modify the table in package result summar page which has methods by order of execution with duration n stuff ?

    Now, it has 

    Method

    Duration (ms)

    Start Time

    Status

      I also want it to show parameter that was passed to the method:

    Method

    Parameter

    Duration(ms)

    Start Time

    Status

    testMethod1

    apple

    5 mins

    Oct 11 15:02:47 GMT+00:00 2012

    PASS

    testMethod1

    orange

    10 mins

    Oct 11 15:07:50 GMT+00:00 2012

    PASS

  10. Hi,

    in Jenkins ver. 1.504 and later I dont get the "Publish TestNG Results" option in "Post Steps" section of the Jenkins Job configuration page. I have installed testng-plugin 1.1 version. Its listed in the installed plugins page. Do I need any other configurations setup for this plugin to be available for the use?

  11. Does the TestNG XML report pattern take in environment/gobal variables? If so, which syntax will work with the pattern?

    Can you explain more what these two checkbox does?
    -Escape Test description string?
    -Escape exception messages?
    The description given was not clear. Maybe an example would be helpful. Thanks!

    1. env variables are not supported. And, updated the page with information regarding the options.

      EDIT: env variables are now supported. (2/9/2014)

      1. How does the test description escaping work? I mean, you can't put real HTML inside the description attribute in XML (I've tried). Or does it escape already escaped HTML like "$lt;b> foo </b>" -> "foo"?

  12. I'm a new for this plug-in, I don't know how to use "TestNG XML report pattern", my jenkins tomcat and project tomcat are 

    deployed in linux system , but my test project is in windows system, and I use testng in my test project, I want to user 

    run a xml file to run my test classes, but I dont know how to run them automaticlly after jenkins build, can anybody help 

    me?

    1. Hi,

      this Plugin is not for running TestNG tests, but for publishing the results. Without a testng-results.xml file this plugin cannot work. For actually running your TestNG tests you need to write a command or an ANT task and execute that prior to publishing. Details on TestNG documentaion: http://testng.org/doc/documentation-main.html#running-testng .

      The "XML report pattern" you mention is the path to your TestNG test results, which usualy is called testng-results.xml. That file is created every time you run your tests through command line or ANT.

      I hope this helped you a bit
      Regards
      Hugo

  13. I think "Show Failed Builds?" should be enabled by default. Otherwise it's confusing seeing an all green chart although half of your builds is failed.

  14. Hi testng-plugin devs,

    I'm searching for a way to "claim" specific TestNG failed tests per build. I've found for now a "calim" jenkins plugin that allows me to "claim" failed builds but does not allow to claim failed TestNG tests.

    What is interestin, this "claim" plugin is able to claim unit tests as failed, and that option becomes available when adding post-build step for generate unit test report from xml.

    Can you introduce similar, per test claim, feature for TesnNG plugin in post build step?

    Thanks,

    Nemanja

  15. Hi, I would like to know if there's a way to get the diff value(s) from the results (like in environment variables)?

    My use case: I would like to send an email only if the diff value of Failed tests is not ±0 (using the Email-ext plugin and Flexible Publish Plugin).

    Thanks.

    1. Not possible currently.

  16. hi,

    Can anyone tell me how to compile and run the testng-plugin. I cloned the code from git but unable to compile and run it.

    Tried with commands "mvn hpi:run" and other compilation commands.

    - Varun

    1. Found the solution for the above, the dependency jar download repositories are not configured as part of the current pom.xml. Fixed and raised the following pull request for the same.

      https://github.com/jenkinsci/testng-plugin-plugin/pull/22

  17. Hello guys, nice job here but i've a question.

    With this plugin in the Jenkins report i can't access to my console log for each test (in the reports with JUnit plugin i can) which is very helpful to me and my developers to see errors. I don't know if i'm missing some configuration or something or it can't be done at all. 

    Thanks!

    1. It can't be done at all right now.

  18. I'm currently hitting this issue: https://issues.jenkins-ci.org/browse/JENKINS-26971

    Is there an ETA on this?  It's really affecting our process - I'll need to downgrade Jenkins to when this was working.  Or perhaps is there a work-around?

    Thanks.

  19. So I see that we can now configure a threshold for considering the builds unstable or even failure.  Thats great, but why is the default 100%?  You cant exceed 100% to get the build marked unstable so to me that means its basically disabling this and no matter what the testng results are then its always success.  Is there some global config somewhere so that I can set the default for  all new jobs kind of thing?  Basically all my jobs that were running just fine now show pass all the time even if the testng results have failures in it.  

    What am I missing because it seems hard to believe that you would release a change that defaults to show false passes....

    1. Hey James Theisen,

      This issue (JENKINS-34822) was fixed in 1.13. If you rollback the plugin to 1.10 (Before the feature was implemented) and then update to 1.13 maybe that will reset the defaults?

  20. Problem

    I am using this testng plugin with 'Flexi publish build' plugin to publish multiple testng reports on build page . However it seems testng plugin stores publish file at same location(under 'testng' folder) and overrides when doing multiple publish . 

    testng plugin stores publish file at following location

    /var/lib/jenkins/jobs/${mybuildName}/builds/21/testng/testng-results.xml

    Jenkins condole log for testng publish plugin

    Because of these I am getting only last file published multiple times on build page instead of multiple files published each time on build page

    Probable solution

    Can we provide 'Report Title' (likewise of HTML publish plugin) in testng plugin as well , so that it stores each publish file(in case of multiple publish) under its own location folder (location folder would be of same name as report title) (likewise of HTML publish plugin) ?

    So now publish file locations would be

    /var/lib/jenkins/jobs/${mybuildName}/builds/21/${myreportTitle1}/testng-results.xml
    /var/lib/jenkins/jobs/${mybuildName}/builds/21/${myreportTitle2}/testng-results.xml
    & so on ..

     

  21. Pipeline in Jenkinsfile - control the pass rate and pipeline status

    step([$class: 'Publisher', reportFilenamePattern: '**/surefire-reports/testng-results.xml',
    unstableSkips: 100,
    unstableFails: 100,
    failedSkips: 20,
    failedFails: 2,
    thresholdMode: 2,
    showFailedBuilds: true])

    -------------------------------------------------

    thresholdMode: 2 → persentage

    failedFails: 2 → allow 2 percent of tests to fail before failing the pipeline

    failedSkips: 20 → allow 20 percent of tests to be skipped before failing the pipeline

     

    same regarding the unstable → will mark the build as unstable