Skip to end of metadata
Go to start of metadata

Plugin Information

View multi-module-tests-publisher on the plugin site for more information.

This plugin stores junit results into an embedded derby database and groups testcases by module,package,class and case

This plugin can be used as a replacement for the built-in junit plugin.

This plugin allows junit tests to be grouped by suites in addition to the being grouped by package, class and case.

It persists the test into a derby database that runs in embedded mode. This is done so that reports can be generated easily using sql queries. This also reduces the memory required to display reports and history

The trend graph for each suite is displayed on the project home page. Three new columns, Success Rate, Failure Rate and Skip Rate are available that displays the percentage rate of success, failure and skip at the project level, suite level, package level, test class level and case level

The screenshot of the configuration page is shown below

19 Comments

  1. Hi, would it be possible to add some more info? The screenshots looks a bit outdated; in fact there is now an additional string (at least I guess) parameter "module names". It seems it's mandatory: if I leave it empty then the test result doesn't show any grouping, not even the package one, which is standard for the Junit Test Publisher plugin.

    Thx!

    1. Hi Rick,

      The module names parameter is a comma separated list of of module names that the junit tests have to be grouped into. I think the issue you are facing is a bug. I will get it fixed and update the wiki soon. As I haven't seen a lot of usage for this plugin by users I haven't kept things up to date. I am curious to your use-case for this plugin?

      1. Hi Vimil,

        Thx a lot :)

        Actually I was just taking a look at the functionality and hoping to have some nice graphs overview of the test results. In my case the junit xml files come out of a maven build, and it would be nice to group the results by each maven project module.

  2. Vimil,

    I too am interested in this plugin, but am not able to get the graphs to show up.  Can you provide a better explanation of how the module names parameter relates to the classname/name of the reported junit tests?

    1. The module names are a comma separated names of suites. So if you have a collection of junit.xml files one for each module then modulenames field is the comma separated names of the value of name attribute of the testsuite tag.

      Ex:  If there are two junit xml files with their testssuite tag having the following values for name attribute

       <testsuite  name="Agent" ...

       <testsuite  name="Core" ...

      then moduleNames field value should be set to 'Agent,Core'

      Let me know if this works for you.

    2. The module names are a comma separated names of suites. So if you have a collection of junit.xml files one for each module then modulenames field is the comma separated names of the value of name attribute of the testsuite tag.

      Ex:  If there are two junit xml files with their testssuite tag having the following values for name attribute

       <testsuite  name="Agent" ...

       <testsuite  name="Core" ...

      then moduleNames field value should be set to 'Agent,Core'

      Let me know if this works for you.

      1. Vimil, that is more clear yes.

        I can now see the graphs.

        However, I'm noticing that the hyperlinks from the Graphs aren't working for me.  They are linking to:

        <ROOT_URL>/testReport/<build_number>/<module_name>/

        However I get a 404 for that URL.  Any ideas?

        1. ROOT_URL doesn't include "job/<job_name>" in the links I get, which is probably why it's not working.

  3. Vimil,

    I'd like to use your plugin, and integrate it's results into my jelly HTML template.  Can you describe how to modify the template here to work with your plugin results?  (I'm guessing your plugin wasn't meant to be setup alongside the standard Junit report publishing, because I get the standard report page when that happens, rather than your custom report page broken down by module.

    Here is the source of the template I'm using, that needs to get modified to consume the results of your plugin.  See the Email-ext plugin for details.

    <!-- JUnit TEMPLATE -->
    
    <j:set var="junitResultList" value="${it.JUnitTestResult}" />
    <j:if test="${junitResultList.isEmpty()!=true}">
      <TABLE width="100%">
        <TR><TD class="bg1" colspan="2"><B>JUnit Tests</B></TD></TR>
        <j:forEach var="junitResult" items="${it.JUnitTestResult}">
          <j:forEach var="packageResult" items="${junitResult.getChildren()}">
            <TR><TD class="bg2" colspan="2"> Name: ${packageResult.getName()} Failed: ${packageResult.getFailCount()} test(s), Passed: ${packageResult.getPassCount()} test(s), Skipped: ${packageResult.getSkipCount()} test(s), Total: ${packageResult.getPassCount()+packageResult.getFailCount()+packageResult.getSkipCount()} test(s)</TD></TR>
            <j:forEach var="failed_test" items="${packageResult.getFailedTests()}">
              <TR bgcolor="white"><TD class="test_failed" colspan="2"><B><li>Failed: ${failed_test.getFullName()} </li></B></TD></TR>
            </j:forEach>
          </j:forEach>
        </j:forEach>
      </TABLE>
    <BR/>
    </j:if>
  4. Sorry for the flurry of questions...

    Is there a way to show the overall Test Result Trend Graph along with the graphs broken down by Module?

  5. This is pretty good plugin. yet i'm having the same problem as Justin that i'm not able to view the test results per module, and i'm not able to see the test failures although i can see them if i used normal JUnit.

    I'm also facing other issue that it adds an <init> class, is that normal?

    I'm trying to launch one project with different set of tests and each time i'd like to see the chart per this set of tests only, without having the aggregated graph aggregating different testsuites into one graph and age of failures changes also with each run with normal Junit publishing.

  6. Hi!

    I don't seem to get any test result and therefore not any graphs.

    In the job console output i see 

    ...Analyzing test results with ProjectResultPublisher...
    Test results parsed....Finished: SUCCESS

    I tried a two very simple test xmls (seen below), I also tried with one xml file with the two test suites within it.

    In the post build action i added

    under Junit Report File Mask: test1.xml, test2.xml

    under Module names: Suite1,Suite2

    The same setup works with the ordinary JUnit plugin, but then i don't have the trend graph per suite which is what I want.

    Running jenkins version 1.559

    Any idea why it is not working?

    <?xml version="1.0"?>

    <testsuites>

      <testsuite errors="0" failures="0" name="Suite1" skipped="2" tests="5">

        <testcase classname="testS1.UNSUPPORTED" name="test1">

          <skipped message="test1" />

        </testcase>

        <testcase classname="testS1.UNSUPPORTED" name="test2">

          <skipped message="test2" />

        </testcase>

        <testcase classname="testS1.PASS" name="test3" />

        <testcase classname="testS1.PASS" name="test4" />

        <testcase classname="testS1.PASS" name="test5" />

      </testsuite>

    </testsuites>

    ..

    ..

    <?xml version="1.0"?>

    <testsuites>

      <testsuite errors="0" failures="0" name="Suite2" skipped="2" tests="5">

        <testcase classname="testS2.UNSUPPORTED" name="test1">

          <skipped message="test1" />

        </testcase>

        <testcase classname="testS2.UNSUPPORTED" name="test2">

          <skipped message="test2" />

        </testcase>

        <testcase classname="testS2.PASS" name="test3" />

        <testcase classname="testS2.PASS" name="test4" />

        <testcase classname="testS2.PASS" name="test5" />

      </testsuite>

    </testsuites>

    Regards Mr.T

    1. Tomas, did you solve this problem?

      I have exactly the same result.

      1. Hi Alexei!

        unfortunately not =(. 
        I could attack my particular problem in another way.
        Which was that a made a script and a particular blacklist file to exclude certain test(s), which where know to fail, from the resulting xml file and the graph
        But my use case is probably different than yours.
        ///Mr.T

  7. Hi Vimil et al.:

    I am seeing the following issue when trying to use the plugin. Can you help me make sense of it and get through the issue.

    Jenkins Plugin Error

    ERROR: Publisher com.cwctravel.hudson.plugins.multimoduletests.ProjectResultPublisher aborted due to exception
    java.lang.NegativeArraySizeException
    at com.cwctravel.hudson.plugins.multimoduletests.junit.io.StringReaderWriter.<init>(StringReaderWriter.java:17)
    at com.cwctravel.hudson.plugins.multimoduletests.junit.io.IOUtil.createReaderWriter(IOUtil.java:11)
    at com.cwctravel.hudson.plugins.multimoduletests.junit.JUnitParser.characters(JUnitParser.java:266)
    at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.characters(AbstractSAXParser.java:538)
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:464)
    at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:808)
    at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
    at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:119)
    at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1205)
    at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:522)
    at com.cwctravel.hudson.plugins.multimoduletests.junit.JUnitParser.parse(JUnitParser.java:119)
    at com.cwctravel.hudson.plugins.multimoduletests.ProjectResultPublisher$ParseResultCallable.invoke(ProjectResultPublisher.java:240)
    at com.cwctravel.hudson.plugins.multimoduletests.ProjectResultPublisher$ParseResultCallable.invoke(ProjectResultPublisher.java:174)
    at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2462)
    at hudson.remoting.UserRequest.perform(UserRequest.java:118)
    at hudson.remoting.UserRequest.perform(UserRequest.java:48)
    at hudson.remoting.Request$2.run(Request.java:328)
    at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
    at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
    at java.util.concurrent.FutureTask.run(FutureTask.java:138)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
    at java.lang.Thread.run(Thread.java:662)

    1. @all, this only happens when i try to use both the Jenkins and the multi-test plugin at the same time. When I use the multi-test plugin alone, I do not see the ERROR but I then I see the same issue as @Tomas S above where I dont see any trend graphs.

  8. Hi I have an error that seems to be from DerbyDB:

    Analyzing test results with ProjectResultPublisher...
    16:53:45 ERROR: Publisher com.cwctravel.hudson.plugins.multimoduletests.ProjectResultPublisher aborted due to exception
    16:53:45 java.io.IOException: java.sql.SQLException: Failed to create database '/root/projects_hudson/_hudson_home/jobs/Project/JUnitDB', see the next exception for details.
    16:53:45 at com.cwctravel.hudson.plugins.multimoduletests.ProjectResultPublisher$ParseResultCallable.invoke(ProjectResultPublisher.java:259)
    16:53:45 at com.cwctravel.hudson.plugins.multimoduletests.ProjectResultPublisher$ParseResultCallable.invoke(ProjectResultPublisher.java:174)
    16:53:45 at hudson.FilePath.act(FilePath.java:981)
    16:53:45 at hudson.FilePath.act(FilePath.java:959)
    16:53:45 at com.cwctravel.hudson.plugins.multimoduletests.ProjectResultPublisher.perform(ProjectResultPublisher.java:122)
    16:53:45 at hudson.tasks.BuildStepMonitor$3.perform(BuildStepMonitor.java:45)
    16:53:45 at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:770)
    16:53:45 at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:734)
    16:53:45 at hudson.model.Build$BuildExecution.post2(Build.java:183)
    16:53:45 at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:683)
    16:53:45 at hudson.model.Run.execute(Run.java:1784)
    16:53:45 at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
    16:53:45 at hudson.model.ResourceController.execute(ResourceController.java:89)
    16:53:45 at hudson.model.Executor.run(Executor.java:240)
    16:53:45 Caused by: java.sql.SQLException: Failed to create database '/root/projects_hudson/_hudson_home/jobs/Project/JUnitDB', see the next exception for details.
    16:53:45 at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source)
    16:53:45 at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknown Source)
    16:53:45 at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
    16:53:45 at org.apache.derby.impl.jdbc.EmbedConnection.createDatabase(Unknown Source)
    16:53:45 at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
    16:53:45 at org.apache.derby.impl.jdbc.EmbedConnection30.<init>(Unknown Source)
    16:53:45 at org.apache.derby.impl.jdbc.EmbedConnection40.<init>(Unknown Source)
    16:53:45 at org.apache.derby.jdbc.Driver40.getNewEmbedConnection(Unknown Source)
    16:53:45 at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
    16:53:45 at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
    16:53:45 at java.sql.DriverManager.getConnection(DriverManager.java:664)
    16:53:45 at java.sql.DriverManager.getConnection(DriverManager.java:270)
    16:53:45 at com.cwctravel.hudson.plugins.multimoduletests.junit.db.JUnitDB.getConnection(JUnitDB.java:661)
    16:53:45 at com.cwctravel.hudson.plugins.multimoduletests.junit.db.JUnitDB.initDB(JUnitDB.java:667)
    16:53:45 at com.cwctravel.hudson.plugins.multimoduletests.junit.db.JUnitDB.<init>(JUnitDB.java:768)
    16:53:45 at com.cwctravel.hudson.plugins.multimoduletests.ProjectResultPublisher$ParseResultCallable.invoke(ProjectResultPublisher.java:231)
    16:53:45 ... 13 more
    16:53:45 Caused by: java.sql.SQLException: Failed to create database '/root/projects_hudson/_hudson_home/jobs/Project/JUnitDB', see the next exception for details.
    16:53:45 at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
    16:53:45 at org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown Source)
    16:53:45 ... 29 more
    16:53:45 Caused by: java.sql.SQLException: Supplied territory description 'english' is invalid, expecting ln[_CO_variant]
    16:53:45 ln=lower-case two-letter ISO-639 language code, CO=upper-case two-letter ISO-3166 country codes, see java.util.Locale.
    16:53:45 at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
    16:53:45 at org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown Source)
    16:53:45 at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source)
    16:53:45 at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source)
    16:53:45 at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown Source)
    16:53:45 at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown Source)
    16:53:45 at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown Source)
    16:53:45 ... 26 more
    16:53:45 Caused by: ERROR XBM0X: Supplied territory description 'english' is invalid, expecting ln[_CO_variant]
    16:53:45 ln=lower-case two-letter ISO-639 language code, CO=upper-case two-letter ISO-3166 country codes, see java.util.Locale.
    16:53:45 at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
    16:53:45 at org.apache.derby.impl.services.monitor.BaseMonitor.staticGetLocaleFromString(Unknown Source)
    16:53:45 at org.apache.derby.impl.services.monitor.BaseMonitor.setLocale(Unknown Source)
    16:53:45 at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
    16:53:45 at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
    16:53:45 at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
    16:53:45 at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
    16:53:45 at org.apache.derby.impl.services.monitor.BaseMonitor.createPersistentService(Unknown Source)
    16:53:45 at org.apache.derby.iapi.services.monitor.Monitor.createPersistentService(Unknown Source)
    16:53:45 ... 26 more

    It seems the database cannot run because it gets it's Locale with an unexpected territory.
    The Jenkins I use is v1.588, it runs on centos 7, but I don't know exactly what java is used. The tomcat jenkins uses has java 1.8, the project is compiled using jdk1.7.0_21(using ant), but I am not certain what java version is used by the job itself.

  9. Am getting a similar "failed to create database" problem. From Jenkins 1.616 running on a RedHat Linux 5.4. From stack trace looks like may not be able to create directory for database:Caused by: ERROR XBM0H: Directory /var/lib/jenkins/jobs/VL_P614/JUnitDB cannot be created.

  10. I am getting a strange UnsatisfiedLinkError using Jenkins 1.656 on Windows:

    Any Help appreciated!

    12:14:11 Analyzing test results with ProjectResultPublisher...
    12:14:13 ERROR: Step ‘Publish JUnit test result report grouped by module’ aborted due to exception: 
    12:14:13 java.lang.UnsatisfiedLinkError: no management in java.library.path
    12:14:13 at java.lang.ClassLoader.loadLibrary(Unknown Source)
    12:14:13 at java.lang.Runtime.loadLibrary0(Unknown Source)
    12:14:13 at java.lang.System.loadLibrary(Unknown Source)
    12:14:13 at sun.management.Flag$1.run(Unknown Source)
    12:14:13 at sun.management.Flag$1.run(Unknown Source)
    12:14:13 at java.security.AccessController.doPrivileged(Native Method)
    12:14:13 at sun.management.Flag.<clinit>(Unknown Source)
    12:14:13 at sun.management.ExtendedPlatformComponent.isUnlockCommercialFeaturesEnabled(Unknown Source)
    12:14:13 at sun.management.ExtendedPlatformComponent.shouldRegisterCMMBean(Unknown Source)
    12:14:13 at sun.management.ExtendedPlatformComponent.getMXBeans(Unknown Source)
    12:14:13 at java.lang.management.ManagementFactory.getPlatformMBeanServer(Unknown Source)
    12:14:13 at org.apache.derby.impl.services.jmx.JMXManagementService$1.run(Unknown Source)
    12:14:13 at org.apache.derby.impl.services.jmx.JMXManagementService$1.run(Unknown Source)
    12:14:13 at java.security.AccessController.doPrivileged(Native Method)
    12:14:13 at org.apache.derby.impl.services.jmx.JMXManagementService.findServer(Unknown Source)
    12:14:13 at org.apache.derby.impl.services.jmx.JMXManagementService.boot(Unknown Source)
    12:14:13 at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
    12:14:13 at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
    12:14:13 at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
    12:14:13 at org.apache.derby.iapi.services.monitor.Monitor.startSystemModule(Unknown Source)
    12:14:13 at org.apache.derby.impl.services.monitor.BaseMonitor.runWithState(Unknown Source)
    12:14:13 at org.apache.derby.impl.services.monitor.FileMonitor.<init>(Unknown Source)
    12:14:13 at org.apache.derby.iapi.services.monitor.Monitor.startMonitor(Unknown Source)
    12:14:13 at org.apache.derby.iapi.jdbc.JDBCBoot.boot(Unknown Source)
    12:14:13 at org.apache.derby.jdbc.EmbeddedDriver.boot(Unknown Source)
    12:14:13 at org.apache.derby.jdbc.EmbeddedDriver.<clinit>(Unknown Source)
    12:14:13 at org.apache.derby.jdbc.AutoloadedDriver.getDriverModule(Unknown Source)
    12:14:13 at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
    12:14:13 at java.sql.DriverManager.getConnection(Unknown Source)
    12:14:13 at java.sql.DriverManager.getConnection(Unknown Source)
    12:14:13 at com.cwctravel.hudson.plugins.multimoduletests.junit.db.JUnitDB.getConnection(JUnitDB.java:661)
    12:14:13 at com.cwctravel.hudson.plugins.multimoduletests.junit.db.JUnitDB.initDB(JUnitDB.java:667)
    12:14:13 at com.cwctravel.hudson.plugins.multimoduletests.junit.db.JUnitDB.<init>(JUnitDB.java:768)
    12:14:13 at com.cwctravel.hudson.plugins.multimoduletests.ProjectResultPublisher.perform(ProjectResultPublisher.java:121)
    12:14:13 at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
    12:14:13 at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:782)
    12:14:13 at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:723)
    12:14:13 at hudson.model.Build$BuildExecution.post2(Build.java:185)
    12:14:13 at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:668)
    12:14:13 at hudson.model.Run.execute(Run.java:1763)
    12:14:13 at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
    12:14:13 at hudson.model.ResourceController.execute(ResourceController.java:98)
    12:14:13 at hudson.model.Executor.run(Executor.java:410)