Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Migrated to Confluence 5.3
Wiki Markup
{hudsonjenkins-plugin-info:pluginId=testopia}

Excerpt

This plug-in integrates Jenkins and with Testopia and generates reports on automated test execution. With this plug-in you can manage your tests in Testopia, schedule and control in Jenkins, and execute using your favorite test execution tool (TestPartner, Selenium, TestNG, Perl modules, prove, PHPUnit, among others).

Note

When filing an issue, remember to include as much data as possible, but avoid including sensitive data, as attachments can take a while to be deleted from JIRA, as the process is manual and quite troublesome (smile)

...

The Testopia plug-in integrates Jenkins with Testopia. Testopia plug-in uses testopia-java-apidriver to access Testopia XML-RPC API. With the information that you provide in the build step configuration the plug-in retrieves automated tests from Testopia. With the plug-in, you are able to execute build steps that call testing tools. It reads TestNGJUnit and TAP test report formats, used to update Testopia test cases' executions.

Tip

The job configuration changed! Now there are three sections, what helps you to configure your job, and add extra build steps, result seeking strategies and general information about Testopioa.

Testopia configuration section

Image RemovedImage Added

Test Execution section

Image RemovedImage AddedAnd

Result Seeking Strategy section

TODOImage Added

Tip

You can use environment variables in Test project name, Test plan name or Build name. For instance, build-project-$BUILD_NUMBER would be replaced by build-project- and the number of the Build executed in Jenkins.

How does the plug-in execute my automated tests?

...

For instance, you could run a ant command passing a Test Class as parameter, more or less something like this, /opt/java/apache-ant-1.8.0/bin/ant -DtestClass=$TESTOPIA_TESTCASE_TEST_CLASSSCRIPT. In this case, Test Case is the name of a custom TESTOPIA_TESTCASE_SCRIPT the value of the script automation field in Testopia.

List of environment variables available during Testopia Plug-in execution:

Already implemented:

...

Test run info

...

  • TESTOPIA_TESTRUN_ID
  • TESTOPIA_TESTRUN_BUILD
  • TESTOPIA_TESTRUN_ENVIRONMENT
  • TESTOPIA_TESTRUN_MANAGER
  • TESTOPIA_TESTRUN_NOTES
  • TESTOPIA_TESTRUN_PRODUCT_VERSION
  • TESTOPIA_TESTRUN_SUMMARY
  • TESTOPIA_TESTRUN_CASES
  • TESTOPIA_TESTRUN_PLAN_ID
  • TESTOPIA_TESTRUN_PLAN_TEXT_VERSION
  • TESTOPIA_TESTRUN_RUN_ID
  • TESTOPIA_TESTRUN_STATUS
  • TESTOPIA_TESTRUN_TARGET_COMPLETION
  • TESTOPIA_TESTRUN_PLAN_TARGET_PASS

...

Test case info

...

  • TESTOPIA_TESTCASE_ID
  • TESTOPIA_TESTCASE_SCRIPT

...

  • RUN_ID
  • TESTOPIA_TESTCASE_BUILD_ID
  • TESTOPIA_TESTCASE_SCRIPT
  • TESTOPIA_TESTCASE_ALIAS
  • TESTOPIA_TESTCASE_AUTHORARGUMENTS
  • TESTOPIA_TESTCASE_REQUIREMENT
  • TESTOPIA_TESTCASE_SORT_KEY
  • TESTOPIA_TESTCASE_SUMMARY
  • TESTOPIA_BUILDTESTCASE_AUTHOR_ID
  • TESTOPIA_TESTPLANTESTCASE_CATEGORY_ID
  • TESTOPIA_ENVIRONMENT

For each Custom Field found, Testopia Plug-in defines an environment variable for with its name and value, as shown above (%CUSTOM_FIELD_NAME%). If the custom field name contains spaces the plug-in will convert them into _'s. i.e., Test Class can be accessed as %TESTOPIA_TESTCASE_TEST_CLASS% in Windows or $TESTOPIA_TESTCASE_TEST_CLASS in *nix-like systems.

  • TESTCASE_DEFAULT_TESTER_ID
  • TESTOPIA_TESTCASE_PRIORITY_ID
  • TESTOPIA_TESTCASE_STATUS_ID
  • TESTOPIA_TESTCASE_AUTOMATED
  • TESTOPIA_TESTCASE_CREATION_DATE
  • TESTOPIA_TESTCASE_ESTIMATED_TIME
  • TESTOPIA_TESTCASE_ESTIMATED_ENV_ID

How does the plug-in know if a test passed or failed?

You can choose among different result seeking strategies in your job configuration. The plug-in supports will support three result formatsTestNGJUnit and TAP

...

TestNG result seeking strategies

...

TestNG class name: The plug-in matches the TestNG class name (e.g.: org.tap4j.TestParser) and the key custom field value.

TestNG method name: The plug-in matches the TestNG method name (e.g.: org.tap4j.TestParser#testTokenizer) and the key custom field value.

TestNG suite name: The plug-in matches the TestNG suite name (e.g.: functional tests) and the key custom field value.

...

JUnit result seeking strategies

...

JUnit case class Already implemented:

...

TAP result seeking strategies

...

TAP file name: The plug-in matches the JUnit case class TAP file name (e.g.: orgtestFtpProtocol.tap4j.TestParsertap) and the key custom field value.

JUnit case name: The plug-in matches the JUnit case class name (e.g.: testTokenizer) and the key custom field value.

JUnit method name: The plug-in matches the JUnit case class name (e.g.: org.tap4j.TestParser#testTokenizer) and the key custom field value.

JUnit suite name: The plug-in matches the JUnit suite name (e.g.:functional tests) and the key custom field value.

...

TAP result seeking strategies

...

Does the plugin record the test results in Testopia?

Yes, the plugin will update the testcase in Testopia and set the status according to the result of the testcase.

Configuration

  1. Download and install the latest version of Testopia (http://www.mozilla.org/projects/testopia).
  2. Create automated tests in Testopia (Check automated field).
  3. Install the Testopia plug-in from the Jenkins Plugin Manager
  4. Define a Testopia configuration from the Configure System page.
  5. Add a Invoke Testopia build step in the job you want execute automated tests.
  6. Configure the required properties.

...

Want to see this plug-in in your language? Send us an e-mail and we will get in touch with the text that needs to be translated.

Feedback

Wiki Markup
Your feedback is very welcomed, but you don't need donate money. You can drop a message in \[http://www.tupilabs.com\] with suggestions and ideas (or even patches or pull requests).

Compatibility Matrix

The plug-in 1.0 and 1.1 versions were pilot projects that should be ignored. They have issues in distributed environments and depend on Maven. Please, consider updating your plug-in to the 2.0 version.

Note

We don't maintain the plug-in for Hudson. Issues happening in Hudson may not be fixed here.

Plugin

1.0

1.1

 

 

 

1.2

1.3 

Testopia

2.5+

 

 

  2.5+

2.5+

2.5+

Jenkins

1.7424+

 

 

 

 

 

 

 

 

1.424+

1.424+

1.424+

Issues

Jira Issues
renderModecachedynamicoff
width800px850px
anonymoustrue
titleJIRA Issues
urlhttp://issues.jenkins-ci.org/sr/jira.issueviews:searchrequest-xml/temp/SearchRequest.xml?jqlQuery=project+%3D+JENKINS+AND+status+in+%28Open%2C+%22In+Progress%22%2C+Reopened%29+AND+component+%3D+%27testopia%27%27testopia-plugin%27&tempMax=1000
height300px

Resources

Testopia Plug-in used TestLink Plug-in as basis. Below you will find resources for Testopia and TestLink plug-ins, as the concept in both plug-ins is very similar.

  1. Article published in Testing Experience magazine issue number 12 (Open Source Tools) written by Bruno P. Kinoshita and Anderson dos Santos. 2010.
  2. Slides used in the lighting talk presented at Encontro Ágil 2010 in IME-USP (Portuguese Only). 2010.
  3. Article published in Automated Software Testing Magazine, volume 3, issue 1. March, 2011.
  4. Slides used in the presentation done in Belgium Testing Days 2011.
  5. Lighting talk for Encontro Ágil 2010, at IME-USP.
  6. More articles and tips on Bruno P. Kinoshita's website.
  7. Sponsor company (until May 2011): Sysmap Solutions - Brazil.
  8. Sponsor company: TupiLabs - Brazil
  9. Presentation at STPCon Spring March 2012 in New Orleans - USA.

Sponsors

Image Removed

  1. Automated Testing Environment by Bugzilla, Testopia and Jenkins by Chin Pin Chang. July, 2014.

Documentation

  1. Various Testopia documentation assembled on Storify
  2. How to handle self signed certificates on Jenkins Server (TestLink Plugin SSL support)

Sponsors

Image Added

Image Added

Release Notes

Release 1.

...

3
  1. JUnit and TestNG result seeking strategies
Release 1.0
  1. First version of the plug-in (the development was guided based on the source code of the following plug-ins: TestLink  CCM , Sonar (how to ref maven installations)).

...

  1. Keep compatibility with latest versions of Jenkins and Testopia.

Open source licenses donated for this project

Image Removed

...

Troubleshooting

  • A test case without a tester assigned doesn't execute.