Test results
- OK - test passed without any problem
- WARN - there was some problem, functionality doesn't do what is expected, but it doesn't cause failure of whole Jenkins or any other significant problems
- FAIL - cause failure of whole Jenkins or similar critical issue which affects all jobs (e.g. when plugin is installed, it's not possible to start Jenkins or save configuration of any job)
Acceptance tests
Most of the test has bean automates as acceptance tests. Instead of retesting manually, feel free to run the suite and report the results here.
Environment |
LTS RC / result |
Tested by |
---|---|---|
|
|
|
Installation, upgrade
Test name |
Short description |
Expected result |
Notes |
LTS RC / result |
Tested by |
---|---|---|---|---|---|
Install from RPM |
Install Jenkins from RPM package |
Jenkins is installed without error and runs |
FC 19, RHEL 6.4, openJDK7 |
|
|
Install from .deb |
Install Jenkins from Debian package |
Jenkins is installed without error and runs |
Ubuntu 14.04, OpenJDK 7 |
|
|
Install from war on Tomcat |
Install Jenkins from war by deploying on Tomcat |
Jenkins is deployed and runs |
Arch, openJDK7, Tomcat 7 |
|
|
JBoss AS |
Install Jenkins from war by deploying on JBoss AS |
Jenkins is deployed and runs |
Arch, openJDK7, JBoss AS 7 |
|
|
Core functionality
Freestyle jobs
Test name |
Short description |
Expected result |
Notes |
LTS RC / result |
Tested by |
---|---|---|---|---|---|
Batch build step |
Add a Windows batch file build step |
Batch build steps are executed successfully |
|
OK / 1.580.1 |
ogondza |
Plug-ins
Green balls
Test name |
Short description |
Expected result |
Notes |
LTS RC / result |
Tested by |
---|---|---|---|---|---|
Success results |
Success results |
Success results are marked as green |
Arch / openJDK7 |
OK / 1.580.1 |
ogondza |
Sonar
Test name |
Short description |
Expected result |
Notes |
LTS RC / result |
Tested by |
---|---|---|---|---|---|
Sonar analysis |
Setup sonar analysis |
Analysis is executed and published on sonar server |
|
|
|
Analysis collector
Test name |
Short description |
Expected result |
Notes |
LTS RC / result |
Tested by |
---|---|---|---|---|---|
Collects and show results |
Setup collection of checkstyle, findbugs, pdm and warnings results collection |
Results are collected and shown |
|
|
|
Jira
Test name |
Short description |
Expected result |
Notes |
LTS RC / result |
Tested by |
---|---|---|---|---|---|
Update JIRA issue |
Update JIRA issue which is related to changes contained in the build |
JIRA is updated and on build page is a link to appropriate JIRA |
automated - run manually |
OK / 1.580.1 |
ogondza |
Deploy
Test name |
Short description |
Expected result |
Notes |
LTS RC / result |
Tested by |
---|---|---|---|---|---|
|
|
|
|
|
|
Deploy on Tomcat 7 |
Build war app and deploy it on Tomcat 7 |
App is successfully deployed and running |
automated - run manually |
OK / 1.580.1 |
ogondza |
Dry
Test name |
Short description |
Expected result |
Notes |
LTS RC / result |
Tested by |
---|---|---|---|---|---|
Publish DRY analysis results |
Setup CPD analysis and publish analysis results |
Results are published |
|
|
|
Backup
Test name |
Short description |
Expected result |
Notes |
LTS RC / result |
Tested by |
---|---|---|---|---|---|
Backup Jenkins configuration |
Setup backup dir and run Jenkins backup |
Once no builds run, backup is performed |
|
|
|
Scp publisher
Test name |
Short description |
Expected result |
Notes |
LTS RC / result |
Tested by |
---|---|---|---|---|---|
Publish artifact over scp |
Setup remote destination and artifact to be archived over scp |
Artifact is archived on remote machine in speficied dir |
automated - run manually |
OK / 1.580.1 |
ogondza |
IRC
Test name |
Short description |
Expected result |
Notes |
LTS RC / result |
Tested by |
---|---|---|---|---|---|
Send IRC notification |
Setup IRC notification and schedule a build |
Notification is send to appropriate IRC channel |
|
OK / 1.580.1 |
ogondza |
Run jenkins bot command |
Run bot command on IRC channel |
Command is executed |
|
|
|
LDAP
Test name |
Short description |
Expected result |
Notes |
LTS RC / result |
Tested by |
---|---|---|---|---|---|
LDAP test |
LdapPluginTest in test-harness |
All tests succeed |
automated - run manually |
OK / 1.580.1 |
ogondza |
LDAP NoDocker Test |
LdapPluginNoDockerTest in test-harness |
All tests succeed |
automated |
OK / 1.580.2 |
scoheb |
Publish Over FTP
Test name |
Short description |
Expected result |
Notes |
LTS RC / result |
Tested by |
---|---|---|---|---|---|
Ftp test |
Ftp testin test-harness |
All tests succeed |
automated - run manually |
OK / 1.580.1 |
ogondza |
Subversion
Test name |
Short description |
Expected result |
Notes |
LTS RC / result |
Tested by |
---|---|---|---|---|---|
Subversion test |
SubversionPluginTest && Subversion_Version154_PluginTest in test-harness |
All tests succeed |
automated - run manually |
OK / 1.580.1 |
ogondza |
Active Directory
Test name |
Short description |
Expected result |
Notes |
LTS RC / result |
Tested by |
---|---|---|---|---|---|
ActiveDirectory test |
ActiveDirectoryTest in test-harness |
All tests succeed |
automated |
OK / 1.580.1 |
scoheb |
Gerrit Trigger
Test name |
Short description |
Expected result |
Notes |
LTS RC / result |
Tested by |
---|---|---|---|---|---|
GerritTrigger test |
GerritTriggerTest in test-harness |
All tests succeed |
automated |
OK / 1.580.1 |
scoheb |
Exploratory Tests
Test name |
Short description |
Expected result |
Notes |
LTS RC / result |
Tested by |
---|---|---|---|---|---|
Stress/Load Test |
Run Xenu link checker while other interactive tests are running, and while jobs are executing |
Performance slower, but still functional and reliable, memory growth reasonable |
|
|
|
Git plugin |
Explore multiple configurations of the Git plugin, including JGit and CliGit implementations, using ssh, https, and git protocol repositories. |
Reasonable behavior, no detected Jenkins related regressions |
|
|
|
Task Scanner plugin |
Explore different configurations of the plugin, including file name patterns, exclude patterns, different task tags, status thresholds, different source file sets inside the workspace. Job has run on master and slave. |
Reasonable behavior, graphical and textual results represent actual occurrences of tags, build status according to threshold settings, no detected Jenkins related regressions |
|
|
|