Created by Unknown User (olivergondza), last modified on Jun 25, 2014

Manual Tests (LTS 1.554.x)

Test results

  • OK - test passed without any problem
  • WARN - there was some problem, functionality doesn't do what is expected, but it doesn't cause failure of whole Jenkins or any other significant problems
  • FAIL - cause failure of whole Jenkins or similar critical issue which affects all jobs (e.g. when plugin is installed, it's not possible to start Jenkins or save configuration of any job)

Acceptance tests

Most of the test has bean automates as acceptance tests. Instead of retesting manually, feel free to run the suite and report the results here.

Environment

LTS RC / result

Tested by

 

 

 

Installation, upgrade

Test name

Short description

Expected result

Notes

LTS RC / result

Tested by

Install from RPM

Install Jenkins from RPM package

Jenkins is installed without error and runs

FC 19, RHEL 6.4, openJDK7


 

Install from .deb

Install Jenkins from Debian package

Jenkins is installed without error and runs

Debian Testing x64, Oracle JDK 8
Debian 6 x86, Oracle JDK 7
Debian 7 x64, Oracle JDK 7
Debian 6 x64, Oracle JDK 7

1.554.1 / OK

mwaite

Install from war on Tomcat

Install Jenkins from war by deploying on Tomcat

Jenkins is deployed and runs

Arch, openJDK7, Tomcat 7



Arch, openJDK7, Tomcat 6

1.554.1 / OK
1.554.2 / OK
1.554.3 / OK

1.554.1 / OK
1.554.2 / OK
1.554.3 / OK

ogondza
ogondza
ogondza

hiteswar
ogondza
ogondza

JBoss AS

Install Jenkins from war by deploying on JBoss AS

Jenkins is deployed and runs

Arch, openJDK7, JBoss AS 7

1.554.1 / OK
1.554.2 / OK 
1.554.3 / OK 

ogondza
ogondza
ogondza

Core functionality 

Freestyle jobs 

Test name

Short description

Expected result

Notes

LTS RC / result

Tested by

Batch build step

Add a Windows batch file build step

Batch build steps are executed successfully

 

1.554.1 / OK
1.554.2 / OK
1.554.3 / OK 

ogondza
ogondza
ogondza

Plug-ins 
Green balls (1.13) 

Test name

Short description

Expected result

Notes

LTS RC / result

Tested by

Success results

Success results

Success results are marked as green

Arch / openJDK7

1.554.1 / OK
1.554.2 / OK
1.554.3 / OK 

ogondza
ogondza
ogondza

Sonar (2.0.1, 2.1) 

Test name

Short description

Expected result

Notes

LTS RC / result

Tested by

Sonar analysis

Setup sonar analysis

Analysis is executed and published on sonar server

 

 

 
Analysis collector (1.35, 1.36) 

Test name

Short description

Expected result

Notes

LTS RC / result

Tested by

Collects and show results

Setup collection of checkstyle, findbugs, pdm and warnings results collection

Results are collected and shown

 

 

 
PMD Plug-in (3.38) 

Test name

Short description

Expected result

Notes

LTS RC / result

Tested by

Automated acceptance tests

All automated Tests pass

Results are collected and shown

 

1.554.2 / OK 

trampi

Checkstyle Plug-in (3.39) 

Test name

Short description

Expected result

Notes

LTS RC / result

Tested by

Automated acceptance tests

All automated Tests pass

Results are collected and shown

 

1.554.2 / OK 

trampi

Findbugs Plug-in (4.56) 

Test name

Short description

Expected result

Notes

LTS RC / result

Tested by

Automated acceptance tests

All automated Tests pass

Results are collected and shown

 

1.554.2 / OK 

trampi

Jira (1.35, 1.37) 

Test name

Short description

Expected result

Notes

LTS RC / result

Tested by

Update JIRA issue

Update JIRA issue which is related to changes contained in the build

JIRA is updated and on build page is a link to appropriate JIRA

 automated - run manually

1.554.1 / OK
1.554.3 / OK 

ogondza
ogondza

Deploy (1.9) 

Test name

Short description

Expected result

Notes

LTS RC / result

Tested by

Deploy on Tomcat 7

Build war app and deploy it on Tomcat 7

App is successfully deployed and running

automated - run manually

1.554.1 / OK
1.554.2 / OK 
1.554.3 / OK  

ogondza
ogondza
ogondza

Dry (2.34) 

Test name

Short description

Expected result

Notes

LTS RC / result

Tested by

Publish DRY analysis results

Setup CPD analysis and publish analysis results

Results are published

 

 

 
Backup (1.6.1) 

Test name

Short description

Expected result

Notes

LTS RC / result

Tested by

Backup Jenkins configuration

Setup backup dir and run Jenkins backup

Once no builds run, backup is performed

 

 

 
Scp publisher (1.8) 

Test name

Short description

Expected result

Notes

LTS RC / result

Tested by

Publish artifact over scp

Setup remote destination and artifact to be archived over scp

Artifact is archived on remote machine in speficied dir

automated - run manually

1.554.1 / OK
1.554.3 / OK 

ogondza
ogondza

IRC (2.23) 

Test name

Short description

Expected result

Notes

LTS RC / result

Tested by

Send IRC notification

Setup IRC notification and schedule a build

Notification is send to appropriate IRC channel

 

1.554.1 / OK
1.554.2 / OK
1.554.3 / OK 

ogondza
ogondza
ogondza

Run jenkins bot command 

Run bot command on IRC channel

Command is executed

 

 

 
LDAP

Test name

Short description

Expected result

Notes

LTS RC / result

Tested by

LDAP test

LdapPluginTest in test-harness

All tests succeed

automated - run manually

1.554.2 / OK
1.554.3 / OK 

eidottermihi
ogondza

Publish Over FTP

Test name

Short description

Expected result

Notes

LTS RC / result

Tested by

Ftp test

Ftp testin test-harness

All tests succeed

automated - run manually

1.554.2 / OK

tobiwan88

Subversion

Test name

Short description

Expected result

Notes

LTS RC / result

Tested by

Subversion test

SubversionPluginTest && Subversion_Version154_PluginTest in test-harness

All tests succeed

automated - run manually

1.554.2 / OK

itgkarl

NodeLabelParameter (1.5.1) 

Test name

Short description

Expected result

Notes

LTS RC / result

Tested by

NodeLabelParamater test

NodeLabelParamater test in test-harness

All tests (except ignored ones) succeed

automated - run manually

1.554.2 / OK

hm_ende

Exploratory Tests

Test name

Short description

Expected result

Notes

LTS RC / result

Tested by

Stress/Load Test

Run Xenu link checker while other interactive tests are running, and while jobs are executing

Performance slower, but still functional and reliable, memory growth reasonable

 

 

 

Git plugin

Explore multiple configurations of the Git plugin, including JGit and CliGit implementations, using ssh, https, and git protocol repositories.
Run tests with multiple platforms (Windows, Linux) and multiple git versions (1.7.1, 1.8,1.9, etc.), multiple git configurations (CLI, JGit), multiple providers (GitHub, BitBucket, local git repository), and multiple permissions (private repositories, public repositories).

Reasonable behavior, no detected Jenkins related regressions

 

1.554.1 / OK

mwaite

Task Scanner plugin

Explore different configurations of the plugin, including file name patterns, exclude patterns, different task tags, status thresholds, different source file sets inside the workspace. Job has run on master and slave.

Reasonable behavior, graphical and textual results represent actual occurrences of tags, build status according to threshold settings, no detected Jenkins related regressions

 

1.554.2 / OK

hm_ende