Plugin Information |
---|
View External Monitor Job Type on the plugin site for more information. |
Adds the ability to monitor the result of externally executed jobs.
Jenkins is useful for monitoring the non-interactive execution of processes, such as cron jobs, procmail, inetd-launched processes. Often those tasks are completely unmonitored (which makes it hard for you to notice when things go wrong), or they send e-mails constantly regardless of the success or failure (which results into the same situation as you'll quickly start ignoring them anyway.) Using Jenkins enables you to monitor a large number of such tasks with little overhead.
Setting up a project
Create a new job and choose "Monitor an external job" as the job type.
Monitoring an execution
For Debian / Ubuntu:
sudo apt-get install jenkins-external-job-monitor # obviously you should replace localhost with a FQDN if you want to run jobs from other machines. export JENKINS_HOME=http://@localhost:8080/ java -jar /usr/share/jenkins/external-job-monitor/java/jenkins-core-*.jar "external-build-job-name" command-to-run java -jar /usr/share/jenkins/external-job-monitor/java/jenkins-core-*.jar "external-build-test" ls -l
Once you set up a project, you can monitor an execution by running a command like this:
$ export JENKINS_HOME=http://user:pw@myserver.acme.org/path/to/jenkins/ $ java -jar /path/to/WEB-INF/lib/jenkins-core-*.jar "job name" <program arg1 arg2...>
For Windows:
> set JENKINS_HOME=http://user:pw@myserver.acme.org/path/to/jenkins/ > java -jar \path\to\WEB-INF\lib\jenkins-core-*.jar "job name" cmd.exe /c <program arg1 arg2...>
If your webserver extracts the jenkins.war
file when it deploys Jenkins then you may use the path directly to the WEB-INF/lib
directory and all other required jars will be found there. Otherwise you may extract these from the war file:
jenkins-core-*.jar remoting-*.jar ant-1.7.0.jar commons-io-1.4jar commons-lang-2.4.jar jna-posix-*.jar xstream-*.jar
All are found in the WEB-INF/lib
path inside the war file. As long as they are all in the same directory, the java -jar /path/to/jenkins-core-*.jar
command will find the other required jars.
- Note: Older versions of Jenkins (before 1.324) also require
winstone.jar
in order to run this command. This jar is found at the top level directory inside the war file, and must be manually added to the classpath (with-classpath
or-cp
) in the java command.
The JENKINS_HOME variable is used to locate the server Jenkins is running, so this must be set. Unless your Jenkins job has build permission for guest users, include the username:password@
portion of the URL, as seen in the examples above.
- Note: Authentication via username:password in JENKINS_HOME is added in Jenkins 1.324; with previous versions either grant anonymous build permission on the job, or use
curl
to post XML to Jenkins (see below).
You can copy jenkins-core-*.jar
and the other required jars to other machines if you want to monitor jobs that are run on a different machine.
stdout and stderr of the program will be recorded, and a non-zero exit code will be considered as a failure.
Monitoring cron jobs
To monitor a cron job, simply run the above set up from your cron script. To avoid receiving e-mails from cron daemon, you might want to write something like:
JENKINS_HOME=http://myserver.acme.org/path/to/jenkins/ 0 * * * * export JENKINS_HOME=$JENKINS_HOME; java -jar jenkins-core-*.jar "backup" backup.sh 2>&1 > /dev/null
Note that you can also move the cron job itself to Jenkins by using free-style software project, which would also allow you to manually execute the job outside the scheduled executions.
Submit a run programatically
The above command submits the execution and its result by sending XML to HTTP. This means you can submit an execution record from any program, as long as you follow the same XML format.
The format is explained below:
<run> <log encoding="hexBinary">...hex binary encoded console output...</log> <result>... integer indicating the error code. 0 is success and everything else is failure</result> <duration>... milliseconds it took to execute this run ...</duration> <displayName>... The name to be displayed rather than the build number ...</displayName> <description>... Description of the build ...</description> </run>
The duration
element is optional, since 1.429 displayName
and description
can also optionally be appended, these three elements can be in any order, but must appear after log and result in that order. Console output is hexBinary encoded so that you can pass in any control characters that are otherwise disallowed in XML.
The above XML needs to be sent to http://myhost/jenkins/job/_jobName_/postBuildResult
.
A simple example using curl would be (using no real data; the sequence 4142430A
encodes a console output of ABC
plus a trailing line feed symbol):
$ curl -X POST -d '<run><log encoding="hexBinary">4142430A</log><result>0</result><duration>2000</duration></run>' \ http://user:pass@myhost/jenkins/job/_jobName_/postBuildResult
Submit a run per CLI
The easiest option is for the execution can be submitted per CLI/ssh command. The gzipped log file can be transported through pipe:
$ cat result.log.gz | ssh jenkins set-external-build-result --display 7d552c4ba_Linux_tb21 --job buildbot --result 1 -b --duration 42 --log -
Sometimes build number is needed, as the calling program might need to put an URL to the submitted build. The CLI command above returns the new build number. The command above can be called programatically from an Java application. You can find a sample for it here.
Changelog
Version 1.7 (Jan 06, 2017)
- Allowing non-administrators to configure or delete jobs, given appropriate permissions.
- Japanese localization update.
Version 1.6 (July 26, 2016)
- JENKINS-36875 Empty screen when creating new jobs
- Follow up of JENKINS-35284
Version 1.5 (July 12, 2016)
- JENKINS-35284 Updated to the new Parent POM and fixed findbugs warnings.
- JENKINS-31162 Item categorization.
Version 1.4 (Nov 18, 2014)
- Updated item label to match new noun-oriented convention.
Version 1.3 (Nov 14, 2014)
- Refined where Build permission was checked to cover also the CLI command.
Version 1.2 (Aug 29, 2013)
- new CLI command (pull #5)
- JENKINS-19377 workaround for log rotation failure
- NPE fix
- translations
32 Comments
Anonymous
If you get:
----Exception in thread "/opt/kuttig/Dev1-Backup-Scripts/svn_backup.sh: stdout copier" java.lang.NoClassDefFoundError: org/apache/tools/ant/BuildException
at hudson.util.StreamCopyThread.run(StreamCopyThread.java:27)
Exception in thread "/opt/kuttig/Dev1-Backup-Scripts/svn_backup.sh: stderr copier" java.lang.NoClassDefFoundError: org/apache/tools/ant/BuildException
at hudson.util.StreamCopyThread.run(StreamCopyThread.java:27)
----call "-Xbootclasspath/a:/usr/local/apache-ant/lib/ant.jar" to your java call (modify the ant path!). So this may result in:
export HUDSON_HOME=http://localhost:8080/hudson; java -Xbootclasspath/a:/usr/local/apache-ant/lib/ant.jar -jar hudson-core-1.136.jar "SVN_Backup" /opt/svn/svn_backup.sh
- zeisss
Kohsuke Kawaguchi
This issue is filed as JENKINS-802.
kevinket -
How about running on a Windows machine. I've created a bat file like this:
set HUDSON_HOME=http://localhost:8080
java -jar C:\hudson_master\lib\hudson-core-1.200.jar "build_release_12.0" call_build.bat release_12.0
I get this error (which indicates it cannot file hudsonhome):
Exception in thread "main" java.lang.NoClassDefFoundError: hudson/remoting/Callable
at hudson.Main.getHudsonHome(Main.java:47)
at hudson.Main.run(Main.java:37)
at hudson.Main.main(Main.java:29)
Caused by: java.lang.ClassNotFoundException: hudson.remoting.Callable
at java.net.URLClassLoader$1.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at java.lang.ClassLoader.loadClassInternal(Unknown Source)
... 3 more
I'd think that this should work on a Windows machine as well.
Unknown User (hudson@drewcox.com)
I had this too (on Linux though), fixed it by copying the "remoting" jar out of the distro and into the same directory as the hudson-core jar. The husdon-core jar has a long classpath defined in its manifest.mf, I haven't quite figured out what dependencies are required for remote monitoring, as I now have this error:
Exception in thread "catalog.full.zip: stdout copier" java.lang.StringIndexOutOfBoundsException: String index out of range: -3
at java.lang.String.charAt(String.java:687)
at hudson.util.EncodingStream.write(EncodingStream.java:21)
at java.io.FilterOutputStream.write(FilterOutputStream.java:108)
at hudson.util.DualOutputStream.write(DualOutputStream.java:29)
at hudson.util.StreamCopyThread.run(StreamCopyThread.java:28)
pcohen -
Hi I've been using Hudson for 2 months now and I must say that it is a really nice tools and there are so many available plugins.
It is really nice.
Today I started to play with the monitoring of external jobs and had an authentication error even when I set in the HUDSON_HOME env variable the userinfo user:password in the URL.
Looking in the code Main.java , I added
That works now but I am not sure this is perfect as it assumes that authentication is expected.
Anyway that could perhaps be interesting to add such feature one way or another.
Thanks
Pascal
Unknown User (oakley@bardo.clearlight.com)
Is it possible to use this technique to send back data formatted as HTML? I'd like to be able to include some hyperlinks and bold text into the information that is generated on the console output page.
Unknown User (josh@chapterthree.com)
I am also wondering about log formats. Haven't had any luck (yet) getting intelligible results via HTTP POST and the XML format.
Unknown User (prempola@gmail.com)
Got this working using the following command.
java -cp remoting-1.313.jar:hudson-core-1.313.jar:servlet.jar hudson.Main "Test" Test.groovy
Unknown User (moon people)
Example of working command:
export HUDSON_HOME=http://localhost:8080/hudson
java -classpath remoting.jar:./lib/hudson-core-1.315.jar:/opt/hudson/apache-tomcat-6.0.20/lib/servlet-api.jar hudson.Main "test-mon" ksh ./test_server_health.ksh
Details:
I was having a lot of issues trying to get external jobs running successfully, but I figured it out. Here's what I did
Hudson version: 1.315
(running all this on OS X 10.5.6)
1) created dir /opt/hudson
2) installed apache-tomcat-6.0.20 into that directory (/opt/hudson/apache-tomcat-6.0.20)
3) dropped hudson.war (version 1.315) into /opt/hudson/apache-tomcat-6.0.20/webapps dir
4) started up tomcat (you may have a few config tweaks to make in server.xml withing your apache-tomcat-6.0.20 dir)
5) went to http://localhost:8080/hudson saw that hudson was running ok
6) Hudson -> New Job -> Monitor an external job (gave it a title of test-mon) -> OK
7) cd into /opt/hudson/apache-tomcat-6.0.20/webapps/hudson/WEB-INF
8) set HUDSON_HOME=http://localhost:8080 (this can also be set to an entirely different hudson server. it does not have to be on your local machine to work.)
9)from the /opt/hudson/apache-tomcat-6.0.20/webapps/hudson/WEB-INF directory I ran the following:
10) java -classpath remoting.jar:./lib/hudson-core-1.315.jar:/opt/hudson/apache-tomcat-6.0.20/lib/servlet-api.jar hudson.Main "test-mon" ksh ./test_server_health.ksh
11) Important: make sure to pass the language executor before the script (note: I ran 'ksh ./test_server_health.ksh' I tried many times without the 'ksh' and it doesn't work)
12) went to http://localhost:8080/hudson and saw my test passed.
the hardest thing I had to deal with was getting the classpath correct. you need remoting.jar, hudson-core-1.315.jar(of course the versioning will be different for you, 1.315 is the latest version as of today) & a servlet.jar (I chose servlet-api.jar from the apache lib directory out of frustration, so there may be another that exists that is the correct one.)
in the command I provided I'm running a korn shell script (which is why you see the ksh before the actual script). you could run anything and typically groovy is sited as an example, but it can be bash, perl, ruby.
also realized that where ever you set HUDSON_HOME to is where the test results get posted, so setting HUDSON_HOME=http://localhost:8080/hudson and looking for results on http://cool.company.com/hudson will make you go crazy
good luck!!
Unknown User (alsen)
Alternative solution for Linux:
http://n2.nabble.com/hudson-monitoring-external-jobs-tp3771019p3771019.html
Unknown User (jpeck)
It seems I also need jna-posix-1.0.3.jar (hudson 1.358)
Unknown User (derekjchoi@yahoo.ca)
I am attempting to connect a set of procmail executions and cronjobs with hudson, which works great. However, one thing that I noticed during my testing is if the hudson server itself goes down, the executions don't run at all, which puts a dependency on Hudson CI being up. Is it possible to pass a command line argument to ignore errors communicating with hudson and continue with the execution?
Andrew Nutter-Upham
This may be too late for Derek, but perhaps it will help others.
Here is a perl script that does the jenkins external job submission. The job will always run; Jenkins notification is secondary.
https://gist.github.com/1237898
Oliver Kurz
Hi Andrew, I am interested in the perl script. The gist seems to have vanished. Would you mind putting it up again?
Ivan Mounitsyn
1. If you're getting java exeptions when setting up monitoring of external jobs that could be caused by the fact that you need commons-io-*.jar added to your client Jenkins setup. The list of 5 jars above seems incomplete.
2. If your external job is memory-hungry (tries to grab more than 2Gb of ram) make sure you're running 64bit Java. Sounds obvious? Well, for me it wasn't ...
Kumar Dev
Hi,
I have setup up a jenkins system and trying to set up a job to monitor a cron job. My jenkins URL is http://dev1:8080 .This is what I'm doing on server dev02:
JENKINS_HOME=http://dev01:8080
export JENKINS_HOME=$JENKINS_HOME; java -jar jenkins-core-*.jar "Dev01 Report" kdev.sh 2>&1 > /dev/null
I am getting below message
Unable to access jarfile jenkins-core-*.jar
I have created a monitoring job Dev01 Report on jenkins dashboard.
I also tried
export JENKINS_HOME=$JENKINS_HOME; java -jar jenkins-core-1.451.jar "SUN11 Report" kdev.sh 2>&1 > /dev/null
but getting same error
Unable to access jarfile jenkins-core-1.451.jar
Can someone please help?
Thanks.
Kumar Dev
I managed to get past this one. I had to copy below files on to the remote server from $JENKINS_HOME/.jenkins/war/WEB-INF/lib/jenkins-core-*.jar
remoting-*.jar
ant-1.7.0.jar
commons-lang-2.4.jar
xstream-*.jar
But now getting this error
$ java -jar jenkins-core-1.451.jar "Dev02 Report" kdev.sh 2>&1 > /dev/null
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/commons/io/input/NullInputStream
at hudson.Main.remotePost(Main.java:151)
at hudson.Main.run(Main.java:74)
at hudson.Main.main(Main.java:56)
Caused by: java.lang.ClassNotFoundException: org.apache.commons.io.input.NullInputStream
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
... 3 more
$
Do I need to copy more files from $JENKINS_HOME/.jenkins/war/WEB-INF/lib to remote server?Any pointers anyone?
Robert Kahlert
You are missing the commons-io*.jar In my release of Jenkins, its commons-io-1.4.jar.
Kumar Dev
Tried running after copying commons-io*.jar file as well and got below error
$ java -jar jenkins-core-1.451.jar "Dev02 Report" kdev.sh 2>&1 > /dev/null
Exception in thread "main" java.lang.NoClassDefFoundError: org/jruby/ext/posix/POSIX
at hudson.Main.remotePost(Main.java:174)
at hudson.Main.run(Main.java:74)
at hudson.Main.main(Main.java:56)
Caused by: java.lang.ClassNotFoundException: org.jruby.ext.posix.POSIX
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
... 3 more
Then copied jna-posix-1.0.3.jar and it went fine. So to summarize, I think we need below files on the remote server for remote monitoring to work properly. My server/os version is SunOS dev02 5.10 Generic_142909-17 sun4u sparc SUNW,Sun-Fire-V445
Files copied
jenkins-core-1.451.jar
xstream-1.3.1-jenkins-9.jar
ant-1.8.0.jar
commons-lang-2.4.jar
jenkins-core-1.451.jar
remoting-2.12.jar
xstream-1.3.1-jenkins-9.jar
commons-io-1.4.jar
jna-posix-1.0.3.jar
Thanks for help.
Kumar Dev
I have put together the steps I followed to "Monitor External Jobs" using Jenkins at link below:
http://kvdev.wordpress.com/2012/03/01/jenkins-monitoring-external-jobs/
Hope everyone finds this useful. Feedback welcome.
zhewei shou
when i export JENKINS_HOME(which is not localhost path), i need pass username & password. but what can i do if the password exists character like '@'?
zhewei shou
when i export JENKINS_HOME(which is not localhost path), i need pass username & password. but what can i do if the password exists character like '@'?
Dave Kelsey
I'm using the latest rev of jenkins 1.472.
I've set it up locally and tried both a remote and local test with
ls -l
This is what I see in the console ouput:
There is a ticket in the issue tracking system:
https://issues.jenkins-ci.org/browse/JENKINS-14107
If other people are experiencing this please create an account and vote for the issue.
Kumar Dev
Hi,
I have set this up to run both remotely and locally. I did face some issues initially but finally managed to fix it. I have put down the steps to setup jenkins from scratch at below link, can you please take a look and confirm that you haven't missed anything:
http://kvdev.wordpress.com/2012/03/01/jenkins-monitoring-external-jobs/
I have set this up on solaris, but the steps should work on any *nix system. Please let me know if this helps.
Thanks.
Dave Kelsey
Kumar,
Thanks for our help. Nope. it didn't work. I notice you are using version 1.451. I tried it on solars and on my mac. I get the same error.
...
Update: Kumar, I rolled back to ver 1.464 and the ls -l example worked. The latest release doesn't work
Aldo Reyes
Hi,
Im enjoying every bit of jenkins so far, I need to send a notification email after a cron job finish, I see that there is no way to add post build tasks on this kind of project, I also tried the strawboss plugin but that doesnt work on newer jenkins versions (it seems outdated) is there a way to do this?
I know a workaround is to use a ssh plugin, but I would like to know if there is something native in jenkins or with a plugin that only lets me add a post build step to these kind of jobs, thanks for the great work guys!
Frank Merrow
So we have a number of Jenkins . . . and for the most part they are completely separate . . .
However, we ran into the situation where one Jenkins didn't have enough slaves to run a particular assessment and so it go put onto a separate Jenkins.
What I would really like is for the status of "that job on that Jenkins" to show up normally on "this Jenkins".
Is that possible . . . this plugin seems to require a change to the way the extrnal process is involked . . . it really don't want to change the "remote" Jenkins job . . . only have it show up here too.
Is that possible?
Frank
ozmen adibelli
Hi,
I can not find this external-monitor-job plugin in "available plugins" page. How can I install ?
Kumar Dev
Hi,
Why do you need such a plugin? I don't think one exists. What is it that you are exactly trying to do? If you are trying to monitor an external job then you need to configure this in jenkins. Follow the instructions listed at below link and let me know:
http://kvdev.wordpress.com/2012/03/01/jenkins-monitoring-external-jobs/
Thanks,
KVD
Vishal Rekala
How do i install this plugin, i don't see it in plugin manager -> Available plugins???
John Lussmyer
As far as I can tell, it's now called "External Monitor Job Type Plugin".
baar se
There seems to be a problem with the plugin and the jenkins CSRF protection.
When I run my application with jenkins CSRF protection I get this error message:
<html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>
<title>Error 403 No valid crumb was included in the request</title>
</head>
<body><h2>HTTP ERROR 403</h2>
<p>Problem accessing /job/test/postBuildResult. Reason:
<pre> No valid crumb was included in the request</pre></p><hr><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.4.z-SNAPSHOT</a><hr/>
</body>
</html>