Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  • High availability(ish).  When one master goes down the other master(s) will continue to execute builds however the in flight jobs on the down downed master will be lost.
  • Slaves are (by default) always shared between masters.  The only way to un-share is to offline or disconnect a slave.
  • Horizontal scalability.  Just continue to add more jenkins masters to distribute the job load between masters.
  • Gearman jobs can start a jenkins build
  • Gearman jobs can stop or abort a jenkins build
  • Gearman jobs can change a build description
  • Gearman jobs can pass in parameters to jenkins builds
  • Gearman jobs can automatically set a slave to offline after running a build
  • Gearman is aware of Jenkins project status: meaning that the gearman plugin will register/unregister projects when the project is enabled or disabled. 
  • Gearman is aware of slave status: meaning that gearman thatthe  gearman will register/unregister slaves when a slave is set online/offline and connected/disconnected.
  • Plugin reloads on jenkins restart: meaning that when jenkins restarts the gearman worker threads are automatically restarted and reconnect to a gearman server. 

...

  • Dynamically update Gearman functions relative to an update to the Jenkins executors.This pliugin does not register functions correctly for Jenkins Matrix Projects.when there is a change to a Jenkins executor on a node.  Currently there jenkins does not fire events when there is a change to the number of executors on a node.
  • The gearman pliugin does not support Jenkins Matrix Projects.  The gearman plugin relies on project and node labels to register functions correctly.  The matrix projects use those labels a lot differently than the other projects.

Getting Started

This assumes some familiarity with Jenkins and Gearman

Install

  • Install the Gearman plugin like any other Jenkins plugin, refer to the Jenkins documentation.  You can also get the plugin directly from the Jenkins CI Repository
  • If you don't already have a Gearman server up and running somewhere you should install one.  This plugin will work with the following gearman servers:
  • The python gear package (recommended).  This is the one we use and test with.
  • The C gearmand server ver 1.1.7 and later. To get later versions of gearman server you may need to build from source.
  • Theoretically the  Theoretically the plugin should work with any gearman server, but we've only used and tested it with the the python gear package.  Install this gearman implementation and the server

Configuration

Image Removed

  • Install the Gearman plugin like any other Jenkins plugin, refer to the Jenkins documentation.  You can also get the plugin directly from the Jenkins CI Repository
  • After installation the Gearman plugin the configuration should appear in the Jenkins global configuration page.  Click on the help bubbles if you need additional help with the configuration.  You should test the connection to your Gearman Server before running the workers.

...

  •  Select the 'Enable Gearman' checkbox and click save button will immediately start the gearman workers on the Jenkins server.

...

  •  

Configuration

Image Added

Workflow

Starting the Gearman workers:

  1. When the gearman plugin is enabled a gearman worker thread is spawned for each executor on the master and slave nodes.  We'll call these "executor worker threads". Each executor worker thread is associated 1:1 with a Jenkins executoran executor on a jenkins node.
  2. We spawn one more  Gearman worker thread to handle job management (i.e. abort job/update description/etc..).  We'll call it the "management worker thread" and it will register a "stop:$hostname" and "set_description:$hostname" function with the gearman server.  We use these functions to manage jenkins builds.  
  3. The gearman plugin will register gearman a function for each Gearman executor depending on the projects, labels and nodes that have been setup on the Jenkins master. You can check the registered gearman functions using the administration protocol.  It should look something like this..

...

  • java client is a simple client included with jenkins-plugin.hpi

Running a Jenkins build

To  build a Jenkins job the gearman client just needs to provide the Gearman hostname, port, function, and UUID to start a jenkins build.   

...

Code Block
python gear_client.py -s MyGearmanSever --function=build:myProject --params='{"OFFLINE_NODE_WHEN_COMPLETE":"false","param1":"moon","param1":"sun"}' --wait

Stopping/aborting a jenkins build

...

Versions

  • 0.1.1
    • Fix job result not being sent back to gearman client, check commit message for more info. 
  • 0.1.0
    • Update to work with Jenkins LTS ver 1.565.3, check commit message for more info.
  • 0.0.7
    • Fix project-node registration. If a node matches any project label, register the generalized job and then also register it for each label in the intersection of project labels and node labels.
  • 0.0.6
  • 0.0.5
    • Set a node offline even if there is an exception
    • Always return WORK_COMPLETE when a build finishes regardless of the result
  • 0.0.4
    • Don't wait for the worker thread to join
    • remove restriction on slave to run single job at a time
    • Use more fine-grained synchronization in GearmanProxy
    • Rework starting/stopping executors
    • moved python examples to jenkins wiki
    • Add OFFLINE_NODE_WHEN_COMPLETE option
  • 0.0.3
    • ignore non-deterministic build failure and log it
  • 0.0.2
    • Bunch of fixes
    • ability to cancel gearman jobs from it's queue
    • ability to set jenkins job descriptions
  • 0.0.1 - initial release.