Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 3 Next »

Plugin Information

View Gearman on the plugin site for more information.

Summary

This plugin integrates Gearman with Jenkins.


Jenkins does not support multiple masters.  You can setup multiple Jenkins masters but there is no coordination between them.

One problem with scheduling builds on Jenkins master (“MasterA”) server is that MasterA only knows about its connected slaves.  If all slaves on MasterA are busy then MasterA will just put the next scheduled build on its queue.  Now MasterA needs to wait for an available slave to run the build.  This will be very in-efficient if your builds take a long time to run.  So.. what if there is another Jenkins master (“MasterB”) that has free slaves to service the next scheduled build?  Your probably saying “Then slaves on MasterB should run the build” then I would say "good thought!".  However MasterB will never service the builds on MasterA's queue.  The client that schedules the builds must know about MasterB and then schedule builds on MasterB. This is what we mean by lack of coordination between masters. This  gearman-plugin attempts to fill the gap.

This plugin integrates Gearman with Jenkins and will make it so that any Jenkins slave on any Jenkins master can service a job in the queue. It will essentially replace the Jenkins build queue with the Gearman job queue.  The job will stay in the Gearman queue until there is a Jenkins node (master or slave) that can run that job.

Getting Started

This assumes some familiarity with Jenkins, Gearman, Eclipse and Maven.

Install

  • Install the Gearman plugin like any other Jenkins plugin, refer to the Jenkins documentation.
  • If you don't already have a Gearman server up and running somewhere you should install one.  To install on Ubuntu just run "sudo apt-get install gearman-job-server". 

Configuration

 

The Gearman plugin configuration should appear in the Jenkins global configuration page.  Read help bubbles for help with configuration.  You should test the connection to your Gearman Server before launching the workers.

Workflow

Here is the typical:

  1. On a 'Launch Worker', a Gearman worker is spawned for each Jenkins executor.  We'll call these "executor worker threads". Each executor worker thread is associated 1:1 with a Jenkins executor (both slave and master).
  2. Now we register jobs for each Gearman executor depending on the projects, labels and nodes that are available in the system. View the image to see how the Jenkins projects get mapped to Gearman functions.
  3. We spawn one more thread to be a Gearman worker to handle job management.  We'll call it the "management worker thread" and it will register a "stop:$hostname" Gearman function.  We use this worker to cancel build jobs.  Build jobs are either on the Gearman queue or it's currently running on a Jekins executor.
  4. Any Gearman client can connect to the Gearman server and send a request to build a Jenkins project or cancel a project.  View the Gearman client examples to see how this can be done.
  5. The Gearman workers will service any client request that come through to start and cancel a Jenkins build.

Note - When changes are made to Jenkins projects, labels or nodes you must restart the Gearman workers.  The current release does not dynamically adjusts to Jenkins changes.

Versions

  • 0.0.1 - initial release.

TODO

  • Dynamically update Gearman functions upon change events to Jenkins projects, labels, and nodes.
  • Return warning and error messages on Gearman job failures.
  • Cancel build jobs from the Gearman queue.

Plugin In Action

  • No labels