Skip to end of metadata
Go to start of metadata

Plugin Information

View Libvirt Slaves on the plugin site for more information.

Add Libvirt Hypervisor slave support to Jenkins

libvirt-slaves 1.8 uses a newer libvirt java binding. If you are upgrading from an earlier version of this plugin, you will have to reconfigure your hypervisor settings and thus also all libvirt-controlled nodes!
The least painful approach (especially for larger setups) is to properly set up the Hypervisor first, followed by editing each node, pointing it to the reconfigured Hypervisor. It is not necessary to delete node configurations!

Description

This plugin for Jenkins CI adds a way to control guest domains hosted on Xen or QEMU/KVM. You configure a Jenkins Slave, selecting the guest domain and hypervisor. When you need to build a job on a specific Slave, its guest domain is started, then the job is run. When the build process is finished, the guest domain is shut down, ready to be used again as required.

Requirements

For a communication with Hypervisors you need to install libvirt library on Jenkins machine. This plugin comes with libvirt java binding, but is just an interface to the "real" C libvirt library. In your distribution you should have a package to install libvirt (ie yum install libvirt on RedHat/Fedora/Centos distributions).

Configuration

Hypervisor

The first step is the Hypervisor configuration. To create a new Hypervisor you need to add a new "Cloud" in the Jenkins "Manage Jenkins" menu.

 

The required parameters to setup are:

  • Hypervisor Type: QEMU / XEN to respect the Hypervisor you have in your system
  • Hypervisor Host: Hostname or IP address to contact your hypervisor
  • Username: username to use for connection.
  • Port: ssh port, if in your hypervisor machine is different than 22
  • URI parameter: by default xen and kvm expose the control API using system. If, for any reasons, you don't have this default value or need to provide further parameters, you can set those here
  • Concurrent Slaves Capacity: If you are running a setup where slaves are being shut down once they are idling and you want to control how many concurrent slave can be run by Jenkins on the particular hypervisor, you can set this threshold here. Providing 0, the default value, disables the threshold. Please note that a threshold will not fail build jobs, Jenkins will simply reissue the slave commissioning once the hypervisor is again running below its threshold, thus delaying the start up of slaves

The connection to Hypervisor will be done using ssh, so you don't need to setup libvirt, exposing your services on tcp that could be a security hole in your infrastructure. Here an example of connection string will be used by Libvirt Slaves Plugin to create a connection with the hypervisor:

 xen+ssh://username@hostname:port/system

you can test you connection typing, from your Jenkins Server:

 virsh connect xen+ssh://username@hostname:port/system

An important thing is the RSA public key exchange between Jenkins Server and Hypervisor Machine: you need to add to .ssh/known_hosts file the rsa.pub file of your jenkins user. This because, for an actual limitation in libvirt java library, it's impossible to provide via code the ssh password, that means Jenkins could not establish a connection if password request will be prompted.

To verify all you parameters you can click on Test Connection button and check the output reported.

Slaves

Now you can setup your nodes in Jenkins and use them to build your projects.

Once the node is created, you'll see the configuration page as shown below:

Here you can configure the following details:

  • Hypervisor: here you select one of the clouds that you configuration at the central Configure Jenkins page
  • Virtual Machine: select one of the virtual machines that you want to use as a slave
  • Revert Snapshot: optionally you can select an existing snapshot of the virtual machine that you want the slave to be reverted to once it is being shut down
  • Startup Idle: this optional value (default is 60) allows you specify an idle timer in seconds. Once the virtual machine has been started, Jenkins will wait that long before starting the actual slave service on the virtual host. If your hypervisor is super quick, set a low value, if it takes a while to get that VM up, increase the timer.

Change Log

Version 1.8.5 (Apr 01, 2015)
  • Fix JENKINS-12523: Could not initialize class org.libvirt.Connect
Version 1.8.1 (Mar 21, 2013)
  • Increased robustness of the hypervisor session handling: explicit disconnect upon Jenkins shutdown, auto-reconnect upon libvirtd restarts
  • Various minor improvements to the logging, chattiness decreased
Version 1.8 (Mar 20, 2013)
  • Fix JENKINS-16889: Fixed Hypervisor session management
  • Feature JENKINS-17293: Concurrent slave threshold, configurable per Hypervisor (Big thanks to Bryan Dagnin for his contribution!)
  • Feature JENKINS-16583: Support for libvirt snapshot mechanism
  • Feature JENKINS-16581: Modernized maven groupId to org.jenkins-ci.plugins
  • Various minor improvements, mostly help/documentation related
  • Latest libvirt java binding. Note: upgrading users will have to reconfigure their hypervisor and node settings!
Version 1.7 (Jan 31, 2013)
Version 1.6 (Apr 2, 2010)
  • Fixed problems with Libvirt on RedHat / Centos operating System
  • Added control for machine without libvirt library installed
Version 1.5 (Mar 30, 2010)
  • Fixed problem on Null object after Hypervisor reconnection
Version 1.4 (Mar 25, 2010)
  • Improvements in Hypervisor connection using libvirt
  • Added a missing help file
Version 1.3 (Mar 20, 2010)
  • Fixed problem with hypervisor reconnection after Hudson restart
  • Add a wait time before launching slave agent
Version 1.2 (Mar 10, 2010)
  • Fixed problem in contextual help files-
  • Catch exception on machine without libvirt installed
Version 1.1 (Mar 08, 2010)
  • No change in sources. A version just to fix a problem during release process.
Version 1.0 (Mar 07, 2010)
  • First version published

6 Comments

  1. I've struggled for the past 3 days trying to get the libvirt plugin to work
    Running XenServer 6.2  and XCP 1.6   I've installed the libvirt yum package by enabling the Base-Centos 5 repos which were already existing in xen, I just had to set the enable flag to 1
    When I did this  and installed libvirt it installed version 0.8.2.  Also, I've setup ssh keys and communication goes through just fine here are the results I get:::
    when not supplying anything URI parameter I get org.libvirt.LibvirtException: unable to connect to 'localhost:8000': Connection refused  
    and if I put in a URI  something like  "system"  I get   org.libvirt.LibvirtException: internal error unexpected Xen URI path '/system', try xen:///

    any ideas?    How can I install a newer version of libvirt?   I downloaded the rpm but it has alot of dependancies and I'm not sure how to make that work.   Maybe I just need to have xapi look at port 8000  but I'm not sure hwo to do that
    These settings don't appear anywhere on the VM:

    After enabling:

    (xend-http-server yes)
    (xend-port 8000)

  2. It looks like 1.8.5 with the fix for JENKINS-12523 Could not initialize class org.libvirt.Connect was released yesterday ( Apr 1 2015 ). How long before this shows up in my Jenkins 1.599 'Manage Plugins' update?

  3. Hi,

    I added a libvirt cloud and tested the connection - all OK. I can see all the configured VMs within the given cloud. But when I try to add a libvirt slave, the combobox with virtual machines is empty. I can't choose my desired virtual machine to run the builds on.

    Can you, please, help me set this up?

    Thank you very much!

    1. Hello Peter Hatina,

      I am facing the same issue i.e. the Virtual Machines are not listed in the drop down section under '"New Node" -> "Virtual Machine"' [After Hypervisor section], though the VM's are listed under the "Libvirt" section of "Manage Jenkins". Are you still facing this issue? If you found the solution for the same, can you please share the steps to make the setup work?

      Thanks in Advance

  4. Hi,

    I have Jenkins running in a Docker Container on an UnRaid server. The same server also supports KVM-based VMs, which I want to use as slaves.

    I am stuck on the "install libvirt library on Jenkins machine" step. libvirt is installed on the host OS, but I get an error from Jenkins when I try to use this plugin. So, I am assuming I need to add it to the Docker container somehow? Assuming this is the case, can anyone explain to me how to do that? (I'm not a Docker expert! (wink) )

     

    TIA