Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Jenkins supports the "master/agent" mode, where the workload of building projects are delegated to multiple "agent" nodes, allowing a single Jenkins installation to host a large number of projects, or to provide different environments needed for builds/tests. This document describes this mode and how to use it.It is pretty common when starting with Jenkins to have a single server which runs the master and all builds, however Jenkins architecture is fundamentally "Master+Agent". The master is designed to do co-ordination and provide the GUI and API endpoints, and the Agents are designed to perform the work. The reason being that workloads are often best "farmed out" to distributed servers. This may be for scale, or to provide different tools, or build on different target platforms. Another common reason for remote agents is to enact deployments into secured environments (without the master having direct access). 

Many people today use Jenkins in cloud environments, and there are plugins and extensions to support the various environments and clouds. These may involve Virtual Machines, Docker Containers, Kubernetes (for example see Jenkins-X), EC2, Azure, Google Cloud, VMWare and more. In these cases the agents are managed for you typically (and in many cases on demand, as needed), so you may not need to read the content of this document for those cases. 

This document describes this distributed mode of Jenkins and some of the ways in which you can configure it, should you need to take control (or maybe you are curious)


Contents

Table of Contents

How does this work?

...

Since each agent runs a separate program called an "agent" there is no need to install the full Jenkins (package or compiled binaries) on an agent. There are various ways to start agents, but in the end the agent and Jenkins master need to establish a bi-directional communication link (for example a TCP/IP socket) in order to operate.

Follow the Step by step guide to set up master and slave agent machines on Windows to quickly start using distributed builds.


Master to agent connections

The most popular ways agents are configured are via connections that are initiated from the master. This allows agents to be minimally configured and the control lives with the master. This does require that the master have network access (ingress) to the agent (typically this is via ssh). In some cases this is not desirable due to security network rules, in which case you can use Agent to master connections via "JNLP".

Agent to master connections

In some cases the agent server will not be visible to the master, so the master can not initiate the agent process. You can use a different type of agent configuration in this case called "JNLP". This means that the master does not need network "ingress" to the agent (but the agent will need to be able to connect back to the master). Handy for if the agents are behind a firewall, or perhaps in some more secure environment to do trusted deploys (as an example). See the sections below to choose the type of agent that is most appropriate for your needs. 

Choosing which agent pipelines and steps run on

As you will see below, agents can be labelled. This means different part of your build, or pipeline, can be allocated to run in specific agents (based on their label). This can be useful for tools, operating systems or perhaps for security purposes (it is possible to set quite detailed access rules of what can run where, based on agent configurations). A server that runs an agent is often referred to as a "Node" in Jenkins terminology. 

Different ways of starting agents

Pick the right method depending on your environment and OS that master/agents run, or if you want the connection initiated from the master or from the agent end.

Have master launch agent via ssh

...

Launching agents this way often requires an additional initial set up on agents (especially on Windows, where remote login mechanism is not available out of box), but the benefits of this approach is that when the connection goes bad, you can use Jenkins's web interface to re-establish the connection.

Launch agent via

...

"JNLP" from agent back to master in a browser

Another way of doing this is to start an agent through Java Web Start (JNLP).

...

Note: If the master is running behind a reverse proxy or similar, you might need to configure "Tunnel connection through" in the "Advanced" section of the JNLP start method on the agent configuration page to make JNLP work.

Launch agent headlessly from agent back to master on command line

This launch mode uses a mechanism very similar to Java Web StartJNLP as described above, except that it runs without using GUI, making it convenient for an execution as a daemon on Unix. To do this, configure this agent to be a JNLP slaveagent, take agent.jar as discussed above, and then from the agent, run a command like this:

...

  • Each computer has an user called jenkins and a group called jenkins. All computers use the same UID and GID. (If you have access to NIS, this can be done more easily.) This is not a Jenkins requirement, but it makes the agent management easier.
  • On each computer, /var/jenkins directory is set as the home directory of user jenkins. Again, this is not a hard requirement, but having the same directory layout makes things easier to maintain.
  • All machines run sshd. Windows agents run cygwin sshd.
  • All machines have /usr/sbin/ntpdate installed, and synchronize clock regularly with the same NTP server.
  • Master's /var/jenkins have all the build tools beneath it --- a few versions of Ant, Maven, and JDKs. JDKs are native programs, so I have JDK copies for all the architectures I need. The directory structure looks like this:

    No Format
    /var/jenkins
      +- .ssh
      +- bin
      |   +- agent  (more about this below)
      +- workspace (jenkins creates this file and store all data files inside)
      +- tools
          +- ant-1.5
          +- ant-1.6
          +- maven-1.0.2
          +- maven-2.0
          +- java-1.4 -> native/java-1.4 (symlink)
          +- java-1.5 -> native/java-1.5 (symlink)
          +- java-1.8 -> native/java-1.8 (symlink)
          +- native -> solaris-sparcv9 (symlink; different on each computer)
          +- solaris-sparcv9
          |   +- java-1.4
          |   +- java-1.5
          |   +- java-1.8
          +- linux-amd64
              +- java-1.4
              +- java-1.5
               +- java-1.8
  • Master's /var/jenkins/.ssh has private/public key and authorized_keys so that a master can execute programs on agents through ssh, by using public key authentication.
  • On master, I have a little shell script that uses rsync to synchronize master's /var/jenkins to agents (except /var/jenkins/workspace). I also use the script to replicate tools on all agents.
  • /var/jenkins/bin/launch-agent is a shell script that Jenkins uses to execute jobs remotely. This shell script sets up PATH and a few other things before launching agent.jar. Below is a very simple example script.

    Code Block
    #!/bin/bash
    
    JAVA_HOME=/opt/SUN/jdk1.8.0_152
    PATH=$PATH:$JAVA_HOME/bin
    export PATH
    java -jar /var/jenkins/bin/agent.jar
    

...

Some interesting pages on issues (and resolutions) occurring when using Windows agents:

...