Jenkins supports the "master/agent" mode, where the workload of building projects are delegated to multiple "agent" nodes, allowing a single Jenkins installation to host a large number of projects, or to provide different environments needed for builds/tests. This document describes this mode and how to use it.It is pretty common when starting with Jenkins to have a single server which runs the master and all builds, however Jenkins architecture is fundamentally "Master+Agent". The master is designed to do co-ordination and provide the GUI and API endpoints, and the Agents are designed to perform the work. The reason being that workloads are often best "farmed out" to distributed servers. This may be for scale, or to provide different tools, or build on different target platforms. Another common reason for remote agents is to enact deployments into secured environments (without the master having direct access).
Many people today use Jenkins in cloud environments, and there are plugins and extensions to support the various environments and clouds. These may involve Virtual Machines, Docker Containers, Kubernetes (for example see Jenkins-X), EC2, Azure, Google Cloud, VMWare and more. In these cases the agents are managed for you typically (and in many cases on demand, as needed), so you may not need to read the content of this document for those cases.
This document describes this distributed mode of Jenkins and some of the ways in which you can configure it, should you need to take control (or maybe you are curious)
|Table of Contents|
How does this work?
Since each agent runs a separate program called an "agent" there is no need to install the full Jenkins (package or compiled binaries) on an agent. There are various ways to start agents, but in the end the agent and Jenkins master need to establish a bi-directional communication link (for example a TCP/IP socket) in order to operate.
Master to agent connections
The most popular ways agents are configured are via connections that are initiated from the master. This allows agents to be minimally configured and the control lives with the master. This does require that the master have network access (ingress) to the agent (typically this is via ssh). In some cases this is not desirable due to security network rules, in which case you can use Agent to master connections via "JNLP".
Agent to master connections
In some cases the agent server will not be visible to the master, so the master can not initiate the agent process. You can use a different type of agent configuration in this case called "JNLP". This means that the master does not need network "ingress" to the agent (but the agent will need to be able to connect back to the master). Handy for if the agents are behind a firewall, or perhaps in some more secure environment to do trusted deploys (as an example). See the sections below to choose the type of agent that is most appropriate for your needs.
Choosing which agent pipelines and steps run on
As you will see below, agents can be labelled. This means different part of your build, or pipeline, can be allocated to run in specific agents (based on their label). This can be useful for tools, operating systems or perhaps for security purposes (it is possible to set quite detailed access rules of what can run where, based on agent configurations). A server that runs an agent is often referred to as a "Node" in Jenkins terminology.
Different ways of starting agents
Pick the right method depending on your environment and OS that master/agents run, or if you want the connection initiated from the master or from the agent end.
Have master launch agent via ssh
Launching agents this way often requires an additional initial set up on agents (especially on Windows, where remote login mechanism is not available out of box), but the benefits of this approach is that when the connection goes bad, you can use Jenkins's web interface to re-establish the connection.
Launch agent via
"JNLP" from agent back to master in a browser
Another way of doing this is to start an agent through Java Web Start (JNLP).
Note: If the master is running behind a reverse proxy or similar, you might need to configure "Tunnel connection through" in the "Advanced" section of the JNLP start method on the agent configuration page to make JNLP work.
Launch agent headlessly from agent back to master on command line
This launch mode uses a mechanism very similar to Java Web StartJNLP as described above, except that it runs without using GUI, making it convenient for an execution as a daemon on Unix. To do this, configure this agent to be a JNLP slaveagent, take
agent.jar as discussed above, and then from the agent, run a command like this:
- Each computer has an user called
jenkinsand a group called
jenkins. All computers use the same UID and GID. (If you have access to NIS, this can be done more easily.) This is not a Jenkins requirement, but it makes the agent management easier.
- On each computer,
/var/jenkinsdirectory is set as the home directory of user
jenkins. Again, this is not a hard requirement, but having the same directory layout makes things easier to maintain.
- All machines run
sshd. Windows agents run
- All machines have
/usr/sbin/ntpdateinstalled, and synchronize clock regularly with the same NTP server.
/var/jenkinshave all the build tools beneath it --- a few versions of Ant, Maven, and JDKs. JDKs are native programs, so I have JDK copies for all the architectures I need. The directory structure looks like this:
/var/jenkins +- .ssh +- bin | +- agent (more about this below) +- workspace (jenkins creates this file and store all data files inside) +- tools +- ant-1.5 +- ant-1.6 +- maven-1.0.2 +- maven-2.0 +- java-1.4 -> native/java-1.4 (symlink) +- java-1.5 -> native/java-1.5 (symlink) +- java-1.8 -> native/java-1.8 (symlink) +- native -> solaris-sparcv9 (symlink; different on each computer) +- solaris-sparcv9 | +- java-1.4 | +- java-1.5 | +- java-1.8 +- linux-amd64 +- java-1.4 +- java-1.5 +- java-1.8
/var/jenkins/.sshhas private/public key and
authorized_keysso that a master can execute programs on agents through
ssh, by using public key authentication.
- On master, I have a little shell script that uses rsync to synchronize master's
/var/jenkinsto agents (except
/var/jenkins/workspace). I also use the script to replicate tools on all agents.
/var/jenkins/bin/launch-agentis a shell script that Jenkins uses to execute jobs remotely. This shell script sets up
PATHand a few other things before launching
agent.jar.Below is a very simple example script.
#!/bin/bash JAVA_HOME=/opt/SUN/jdk1.8.0_152 PATH=$PATH:$JAVA_HOME/bin export PATH java -jar /var/jenkins/bin/agent.jar
Some interesting pages on issues (and resolutions) occurring when using Windows agents:
- Windows slaves agents fail to start via DCOM
- Windows slaves fail to start via ssh
- Windows slaves fail to start via JNLP