Skip to end of metadata
Go to start of metadata

Allow Jenkins to start slaves on EC2 or Eucalyptus on demand, and kill them as they get unused. 

With this plugin, if Jenkins notices that your build cluster is overloaded, it'll start instances using the EC2 API and automatically connect them as Jenkins slaves. When the load goes down, excess EC2 instances will be terminated. This set up allows you to maintain a small in-house cluster, then spill the spiky build/test loads into EC2 or another EC2 compatible cloud.

Plugin Information

View Amazon EC2 on the plugin site for more information.

Older versions of this plugin may not be safe to use. Please review the following warnings before using an older version:

This plugin is up for adoption. Looking for a new maintainer Click here to learn more!

Usage

First, go to EC2 and sign up for the service. Once you've installed the plugin, you navigate to the main "Manage Jenkins" > "Configure System" page, and scroll down near the bottom to the "Cloud" section. There, you click the "Add a new cloud" button, and select the "Amazon EC2" option. This will display the UI for configuring the EC2 plugin.  Then enter the Access Key and Secret Access Key which act like a username/password (see IAM section). Because of the way EC2 works, you also need to have an RSA private key that the cloud has the other half for, to permit sshing into the instances that are started. If you have already been using EC2 and have your own key, you can paste it here. Otherwise, you can have Jenkins generate one. If you let Jenkins generate one, you should save this private key in your file system as well, as you'll need this to interactively logon to EC2 instances.

Once you have put in your Access Key and Secret Access Key, select a region for the cloud (not shown in screenshot). You may define only one cloud for each region, and the regions offered in the UI will show only the regions that you don't already have clouds defined for them.

Use "Test Connection" button to verify that Jenkins can successfully talk to EC2. If you are using UEC you need to click on Advanced and fill out the endpoint details for your cluster.

Next, configure AMIs that you want to launch. For this, you need to find the AMI IDs for the OS of your choice. ElasticFox is a good tool for doing that, but there are a number of other ways to do it. Jenkins can work with any Unix AMIs. If using an Ubuntu EC2 or UEC AMI you need to fill out the rootCommandPrefix and remoteAdmin fields under advanced. Windows is currently unsupported.

Configuring labels allow Jenkins to pick the right AMI to start. For example, if all your existing slaves labeled "solaris" are fully busy and you have more builds that are tied to the "solaris" label, Jenkins will start the AMIs that have the "solaris" label.

Init script is the shell script to be run on the newly launched EC2 instance, before Jenkins starts launching a slave agent. If the AMI doesn't have Java pre-installed, you can do this in the init script. This is also a good place to install additional packages that you need for your builds and tests.  The init script is located at /tmp/init.sh and is owned and run by the user account specified in the "Remote User" field (so use of "sudo" may be required for non-root accounts).

Configure several AMIs if you need different OS images.

With this setting, your Jenkins will automatically start instances when the load goes up, and the instances will be terminated (or stopped - see below) automatically if it's idle for more than 30 mins.

By default, instances are terminated when the idle timeout period expires. You can change this by specifying the Stop/Disconnect on Idle Timeout flat in the Advanced properties of the AMI configuration. If this is specified, the instance is stopped when the timeout expires. If the instance is required again, then the plugin will look for a stopped instance that exactly corresponds to the AMI specification and resume it if found. Otherwise a new instance is started. Having the instances be stopped instead of terminated is useful when you are using EBS volumes and want to keep them mounted for the life of the instance and reuse the instance for long periods of time. This can greatly reduce the startup time of the instance since it does not have to build the volume from the snapshot.

Spot Instances

Spot instances are similar to EC2 instances, except for a few key differences. They are generally more affordable, but they have the possibility of being terminated if someone has bid more on them than you have and can take longer to provision.  To mitigate some of these issues, your Jenkins and Slave AMIs will need a bit of configuration to convert the Spot slaves to register with Jenkins when they spawn up. Due to these changes, it may appear that a Spot slave will have failed (with a red X), but the message associated with the error will inform you if the Spot instance just has not called back yet or not. For more information on Spot instances, read the information here: http://aws.amazon.com/ec2/spot-instances/ .

Enable Spot Request

To enable use of Spot as opposed to EC2 for an instance, the "Use Spot Instance" check box must be checked.  Also, a bid price must be specified.  If you want to determine what the current price of the instance is without going to the AWS website, pick your region and instance type (as you already should) and click "Check Current Spot Price".
To choose between a Persistent or One Time bid (to keep the bid alive until cancelled or to stop the bid after it has been fulfilled once), choose from the drop down menu.

Configure Jenkins for Spot Support

For Jenkins, the major configuration change will be if you have a restrictive firewall, since these instances need to connect back to Jenkins.  The first configuration change to Jenkins is to change your Jenkins URL option in the Configure Jenkins page to be the external URL to the server.  One port that needs to be open is the one that you use to access Jenkins, the other is the JNLP port, which is generally randomly assigned.  To set the JNLP port to something predictable, follow the Connection Mechanism section on this page. Jenkins CLI

Configure AMI for Spot Support

In order for your EC2 instance to know that it is to be a Jenkins slave, it must be preconfigured with start up commands so that it can register itself with Jenkins.  The Jenkins information is passed to the Spot slaves via EC2 user-data.  This information includes the name that Jenkins has given the slave, and the configured URL for the Jenkins master node.  

Sample scripts for assisting in configuring an Ubuntu based AMI to work with the Jenkins ec2-plugin and Spot slaves are included with the installation of the plugin. 
Config Script:

(jenkins_server)/plugin/ec2/AMI-Scripts/ubuntu-ami-setup.sh

Startup Script:

(jenkins_server)/plugin/ec2/AMI-Scripts/ubuntu-init.py

The config script is run by the user from the EC2 instance with root access.  It installs Java onto the instance and fetches the startup script and sets it to run when the machine starts up.  It can be retrieved from the above url using a command like wget, or copied over using a tool like scp.

wget (jenkins_server)/plugin/ec2/AMI-Scripts/ubuntu-ami-setup.sh

Once the scripts have been downloaded, the script can be run. Running this script will also run the ubuntu-init.py script, so there is no need to run it on its own.

sudo sh ubuntu-ami-setup.sh jenkins_server{:port}

Note: ‘http://” will be prepended to the jenkins_server parameter

The config script then fetches the startup script and sets up the AMI to register itself with a Jenkins master node when it gets started.

After setting up the image, you can save the image using Amazon’s EC2 web console. To do this, right click on your instance from the console and select “Create Image (EBS AMI)”.

In order to set up additional images using other operating systems, you can configure your own startup script based on the startup script above.  This script should perform the following actions when the machine is started up:

# Verify that Java is installed

# Install Java if it is not installed

# Read user data for the EC2 instance. It is available from [http://169.254.169.254/latest/user-data]

# Values are passed in with the format of JENKINS_URL=[jenkins_url]&SLAVE_NAME=[slave_name]&USER_DATA=[other_user_data]

# Parse the values to retrieve the Jenkins_Url and Slave_name
# Fetch the slave.jar from the Jenkins master using wget (or something similar)

wget [Jenkins_Url]jnlpJars/slave.jar -O slave.jar
# Register the salve to the Jenkins master node

java -jar slave.jar -jnlpUrl [Jenkins_Url]computer/ slaveName slave-agent.jnlp

IAM setup

It's possible to create a separate account for jenkins using the Amazon IAM feature. Go to the IAM tab in AWS console and create a user. Then go to user's Permissions tab and use this policy (change username if your user is not called jenkins):

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Stmt1312295543082",
            "Action": [
                "ec2:DescribeSpotInstanceRequests",
                "ec2:CancelSpotInstanceRequests",
                "ec2:GetConsoleOutput",
                "ec2:RequestSpotInstances",
                "ec2:RunInstances",
                "ec2:StartInstances",
                "ec2:StopInstances",
                "ec2:TerminateInstances",
                "ec2:CreateTags",
                "ec2:DeleteTags",
                "ec2:DescribeInstances",
                "ec2:DescribeKeyPairs",
                "ec2:DescribeRegions",
                "ec2:DescribeImages",
                "ec2:DescribeAvailabilityZones",
                "ec2:DescribeSecurityGroups",
                "ec2:DescribeSubnets"
            ],
            "Effect": "Allow",
            "Resource": "*"
        }
    ]
}

If you want to launch slaves with IAM Instance Profile, "iam:PassRole" permission is required.

Issues with Authentication Timeout

If you have issues with authentication timing out as a node is starting (see the stack traces in JENKINS-30284) you can use these System Properties to adjust the number of tries and timeout values:

jenkins.ec2.bootstrapAuthSleepMs (default 30000)
jenkins.ec2.bootstrapAuthTries (default 30)

Configure plugin via Groovy script

Either automatically upon Jenkins post-initialization or through Jenkins script console, example:

import com.amazonaws.services.ec2.model.InstanceType
import com.cloudbees.jenkins.plugins.awscredentials.AWSCredentialsImpl
import com.cloudbees.plugins.credentials.*
import com.cloudbees.plugins.credentials.domains.Domain
import hudson.model.*
import hudson.plugins.ec2.AmazonEC2Cloud
import hudson.plugins.ec2.AMITypeData
import hudson.plugins.ec2.EC2Tag
import hudson.plugins.ec2.SlaveTemplate
import hudson.plugins.ec2.SpotConfiguration
import hudson.plugins.ec2.UnixData
import jenkins.model.Jenkins

// parameters
def SlaveTemplateUsEast1Parameters = [
  ami:                      'ami-AAAAAAAA',
  associatePublicIp:        false,
  connectBySSHProcess:      true,
  connectUsingPublicIp:     false,
  customDeviceMapping:      '',
  deleteRootOnTermination:  true,
  description:              'Jenkins slave EC2 US East 1',
  ebsOptimized:             false,
  iamInstanceProfile:       '',
  idleTerminationMinutes:   '5',
  initScript:               '',
  instanceCapStr:           '2',
  jvmopts:                  '',
  labelString:              'aws.ec2.us.east.jenkins.slave',
  launchTimeoutStr:         '',
  numExecutors:             '1',
  remoteAdmin:              'ec2-user',
  remoteFS:                 '',
  securityGroups:           'sg-11111111',
  stopOnTerminate:          false,
  subnetId:                 'subnet-SSSSSSSS',
  tags:                     new EC2Tag('Name', 'jenkins-slave'),
  tmpDir:                   '',
  type:                     't2.medium',
  useDedicatedTenancy:      false,
  useEphemeralDevices:      true,
  usePrivateDnsName:        true,
  userData:                 '',
  zone:                     'us-east-1a,us-east-1b'
]

def AmazonEC2CloudParameters = [
  cloudName:      'MyCompany',
  credentialsId:  'jenkins-aws-key',
  instanceCapStr: '2',
  privateKey:     '''-----BEGIN RSA PRIVATE KEY-----
MIIEogIBAAKCAQEAkjaolzS/3wlfDfQkaXyuBQ5VaodY9/0tm8BUdcfGmv7MQLUg
G12MNC5MLcR0Oj4NIOJZEZvH3afPmk5UZ1hZWRWupokSXDM3alNFoWzt0SKDTsig
nHrqouojRbTqZG8dlmAOHom3mTzj3OFG7PyLdsAEOcrVXwnqgBn7mDriXz+dLkFY
QctsJHXpm3XBNVaZ/Rmx7vAwFSg3XO0DkjRjR2mXadyh4yQX/bMqr1VEAYBBjGtZ
ri9OXAnoBMduNndeRTQ6i4xA2mBW7zcg4qksHUd+0jKN5t8cVxqOSOcCCztgwaXh
xEa9/SMIS+FR6NOrUQ+w0MxWsspHogWNWif9IQIDAQABAoIBAGLt9r5wY46OsbEo
SubRBJHJNAQjVhBdTtm+lacnp/rBggQaSYIalr3LwaGJ9jZeO9XPMQLYC7CvVamL
bFKMlc/P+t8VicW2pb0cYNWrdXH2zy+kUf/urwnSMFF+6sVP5Y4UqhkBh6d4apjO
GIZLDjYoD/PmiN6IQBGzAufql7ZntgUohYYkHM/ovskZSR6fSKXn91sirlmisfhE
/74kGfJF2+S/+HYtpcCgYkSYs/c0Ibzw8wEnNaCK+I0zn4Ppp53ip3hOiI3+0EVY
qnNisqL5yj8wjj1QFfwkVyWCtr7p0U4A4aDza35rxDKpZW/PcZNRK5pbLQzriqo5
J9DOQJ0CgYEA2HGwf+GjRMoJCcONjHKP8NJ3KoSBFj0ujJAxhIOyxJveMMS5agCH
94yNReZEppV7C/1fpcPb9GL38tfAb6VdGHOlFmq7djgkCKH+F7/PvDJ+u+1G871K
YtvEFlHT6IPUouEfSj+7/eRxZwNEuKkM2x4dOqPXbvKU63HJkxRFdz8CgYEArO89
WARI2+o82V3ldPEZAIfri+4HD0nYW7UY4hbExdyuYTKL619Wt1nr91ubCnpR5/1s
xfesBGYHlqsAuHi4tXCaU9aDyK9j+MnWUkDMvG5RXWzLDmrrfmFlohHc6r7HuVuR
gtVayj8izcZpXew6Vo3ENRdvfxCzT2V7OPnG058CgYAOb465CMCN7vepWgyPyHhH
NJJUGKBPbmczYs6aqvn6WPb5w7UmF8D5xrsJZXFAtwmM5CpD8QszgoJNBQzFpX7P
Ca+CDj5QhTAKD1vWE6n0QF3phMrNqNtUOpoabvy2Lky5TFB88EFGjrzthO9JbaT4
3EpQxeqxcKZ0CZPLJnf3mwKBgFu46IhufVZm/q8rpjBIUEJ/1Ob68LOjLyY0/2Wr
PeLUEYlsDdphTtUg1I71/12nUxoAyFiX7JzIoO3A9TjijtVtS+17sZoXrKagJxSp
We33dSBgO7MB8rWtYwJ7BvlbBwPBFYSXNPhgVE1gFzLBwI930cF3FKQIb5KE+L5X
fKVxAoGAcvNM9HpgtR3ngP7xWkeOWVkV6NDc2GbuYptbAMM7lY2DzG2Dbq1ru6iJ
n5CNoNomPrHA05Zx2e+DbmrDbJVowSlX5xJKbc3ttYsBZlqYdZmWllpG1np8snwd
I2vmggm6Uubt0s433SbMwgXonolPH0N7s8VdzVf0/moMUujYcE0=
-----END RSA PRIVATE KEY-----''',
  region: 'us-east-1',
  useInstanceProfileForCredentials: false
]

def AWSCredentialsImplParameters = [
  id:           'jenkins-aws-key',
  description:  'Jenkins AWS IAM key',
  accessKey:    '01234567890123456789',
  secretKey:    '01345645657987987987987987987987987987'
]

// https://github.com/jenkinsci/aws-credentials-plugin/blob/aws-credentials-1.23/src/main/java/com/cloudbees/jenkins/plugins/awscredentials/AWSCredentialsImpl.java
AWSCredentialsImpl aWSCredentialsImpl = new AWSCredentialsImpl(
  CredentialsScope.GLOBAL,
  AWSCredentialsImplParameters.id,
  AWSCredentialsImplParameters.accessKey,
  AWSCredentialsImplParameters.secretKey,
  AWSCredentialsImplParameters.description
)

// https://github.com/jenkinsci/ec2-plugin/blob/ec2-1.38/src/main/java/hudson/plugins/ec2/SlaveTemplate.java
SlaveTemplate slaveTemplateUsEast1 = new SlaveTemplate(
  SlaveTemplateUsEast1Parameters.ami,
  SlaveTemplateUsEast1Parameters.zone,
  null,
  SlaveTemplateUsEast1Parameters.securityGroups,
  SlaveTemplateUsEast1Parameters.remoteFS,
  InstanceType.fromValue(SlaveTemplateUsEast1Parameters.type),
  SlaveTemplateUsEast1Parameters.ebsOptimized,
  SlaveTemplateUsEast1Parameters.labelString,
  Node.Mode.NORMAL,
  SlaveTemplateUsEast1Parameters.description,
  SlaveTemplateUsEast1Parameters.initScript,
  SlaveTemplateUsEast1Parameters.tmpDir,
  SlaveTemplateUsEast1Parameters.userData,
  SlaveTemplateUsEast1Parameters.numExecutors,
  SlaveTemplateUsEast1Parameters.remoteAdmin,
  new UnixData(null, null, null),
  SlaveTemplateUsEast1Parameters.jvmopts,
  SlaveTemplateUsEast1Parameters.stopOnTerminate,
  SlaveTemplateUsEast1Parameters.subnetId,
  [SlaveTemplateUsEast1Parameters.tags],
  SlaveTemplateUsEast1Parameters.idleTerminationMinutes,
  SlaveTemplateUsEast1Parameters.usePrivateDnsName,
  SlaveTemplateUsEast1Parameters.instanceCapStr,
  SlaveTemplateUsEast1Parameters.iamInstanceProfile,
  SlaveTemplateUsEast1Parameters.deleteRootOnTermination,
  SlaveTemplateUsEast1Parameters.useEphemeralDevices,
  SlaveTemplateUsEast1Parameters.useDedicatedTenancy,
  SlaveTemplateUsEast1Parameters.launchTimeoutStr,
  SlaveTemplateUsEast1Parameters.associatePublicIp,
  SlaveTemplateUsEast1Parameters.customDeviceMapping,
  SlaveTemplateUsEast1Parameters.connectBySSHProcess,
  SlaveTemplateUsEast1Parameters.connectUsingPublicIp
)

// https://github.com/jenkinsci/ec2-plugin/blob/ec2-1.38/src/main/java/hudson/plugins/ec2/AmazonEC2Cloud.java
AmazonEC2Cloud amazonEC2Cloud = new AmazonEC2Cloud(
  AmazonEC2CloudParameters.cloudName,
  AmazonEC2CloudParameters.useInstanceProfileForCredentials,
  AmazonEC2CloudParameters.credentialsId,
  AmazonEC2CloudParameters.region,
  AmazonEC2CloudParameters.privateKey,
  AmazonEC2CloudParameters.instanceCapStr,
  [slaveTemplateUsEast1]
)

// get Jenkins instance
Jenkins jenkins = Jenkins.getInstance()

// get credentials domain
def domain = Domain.global()

// get credentials store
def store = jenkins.getExtensionList('com.cloudbees.plugins.credentials.SystemCredentialsProvider')[0].getStore()

// add credential to store
store.addCredentials(domain, aWSCredentialsImpl)

// add cloud configuration to Jenkins
jenkins.clouds.add(amazonEC2Cloud)

// save current Jenkins state to disk
jenkins.save()

 

Change Log

Version 1.39 (Mar 11, 2018)

  • JENKINS-47985 EC2 Plugin doesn't store AMITypeData in config.xml
  • JENKINS-46869 Can not register an EC2 instance as a node agent
  • JENKINS-47130 EC2 plugin 1.37 fails to provision previously defined slaves

Version 1.38 (Dec 6, 2017)

Version 1.37 (Sept 25, 2017)

  • This is a bad scary version due to: JENKINS-47130 (see above), please use 1.39 or higher instead
  • Warning:  This version introduces a bug ( JENKINS-47985 - Getting issue details... STATUS ) which drops stored passwords from Windows AMIs.  The fix has been merged to mainline and should hopefully be available when version 1.39 gets released.

Version 1.36 (Oct 2, 2016)

  • JENKINS-38481 AWS method change causes too many slaves to be launched

Version 1.33 (May 9, 2016)

  • JENKINS-34667 Provision attempt is made when possible slaves count is zero (backed out)

Version 1.32 (May 8, 2016)

Warning: Please use 1.33 instead, it has a critical fix.

  • Welcome Johnny Shields as co-maintainer!
  • JENKINS-26371 Ensure instance initiated shutdown behavior is consistent with stopOnTerminate flag
  • JENKINS-27529 Poll for spot instances instead of JNLP launcher
  • JENKINS-32588 Unable to launch linux slaves using ec2 plugin using Eucalyptus
  • JENKINS-32584 ec2-plugin counts every instance (not just Jenkins slave instances) when counting slaves; incorrectly hits slave cap
  • JENKINS-32690 Make manually provision really work
  • JENKINS-32915 Spot instance launched one after another until capacity reached for single task in queue
  • JENKINS-33945 can't provision new nodes when a matching node is marked offline
  • JENKINS-34667 Provision attempt is made when possible slaves count is zero
  • EC2UnixLauncher can survive Connection IOExceptions
  • Bootstrap retries with fresh connection
  • Many pmd/squid fixes
  • Remove AWS fault instance
  • Increase minimal supported version to 1.625.1 (to enable JDK7)
  • Use AWS credentials to manage the IAM access key
  • Applied logic to create block device mappings for both ondemand and spot slaves
  • With slaves now being fully controlled by Jenkins master, there is no longer a need to be able to set "persistent" spot bids

Version 1.31 (Jan 25, 2016)

  • JENKINS-32584 ec2-plugin counts every instance (not just Jenkins slave instances) when counting slaves; incorrectly hits slave cap

Version 1.30 (Jan 23, 2016)

  • Add config to prefer the public IP to private IP when ssh-ing into slave
  • Added common method to compute tag value and also created constants for demand and spot
  • JENKINS-27601 instance caps incorrectly calculated
  • JENKINS-23787 EC2-plugin not spooling up stopped nodes
  • Depend on the aws-java-sdk plugin to limit AWS SDK duplication
  • Upgrade AWS SDK to 1.10.26
  • Terminate instance even if ec2 node deletion failed
  • JENKINS-27260 SPNEGO for Windows in the EC2 Plugin
  • JENKINS-26493 Use new EC2 API endpoint hostnames
  • JCIFS first tries to resolve a dfs path would timeout causing a long startup delay
  • JENKINS-28754 Jenkins EC2 Plugin should show timestamp in slave logs
  • JENKINS-30284 EC2 plugin too aggressive in timing in contacting new AWS instance over SSH
  • Use AWS4SignerType instead of QueryStringSignerType
  • Add minimum timeout for windows launching
  • Better exception handling in uptime check
  • JENKINS-29851 Global instance cap not calculated for spot instances
  • JENKINS-32439 JENKINS-32439 Incorrect slave template (AMI) found when launching slave
  • Improve logging to be less verbose

Version 1.29 (Aug 2, 2015)

  • Modify sed command for Ubuntu to ignore comments
  • Add option to launch EBS optimized instances
  • Fix to not hide windows connection errors.
  • Fix confusing log message in windows launcher
  • Update AWS SDK to 1.10.0 (new instance types)
  • JENKINS-28268 - Allow ability to programmatically update AMI using Groovy

Version 1.28 (Jun 7, 2015)

  • JENKINS-26854 - Fixing 'RequestExpired'
  • Fixing instance profile credentials with 'Check AMI' and 'Check Current Spot Price' buttons
  • Masking BouncyCastle for when the system version is incompatible with the required version
  • Fix java download/install
  • Check more places where passing a null instanceID can do a search of random spot instances
  • WiP - Launch slave agent via ssh client process
  • JENKINS-24359 - Overcoming limit of one cloud per region.
  • JENKINS-27260 - SPNEGO for Windows in EC2 Plugin

Version 1.27 (Mar 30, 2015)

Version 1.26 (Jan 27, 2015)

Version 1.25 (Jan 20, 2015)

  • Make temporary directories configurable (JENKINS-26232)
  • Add immediate check for useInstanceProfileForCredentials option
  • Include new parameter in jelly's with-clauses, respect it in doFillRegions
  • Fixing zombie workers
  • Add option to obtain EC2 credentials from instance meta data service
  • Retry updating remote tags also for on-demand instances
  • Support cloud-formation compatible user-data in spot-instance slaves
  • Handle AssociatePublicIp in spot instance slaves
  • Consider only slaves for the requested label when decrementing excess Workload by number of pending spot instance slaves
  • Fix the stopping/pending message when launching a slave
  • Fix bug that caused infinite loop of NullPointerExceptions during provisioning of windows instances.
  • Fix connection to Windows slaves outside us-east-1. (JENKINS-19943)
  • Give the user a readable error message when testing the connection instead of stack trace if their credentials are incorrect. (JENKINS-24676)
  • EC2 plugin incorrectly reports current instance count (JENKINS-19845)
  • fix EC2SpotSlaves from fetching tags from a completely different instance

Version 1.24 (Aug 5, 2014)

  •  

Version 1.23 (Jun 30, 2014)

  •  Fixed critical problems in 1.22.

Version 1.22 (Jun 27, 2014)

  •  This was a bad release, not recommended. Use 1.23 instead.

Version 1.21 (Feb 28, 2014)

  •  

Version 1.20 (Nov 8, 2013)

Version 1.19 (Oct 8, 2013)

Version 1.18 (April 9, 2013)

  • Add m3.xlarge and m3.2xlarge instance types
  • Failure starting slave nodes (JENKINS-15319)
  • Tags feature is broken (JENKINS-15239)
  • EC2 Nodes which share an AMI ID get the wrong labels (JENKINS-7690)
  • Sometimes starts the wrong instance (JENKINS-15158)
  • Stopped (as opposed to terminated) slaves are counted against the active instance count for the purpose of launching; can prevent launching of instances (JENKINS-7883)
  • Upgrade aws-java-sdk dependency to 1.3.30
  • Explicitly add MIT license to all plugin code
  • Fallback a manual or timeout-based terminate to stop if terminate fails (to avoid charges)
  • Give Jenkins nodes useful names (JENKINS-15078)
  • Keep track of instances being provisioned; use this count when determining total/AMI instance caps (JENKINS-6691)
  • Bring back remoteFS in the slave configuration page
  • Let user configure node.mode for EC2 slaves

Version 1.17 (September 12, 2012)

  • Resume stopped EC2 instances (JENKINS-14884)
  • Added support for EC2 tags, VPC subnets/security groups
  • Added support for public/private DNS
  • EC2 documentation for VPC/security groups not clear (JENKINS-15149)

Version 1.16 (May 26, 2012)

Version 1.15 (May 21, 2012)

  • Stopped (as opposed to terminated) slaves are counted against the active instance count for the purpose of launching; can prevent launching of instance (JENKINS-7883)
  • Clarification and updating of help (JENKINS-12789)
  • The init script was called each time instance was connected to (JENKINS-12771)
  • EC2 slaves fail to launch when using versions prior to 1.9 (JENKINS-7219)
  • Force registration of converter (JENKINS-10118)
  • Convert to Amazon EC2 libraries(JENKINS-12539)
  • Allow non-root user name (JENKINS-5867)
  • Allow specification of security group (JENKINS-8617)
  • Add support for M1 Medium instance (JENKINS-13432)
  • Allow instances to be stopped (instead of terminated (JENKINS-12772)
  • Option to set zone as well as region for instance (JENKINS-8946)

Version 1.14 (Feb 22, 2012)

  • Fixed a typo in Tokyo region name, and added Oregon and Sao Paulo regions.

Version 1.13 (Jul 29, 2011)

Version 1.12 (Jul 19, 2011)

  • Label expressions are handled correctly (JENKINS-9773)
  • Fixed a false-positive "check AMI" validation error (JENKINS-9415)

Version 1.11 (Mar 15, 2011)

  • Fixed a bug in the form validation (JENKINS-6063)
  • Reuse the client for better resource usage (pull request).
  • SSH port is now configurable
  • Eucalyptus doesn't report the IP address right away, so we need to keep checking. (JENKINS-5851)
  • Added new Tokyo region.

Version 1.10 (Nov 7 2010)

  • Added APAC region support.

Version 1.9 (Aug 11 2010)

  • Fixed bug in backwards-compatibility of persisted configuration (JENKINS-6782)

Version 1.8 (Apr 09 2010)

  • Fixed the fatal problem in configuration persistence (JENKINS-6113)

Version 1.7 (Mar 17 2010)

Version 1.6 (Feb 15 2010)

Version 1.5 (Nov 20 2009)

  • Number of executors wasn't properly persisted (JENKINS-4906)

Version 1.4 (Aug 5 2009)

  • Jenkins does several retries on SSH logins to give sshd extra time to initialize itself (JENKINS-4119)

Version 1.3 (July 27 2009)

  • User-data can be now specified to the launching instances (JENKINS-4115)
  • # of executors can be now configured (JENKINS-4116)

Version 1.2 (Jun 18 2009)

  • Fixed the dependency issue with recent Jenkins

Version 1.1 (May 28 2009)

  • Re-implemented the instance cap so that it uses AWS API for # of instances to prevent run-away EC2 instances.
  • Improved the performance of instance launches
  • Fixed a bug where a long init script may cause Jenkins to start additional instances.
  • If the init script has run once, don't run it again when reconnecting.

Version 1.0

  • Initial release

Developer Information

EC2 plugin build

84 Comments

  1. So I am using this plugin, and I have a tag applied to the instances ( 32bit ).  I have one job which I want to run on this instance, which I have checked the "restrict build" box in the config, and taged it correctly.  All of that is working great, however now I am noticing OTHER jobs are running on this instance as well.  Shouldnt all other jobs run on the master ?  How can I make sure NOTHING runs on this slave but jobs that are tagged to do so ? 

    -John

    <edit> 

      I got this working by tagging all my builds, guess this was required.

    1. Is this the expected behaviour? I have not actually started using it yet but wanted to read as much as possible before giving it a go.

      1. Yes, I had to tag all my jobs when using this

  2. This plugin should have an option like in other slaves on usage "leave to tied jobs only" or "use as much as possible"

  3. Any idea when this 1.18 is going to be released ?  The wiki is updated but the latest release is still 1.18 ?

    1. And of course... it just updated :P

  4. Ok, so the "latest version" on this page show 1.18 but Jenkins auto update still things 1.17 is the latest.  Any idea how long it takes for that to be updated? 

  5. Hi all. I just installed this, and when I tried setting it up, it failed getting the Region. Looking at the Jenkins error log, I see it has the following error in many exceptions:

    ...Exception: Unable to execute HTTP request: Connection to http://ec2.amazonaws.com refused

    Looking at the URL, I tried using curl -vv to it and got the following: 

    [root@ip-10-147-175-47 ~]# curl -vv http://ec2.amazonaws.com/
    * About to connect() to ec2.amazonaws.com port 80 (#0)
    *   Trying 72.21.215.33... connected
    * Connected to ec2.amazonaws.com (72.21.215.33) port 80 (#0)
    > GET / HTTP/1.1
    > User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.14.3.0 zlib/1.2.3 libidn/1.18 libssh2/1.4.2
    > Host: ec2.amazonaws.com
    > Accept: */*
    >
    < HTTP/1.1 301 Moved Permanently
    < Location: http://aws.amazon.com/ec2
    < Content-Length: 0
    < Date: Tue, 07 May 2013 12:13:44 GMT
    < Server: AmazonEC2
    <
    * Connection #0 to host ec2.amazonaws.com left intact
    * Closing connection #0

    Does anyone have an idea?

    1. Further investigation shows it's related to the proxy. I'm still failing when I try switching between my proxies, but I still have to use one of them. Is there a special proxy setup to support this plugin?

  6. If you want to manage the risk associated with placing the access and secret keys in Jenkins you can create an AWS IAM group with the following policies and create a new user in this group and use the access keys of this user instead of those of a more powerful user.

    {
      "Statement": [
        {
          "Action": [
            "ec2:DescribeRegions",
            "ec2:DescribeKeyPairs",
            "ec2:DescribeImages",
            "ec2:StartInstances",
            "ec2:StopInstances",
            "ec2:TerminateInstances"
          ],
          "Effect": "Allow",
          "Resource": [
            "*"
          ]
        }
      ]
    }
    
    1. Are you sure these permissions are sufficient? Jenkins fails when trying to list available regions with this user in the current version.

    2. Are you sure these permissions are sufficient? Jenkins fails when trying to list available regions with this user in the current version.

  7. I've been adding missing permissions and I think I've got them all sorted now.  This is what I've been using:

    {
      "Statement": [
        {
          "Action": [
            "ec2:DescribeRegions"
            "ec2:DescribeInstances",
            "ec2:DescribeImages",
            "ec2:DescribeKeyPairs",
            "ec2:DescribeSubnets",
            "ec2:DescribeSecurityGroups",
            "ec2:DescribeAvailabilityZones",
            "ec2:CancelSpotInstanceRequests",
            "ec2:GetConsoleOutput",
            "ec2:RequestSpotInstances",
            "ec2:RunInstances",
            "ec2:StartInstances",
            "ec2:StopInstances",
            "ec2:TerminateInstances",
            "ec2:CreateTags",
            "ec2:DescribeTags"
          ],
          "Effect": "Allow",
          "Resource": "*"
        }
      ]
    }
  8. Hi

    Are there any plans to add support for Windows EC2 slaves?

    Thanks,
    Tomas

  9. I am currently running Jenkins in a multi-product based dev shop and am trying to associate billing charges from EC2 instance usage in AWS back to specific product teams.

    Is there a currently available solution available to provide this mapping? Currently my only visibility is a list of servers with unique tag IDs utilized in AWS over the billing period.

    Thanks,

    Brandon

    1. Brandon,

         You can add a labels to the instances and then use AWS Cost Management  to bill each to your respective cost centers.   Hope that helps.

      -John

  10. Generating a key from the GUI fails with a javascript error (tested with Chrome and IE):

  11. Has the Windows instance type provisioning ever worked?  The code is there but I can't seem to connect to WinRM.  I've made sure WinRM is up and running and verified I can connect using PS from another instance.

  12. I see you have to paste the private rsa key in plain text in the configuration. Isn't this insecure? Is there any way to just pass the location of the private key in the ~/.ssh/ directory ?

  13. we are using this plugin. We have a case. After creating jenkins slave  by root we need to su to another user. Let's call the user agent-user. 

    agent-user is used to share NFS files in many instances. 

    Now there is no such option "agent user". So we need to add such command like su agent-user -c 'command'

  14. I have configured everything and looks like it is working. When I run the job, it starts a new instance and executes the job on it.

    What I do not understand is when it'll start new instances. According to the above documentation, it should start new instances when it detects that the first instance is overloaded.

    I have created a jenkins job that start a Ubuntu instance and run "stress" command on it that occupies the CPU, Memory and HDDs. I run this job 10 times with different parameters. The instance is using all of its resources but still, the plugin, isn't starting a new instance. 

    I have configured another AMI in the Jenkins configuration and added it to the same Label. Still nothing. Jenkins/EC2 plugin is using only one instance.

    Can someone explain how should it work or what am I doing wrong?

    Thanks

    Gil

    1. I wasn't aware that "instance cap" parameter on main cloud configuration and on AMI configuration hadn't stated the same meaning.

      The main cloud configuration means what I was looking for, how many simultaneous instances to start. The one in the AMI configuration means what is the maximum number of instances to start.

      Thanks

      Gil

      1. Hi Gil,

          How did you get it worked. I ran into same issue. Running couple of jobs with the same label. Whenever I start the job first it able to spin up the instance and able start job. Whenever all executors are busy and new job came up EC2 plugin didn't adding new slave to the cluster.

        What I did:

        Created same label for two AMIs. Updated Main cloud configuration instance cap to 10 and AMI instance cap to 3 for the two AMIS.

        Architecture:

        Master + 4 static slaves (not created with ec2 plugin) + 2 ec2 dynamic slaves running(generated with 2 different labels)

        Can you please help me on this. 

      2. I'm also having a similar issue. However, I set the "instance cap" parameter to 50 at the Cloud level and the AMI level. I keep getting this error: "java.lang.Exception: Cloud or AMI instance cap would be exceeded for:"

        Right now, it's not even launching one instance due to this error. Anyone know what I'm missing?

        1. Me too! Something has broken in a recent plugin update.

  15. It would be nice to have an option to use EC2 Private IP address (EC2UnixLauncher.java => private String getEC2HostAddress) not only private DNS. 

    Right now it is not possible to access EC2 instance with public IP enabled, when private and public DNS name not visible/accessible, but private IP address is visible and accessible. The EC2 plugin can be configured to use either public DNS or Private DNS for such configuration that make it impossible to connect to instance.

    1. Uncheck the option Use Private DNS and appears to be picking the private IP.

  16. I am using v1.33 and in configuration page "Amazon EC2 Crendentials" list is empty, although I have added some AWS credentials and verified connection.

    Running recently installed Jenkins 2.6, almost clean. Am I missing some plugins/dependencies ?

  17. Hello, 

    I'm using v1.33. I use it for Spot instances creation.  I have Instance Cap setting 50.
    From time to time I get the following exception, when try to create a new Spot instance:

    java.lang.Exception: Cloud or AMI instance cap would be exceeded for: EC2 Spot Builder at org.kohsuke.stapler.HttpResponses.error(HttpResponses.java:83)

    As I see the number of spot instances is much less than 50 ...

    Please advise.

    EC2 plugin version I use is 1.33.Jenkins version is 1.609.3.
    Thanks,Roman

  18. Hello, 

    I'm using v1.33. I use it for Spot instances creation.  I have Instance Cap setting 50.
    From time to time I get the following exception, when try to create a new Spot instance:

    java.lang.Exception: Cloud or AMI instance cap would be exceeded for: EC2 Spot Builder at org.kohsuke.stapler.HttpResponses.error(HttpResponses.java:83)

    As I see the number of spot instances is much less than 50 ...

    Please advise.

    EC2 plugin version I use is 1.33.Jenkins version is 1.609.3.
    Thanks,Roman

  19. So I'm trying to make this work with Windows slaves that have an ssh server enabled (Cygwin ssh service).

    When I select the "Unix" slave type, mostly everything works; except when the slave.jar is being started. 
    E.g. when I set "/tmp" as temp dir, the command run on the slave is "java  -jar /tmp/slave.jar". While Cygwin understands the file path syntax, java on Windows does not, so it tells me "/tmp/slave.jar" is not found.

    Any idea on how to make it work with the existing plugin?

    I guess I could modify the plugin for myself, to do "cd /tmp; java -jar slave.jar" which seems to work better (Cygwin shell understands the "/tmp", and java does not need to load a file with path separators).

  20. Hi,

    I am using this plugin to launch EC2 (windows) slaves for a job. My job has 1700 parallel builds and I want to 100 instances to be spawned(Instance cap is 100).

    The plugin is able to launch new EC2 machines. However, the rate of spawning new machines is very slow. A new instance is spawned every 90 seconds. Thus invoking 100 instances takes 9000 seconds=2.5 hours+. As mentioned in the description of this plugin if Jenkins notices that your build cluster is overloaded, it'll start instances using the EC2 API

    The job takes only 1 hour to complete if these 100 instances are already running.

    How can I increase the EC2 spawn rate. Or, how do I decrease the interval of launching new EC2 instances? Kindly help with the same.

    Thanks and Regards

    1. Hi Vinay,

      I am Prabhu and i am trying to configure the windows slaves getting WinRM error, would you kindly give me the steps you followed to configure the windows slaves ?

       

      1. Window AMI is not supported at this moment (see  JENKINS-4995 - Getting issue details... STATUS )

    2. Explanation: EC2 plugin tries to avoid provisioning extra resources until proven required, so this plugin would not provide a new instance in EC2 (billed per hour) in case Jenkins just had to wait a few seconds more to get some build resources released so the internal logic is based on previous build history and such a guess can be wrong.

      Please note EC2 will announce pricing model change, will bill by the second

      Ref: https://aws.amazon.com/ec2/pricing/on-demand/

  21. First of all great plugin. We have been using the plugin the past year and i works great.

    We now have a fairly advanced setup and need to modify two AMI ID's (programmatically from a slave building a new AMI), which by the plugin is stored in the jenkins_home/config.xml on the master. The problem is that jenkins-cli.jar and Jenkins REST API do not provide access to the root config.xml. Apparently the only way to modify the config.xml is to log on to the master and manipulate the configuration from there. It would be great if the plugin could extend the jenkins-cli to allow remote manipulation of the AMI ID.

    According to the Jenkins documentation "Plugins installed on Jenkins server can add custom CLI commands". We were hoping you would implement such feature.

    Thanks.

  22. Hi,

    This plugin has been working very well for me in the past and I've been using it to provision new ec2 build nodes at build request. I was running Jenkins 1.631 (and this plugin at 1.29) but now after I upgraded to Jenkins 2.46.2 (and this plugin at 1.36) this plugin doesn't work anymore and the credentials disappeared (I backed up my JENKINS_HOME). Now, when I want to configure the plugin again, every time I add my AWS keypair credentials it won't save it; the pulldown menu always shows - none - no matter what I add in the credentials page.

    1. Replying to myself; I've resolved the issue.

      What I did wrong: Upgraded Jenkins from 1.631 to 2.46.2 in one shot.

      How I resolved it: Upgraded incrementally from 1.631 to 1.651.3 first. Then upgraded to 2.7.1, then 2.19.1, then 2.32.1, etc.

      Hope this was helpful.

  23. Hi

    Is there a way to set a specific elastic ip to the slave?

  24. I just noticed that when a slave is launched via this plugin, then slave.jar gets copied to /tmp, instead of the directory specified as the "remote FS root". 

    First of all, should this be considered a bug? On most Linux distros, the contents of /tmp get cleaned up on a regular basis. (The default for RedHat/CentOS is to delete files that haven't been accessed in 10 days, with some exceptions.) Putting stuff in /tmp, and expecting it to stick around indefinitely, is a bad practice. Was there a reason for doing it like this?

    Second, is there a way to override this behavior, and force the jar to be stored in the Jenkins home (which is what happens with normal slaves)?

  25. I am trying to configure the windows slave, i can see the new slave is spinned up, but keep seeing this message, would you kindly help me with a solution ?

    Waiting for WinRM to come up. Sleeping 10s.
    Connecting to ec2-windows-host.compute-1.amazonaws.com(<<host ip>>) with WinRM as Administrator....
    Waiting for WinRM to come up. Sleeping 10s.

     

    1. I'm also trying to configure for windows. I see the EC2 instance spin up and I have user data to set up winrm, but I get the same endless loop. Did you have any progress for this, or does anyone know a resolution?

      I am able to connect to the slave via RDP, and with powershell "Test-WsMan" and "Invoke-Command" by passing the credential for the administrator.

      My user data:

      <script>
      winrm quickconfig -q && winrm set winrm/config/client/Auth @{Basic="true"} && winrm set winrm/config/service/Auth @{Basic="true"} && winrm set winrm/config/service @{AllowUnencrypted="true"} && winrm set winrm/config/winrs @{MaxMemoryPerShellMB="1024"} && netsh advfirewall firewall add rule name="WinRM-HTTP" dir=in localport=5985 protocol=TCP action=allow
      </script>
      <persist>true</persist>
      1. My issue was that I also needed to open the port for 445.

        netsh advfirewall firewall add rule name="SMB-HTTP" dir=in localport=445 protocol=TCP action=allow
        1. Hi David,

          Are you now able to run windows slaves? I'm having the same looping winrm problem.

          Rob

  26. Is there any way to specify multiple subnet IDs in the config? I'm having a problem launching nodes if I enter more than one, so am currently restricted to a single availability zone. 

    1. A very important issue, it is possible to start spot instance in cheapest subnet from aws api - https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/spot-fleet-examples.html#fleet-config2
      but it is impossible to do the same thing with this plugin (sad)

      when I try to make several records with the same label in different subnets - always starts the first record

  27. Hi Chris, I've created several AMIs for this purpose

  28. Hi,

    I am a new one start to use this plugin and came into an issue..

    After I input informations in jenkins plugin, I tried to click "Provision via Test" in Nodes and got below error:

    --------------------------------------------------------------------------------------------------------------------------------------------------

    com.amazonaws.services.ec2.model.AmazonEC2Exception: Value (arn:aws:iam::XXXXXXXXXXXX:role/AXXX-FullAdmin) for parameter iamInstanceProfile.arn is invalid. Invalid IAM Instance Profile ARN (Service: AmazonEC2; Status Code: 400; Error Code: InvalidParameterValue; Request ID: 9387cf47-9137-41c0-9aea-652bc4063a7e)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1639)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1304)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1056)

    So I got back to plugin config:

    And I think currently I input correct IAM Instance Profile as I copied from AWS->Roles→Summary Role ARN:

     

    Can you pls help me?

     

  29. Please pick up the latest commits.  We need to use this plugin to create parallel slaves ASAP. thank you.

  30. I'm trying to setup with the block device mapping and having some issues. Is EBS Optimized required for the device mapping? When I enable that I get this error

    EBS-optimized instances are not supported for your requested configuration

    I'm wanting to have a 100 GB EBS root volume, and so have the mapping set to

    /dev/xvda=:100:true:gp2

    1. I figured it out. EBS-optimized is unrelated, and the error was because I'm using t2.medium instance types. To map a custom sized root volume I needed to use device /dev/sda1.

  31. I'm trying to use EC2 plugin with IAM role, assigned to the Jenkins master.

    Launching new slave fails with:

     

    com.amazonaws.services.ec2.model.AmazonEC2Exception: You are not authorized to perform this operation. Encoded authorization failure message: lHuEfanNqJBznhhoWAOq6YC7LpXGrt6Ibhb
    cOuEs5ZA7mvLQ2iSZOewOEERjC6VVwV25yLBoNWf5PO9On2ybgaAobi8tFdOmhsj9cnZh0t7IBk7KZ-KChEb-VOjg7jZsfPh9JI6IZe_hqUqrkKO5gm-YnKO707mUtWtEr7qUWPwrQDrQ4Vi0ZdblOO2__HgElb0TuZaqJeqcJpNv9L2P
    YIZ6OihP5fKZGWSYAVRmaUaM3y_6aIR2QIEOjCuJ1uhZvwrT00x_Ge_kA2a1b2j_rONS9ZtE8UNTP5qMnqMkQYea_yGEQFfHxsJ9liVd03_eIP_-bQ19kGI2ZVdmVumxXKW5F6sRYdW3_wozCBmp2e41e7zsPiaBODojdxwrXN2MVnJQI
    7ikWZW99hQ7ApuJrUWPuK8mX54FWjl_l5SiyLaXpdOLChTP_QrwvNUt1wngiBTG1pcXbdIH0IfxQV11h1RWs0XgRJkPGfESUiZhFWCp_30g8exSdahhRjUCSPcSsQIFR5hUQapbP9zpqW7fragkqna_Q4aEyWf4sr8 (Service: Amaz
    onEC2; Status Code: 403; Error Code: UnauthorizedOperation; Request ID: 2bf1b6c4-4376-4ddf-9c90-ff325979d6a4)

    even I'm able to launch new EC2 using the same params using AWS CLI:

     

    aws ec2 run-instances --image-id ami-XXXXXX  --subnet-id  subnet-XXXX --security-group-ids sg-XXXX  --key-name jenkins-ci-slave --instance-type c4.large

    Any ideas?

    TIA,

    Vitaly

    1. Solved: it was permission issue related to launching AMI with encrypted volumes.

  32. Currently I am not seeing anything in the cloud statistics plugin? Is the EC2 plugin hook up to that?

  33. The /usr/bin/userdata script is outdated. openjdk-7-jre package is no longer available to download on Ubuntu Xenial. Changing it to openjdk-8-jre manually fixed the problem.

  34. A warning to anyone running Windows instances with this plugin:  The current version of this plugin (1.38) is not safe to use as it drops the stored Windows machine passwords whenever the Jenkins master is restarted:
    JENKINS-47985 - Getting issue details... STATUS

    The bug appears to have been introduced to the codeline in the 1.37 release.  There is a fix pending, but it has not been acted on since Nov.  For now, I suggest not upgrading beyond 1.36 if you use Windows instances.

  35. I have a need to bind a specific network interface to an EC2 instance to complete a job (due to a node-locked license).  I don't see a way to bind a network interface to a slave.  Is there a trick to doing this?

    1. Hi kevin connors,

      you can assign Role to a slave which has a policy with attach network interface permission, after that in init script you need to install awscli package and attach network interface during instance initialization (wink)

      1. Perfect.  Works great.  Thanks.

  36. First-off - fantastic plugin, thanks! 

    I'm using this together with the CloudBees Amazon Web Services Credentials Plugin, using an assume-role to manage the EC2 slave instances. This works initially to communicate with EC2 and manage slaves, but as the credentials provided by the STS for assuming the role are only temporary - between 15mins and 1hour (default) - after the credentials expire, the plugin is no longer able to authenticate with AWS, and therefore fails to manage slaves. It looks like the plugin only assigns the credentials on initialisation, and doesn't request new credentials on expiry, causing this problem. Manually going to the Jenkins → Manage Jenkins → Configure System UI page, and just saving that page seems to re-initialise the credentials, causing the plugin to work until the new credentials expire. 

    Interested to hear if anyone else is experiencing this issue, has a workaround (I looked at a managed script to re-save the configuration and force new credentials, but couldn't seem to get this working), or if there is a plan to address this? Will also post an issue on Github and dive into the code when I get the time.

  37. I have a few types of slaves, but userdata is almost the same.

    Is there a way for using some file content as userdata? 

  38. I am using version 1.38 of the plugin. I create cloud and template definitions using the API and then I check them by running the script at the end of the comment which shows that the cloud and template configurations were created correctly for the given label. Then I try to run a pipeline that makes use of the template and I get Jenkins doesn’t have label jenkins-docker-agent-useast1-t2small

     

    A restart of Jenkins fixes the problem which makes me think that this is some sort of internal cache issue.

     

    Is there a way of forcing a cache refresh after doing a Jenkins.instance.save()?

     

    def label = "jenkins-docker-agent-useast1-t2small"
    def l = hudson.model.labels.LabelAtom.get(label)
    Jenkins.instance.clouds.each {
      def cloud = it
      println "Cloud name: " + cloud.displayName
      println "can provision label with name " + label + ": " + cloud.canProvision(l)
      it.getTemplates().each {
        def template = it
        println "Template name: "+template.displayName
        it.labelSet.each {
          print "Label " + it.toString() + " matches " + label + ": "
          println it.toString().equalsIgnoreCase(label)
        }
      }
    }
     
    1. Actually this is not a good solution. When I do this then the pipeline downstream doesn't pick up the newly created template. The solution was to get a reference to the current cloud, create a new ArrayList based on cloud.getTemplates, add the new template to this list, create a new cloud duplicating the values of the old one and passing the new templates list to the constructor, deleting the old cloud from the clouds list, adding the new cloud, and then saving the instance

      1. Can you share your code? This is similar to what I'm trying to accomplish. I'm struggling with writing a script that can execute on a regular (e.g. daily) basis that will:

        access the list of AWS AMIs that have a certain tag (e.g. jenkins-slaves-*)

        Loop through the existing configuration and add any AMIs as new slaves if they are not yet configured in Jenkins.

        Modify configurations of existing slaves if they are currently configured in Jenkins (e.g. AMI ID has changed).

        Although I am very competent in bash and python, I've never created a Groovy script and am struggling with some of these tasks so any hints you can offer would be greatly appreciated.

  39. The scripts and documentation regarding "SLAVE_NAME" have been invalidated by commit ae7e53ab99028fd8ede1a3e41907812c2edb0eec 

    SLAVE_NAME is still referred to in the ubuntu-init.py and this article, but there are no other references to that in the code, and the userdata seems to now be just a text field that defaults to empty string. (I tried aimlessly for an hour looking at the code and experiment if there was some secret I was missing.)

    Since the slave name is no longer exposed in userdata, what are you supposed to do to replace slaveName when you call the following?

     

    java -jar slave.jar -jnlpUrl [Jenkins_Url]computer/ slaveName slave-agent.jnlp
    
    
    

    I'm happy to help out fixing either the scripts or this document if it will prevent someone else from banging their head for 3 hours.

  40. Hi,

    I am using the plugin (v1.39) and am struggling with it being able to create more than 1 EC2 instance. I have configured the global instance cap as empty and then instance cap for the AMI to be 5 and set the number of executors to 1. I have set a label value and also added some tags.

    When I execute a build that keeps the execution running on the node i.e. by having an input step, and then run another build that requires the same AWS instance type the plugin does not start another AWS instance but instead just waits for the first job to finish.

    I have looked at the code and there seems to be a flag available to force a new instance to be created but I am unable to set that anywhere in the plugin configuration but I should not have to as I want the number of instances to scale up to the instance cap I have set.

    Any help would be greatly appreciated.

    Thanks.

  41. Hi,

    Is this documentation out of date? My scenario is this:

    • I want to run the slave on a docker container on the EC2 instance spawned by this plugin (No, I don't want to use ECS).
    • To achieve the above, I need to use a custom init script with


    The problem is that nothing is passed through via user data (like it states should be on this page) so I can't get the agent name or the Jenkins URL. The URL isn't too much of a problem because I can autodiscover it. Same with the agent name.

     

    I do need the agent secret though, which would be a nice addition.

     

    Thanks

    ---

    Edit: As Patrick Santos pointed out above, this was removed by https://github.com/jenkinsci/ec2-plugin/commit/ae7e53ab99028fd8ede1a3e41907812c2edb0eec. Seems weird to take away that functionality.

     

    1. Currently I get the slave name by this hackery by reverse-enginnering the name of "description (instance id)", and use a "user" API key instead of a secret, since as you mentioned, the secret isn't passed easily anywhere from the code that I could find.

       
      #! /bin/bash
      # Stop on error
      set -e
      # Grab user data - see https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html#instancedata-add-user-data
      AWS_INSTANCE_ID=$(curl http://169.254.169.254/latest/meta-data/instance-id)
      # Replace with whatever you put as the 'description' for the Jenkins AMI config, as this is parsed into the name of the Jenkins slave
      JENKINS_AMI_DESCRIPTION=mybuilder
      SLAVE_NAME="${JENKINS_AMI_DESCRIPTION} ($AWS_INSTANCE_ID)"
      echo Slave name is "${SLAVE_NAME}"
      sudo yum install -y perl-URI
      SLAVE_URI=$(perl -MURI::Escape -e 'print uri_escape($ARGV[0])' "$SLAVE_NAME")

       

      Instead of the secret I use a user authentication token as environment variable API_TOKEN, so my launch command then looks like this.

       
      java -jar agent.jar -jnlpUrl "${JENKINS_URL}/computer/${SLAVE_URI}/slave-agent.jnlp" -jnlpCredentials $API_TOKEN -workDir "${AGENT_WORKDIR}"

      Where JENKINS_URL is the the URL to access the Jenkins master, SLAVE_URI is the URL-encoded slave name, API_TOKEN is an authentication token for my Jenkins "user" that is dedicated for initializing the node, and AGENT_WORKDIR is some folder I want the slave to use.

       

  42. Hey, 

     

    I am seeing some inconsistent behavior with this plugin (version 1.39). 

    When there are no workers up and I try to run a job, a worker is brought online. 

    But when the only workers that is up has been marked offline due to the disk space being full, the job just sits in the cue, and no worker is brought online.

     

    Alos I am curious to know if I can boot an EC2 worker from the CLI or the API using the EC2 plugin? 

    What I would like to do is poll the number of executors available and start to scale up when there is a certain percentage of free executors like 20% of total for example. The main reason to do this because an EC2 instance take a long time to come up and I want the Jobs to not have to wait. 

     

    Thank you

  43. Is this plugin currently working with Windows instances? I'm having issues with the looping WinRM 10 seconds sleep.

  44. Does EC2 plugin support AWS Workspaces (https://aws.amazon.com/workspaces/) ?

    Can we create on-demand aws workspaces using EC2 plugin ?

    If there is any documentation available on the same, please share it.

  45. Hello, we use your plugin and it works well. Thanks you for that.

    But we wanted to reuse a little more the EC2 instances created, can we prevent the EC2 instance to terminate at the end of its use of a node ? maybe the EC2 can stick around for a copple of minuts, to be reused when another pipeline want a node?

    1. sure - it's default behaviour.

      you have two params to play with:

      • idle time - how many minutes EC2 instance will wait before shutdown
      • shutdown/terminate - define policy for EC2 stopping
      1. Thx for your repply, so as you said in your doc, to confirm, I need to configure this with a groovy script ? 

        1. No, just in the plugin settings menu.

  46. I am using this plugin to spawn dynamic Jenkins Windows EC2 instances for doing automation tests (GUI)
    But the problem is that when I login to EC2 instances, I do not see the browser screenshots.
    I understand since we are using WinRM as Slaves, its difficult to see the GUI. Any other options to see the GUI tests initiated by Jenkins on Windows Ec2?
    Do we have any plugin to provision dynamic Slaves using JNLP?

  47. The plugin seems to require an ssh private key but I don't see where to specify the corresponding EC2 Key pair to attach to instances at launch time.

    Is there a way to specify? How does the public key get added to newly provisioned slaves?

    1. Figured it out. Had to read a separate doc that informed me to create a new keypair named jenkins

  48. Hi,

         We are using this plugin for launching the Windows Slaves ,they get launched alright ,but the actual builds running on these slaves becomes painfully slow,for example the svn checkout which takes around 30 seconds through on static slave now takes more than 15 minutes,this may be due the WinRM service as transport,which may be slow.Do we have the option to switch to JNLP connection ?

    Using Winrm service is making this plugin useless for windows or  Im doing something wrong?

    Thanks 

     

  49. Hi,

    Is it possible to not have to put the SSH private key in plain text but instead to refer to a SSH key from Jenkins Credentials ?

    I am currently using the Jenkins Configuration-as-Code plugin and having to put the entire key in my YAML file is a bit problematic as this configuration file will be in GIT... I think that a reference to an existing Jenkins Credentials key would help here.

    Do you think it is feasible ?

    Thanks

    1. Guillaume Dupin you probably should rather send that kind of requests on the users ML (https://jenkins.io/mailing-lists/) for much more audience. Or, even a JIRA ticket would likely make sense here for this pretty clear (and IMO useful) use case.

      I agree with you it would be really nice to integrate with the credentials plugin. I actually handled that case very recently in Jenkins Essentials. The configuration-as-code plugin supports variables, and can automatically source them from various locations, included Hashicorp Vault, or Docker Secrets. I used the latter in https://github.com/jenkins-infra/evergreen/blob/master/distribution/environments/aws-ec2-cloud/config/as-code/ec2-cloud.yaml, and passed it under /run/secrets/PRIVATE_KEY to make it available at startup, in https://github.com/jenkins-infra/evergreen/blob/master/distribution/environments/aws-ec2-cloud/CloudFormation/cloudformation-template.json#L59.

      Hope this helps.

  50. Baptiste Mathus ok thanks for the advice and I will open a JIRA ticket to propose it.

    Thank you for the examples ! One point that is not clear for me tough : the content of the "privateKey" parameter in your example seems to be directly the key whereas when I tested it on my jenkins instance, the content apparently needs to be the encrypted form of the key (when I look at the config.xml file, it is something like this : "<privateKey>{AQAA...w==}</privateKey>".

    And when I set it in my jenkins.yml for JCasC, I had to put this encrypted form instead of the real key content. It is not very clear for me if I must deal with keys directly or with their "encrypted-by-master" form... Because in this case, it will be difficult to move the credentials from one instance to another (I will have to move also the "secrets" Jenkins master directory...)

    1. Here is the JIRA ticket :  JENKINS-52547 - Getting issue details... STATUS

      UPDATE : for my JCasC configuration, both are working : plain text and master-encrypted private key (although I am not sure if it is normal that the encrypted-form works too...)

  51. An instance of the slave is unstable and I am in trouble.
    How do I pass "-text" to slave.jar?

Write a comment…