Due to some maintenance issues, this service has been switched in read-only mode, you can find more information about the why

and how to migrate your plugin documentation in this blogpost

Skip to end of metadata
Go to start of metadata

Allow Jenkins to start agents on EC2 or Eucalyptus on demand, and kill them as they get unused.

With this plugin, if Jenkins notices that your build cluster is overloaded, it'll start instances using the EC2 API and automatically connect them as Jenkins agents. When the load goes down, excess EC2 instances will be terminated. This set up allows you to maintain a small in-house cluster, then spill the spiky build/test loads into EC2 or another EC2 compatible cloud.


Usage

First, go to EC2 and sign up for the service. Once you've installed the plugin, you navigate to the main "Manage Jenkins" > "Configure System" page, and scroll down near the bottom to the "Cloud" section. There, you click the "Add a new cloud" button, and select the "Amazon EC2" option. This will display the UI for configuring the EC2 plugin.  Then enter the Access Key and Secret Access Key which act like a username/password (see IAM section). Because of the way EC2 works, you also need to have an RSA private key that the cloud has the other half for, to permit sshing into the instances that are started. If you have already been using EC2 and have your own key, you can paste it here. Otherwise, you can have Jenkins generate one. If you let Jenkins generate one, you should save this private key in your file system as well, as you'll need this to interactively logon to EC2 instances.

Once you have put in your Access Key and Secret Access Key, select a region for the cloud (not shown in screenshot). You may define only one cloud for each region, and the regions offered in the UI will show only the regions that you don't already have clouds defined for them.

Use "Test Connection" button to verify that Jenkins can successfully talk to EC2. If you are using UEC you need to click on Advanced and fill out the endpoint details for your cluster.

Next, configure AMIs that you want to launch. For this, you need to find the AMI IDs for the OS of your choice. Packer is a good tool for doing that. Jenkins can work with any Unix AMIs. If using an Ubuntu EC2 or UEC AMI you need to fill out the rootCommandPrefix and remoteAdmin fields under advanced. Windows is currently somewhat supported.

Configuring labels allow Jenkins to pick the right AMI to start. For example, if all your existing agents labeled "solaris" are fully busy and you have more builds that are tied to the "solaris" label, Jenkins will start the AMIs that have the "solaris" label.

Init script is the shell script to be run on the newly launched EC2 instance, before Jenkins starts launching a agent agent. If the AMI doesn't have Java pre-installed, you can do this in the init script. This is also a good place to install additional packages that you need for your builds and tests.  The init script is located at /tmp/init.sh and is owned and run by the user account specified in the "Remote User" field (so use of "sudo" may be required for non-root accounts).

Configure several AMIs if you need different OS images.

With this setting, your Jenkins will automatically start instances when the load goes up, and the instances will be terminated (or stopped - see below) automatically if it's idle for more than 30 mins.

By default, instances are terminated when the idle timeout period expires. You can change this by specifying the Stop/Disconnect on Idle Timeout flat in the Advanced properties of the AMI configuration. If this is specified, the instance is stopped when the timeout expires. If the instance is required again, then the plugin will look for a stopped instance that exactly corresponds to the AMI specification and resume it if found. Otherwise a new instance is started. Having the instances be stopped instead of terminated is useful when you are using EBS volumes and want to keep them mounted for the life of the instance and reuse the instance for long periods of time. This can greatly reduce the startup time of the instance since it does not have to build the volume from the snapshot.

Spot Instances

Spot instances are similar to EC2 instances, except for a few key differences. They are generally more affordable, but they have the possibility of being terminated if someone has bid more on them than you have and can take longer to provision.  To mitigate some of these issues, your Jenkins and Agent AMIs will need a bit of configuration to convert the Spot agents to register with Jenkins when they spawn up. Due to these changes, it may appear that a Spot agent will have failed (with a red X), but the message associated with the error will inform you if the Spot instance just has not called back yet or not. For more information on Spot instances, read the information here: http://aws.amazon.com/ec2/spot-instances/ .

Enable Spot Request

To enable use of Spot as opposed to EC2 for an instance, the "Use Spot Instance" check box must be checked.  Also, a bid price must be specified.  If you want to determine what the current price of the instance is without going to the AWS website, pick your region and instance type (as you already should) and click "Check Current Spot Price".
To choose between a Persistent or One Time bid (to keep the bid alive until cancelled or to stop the bid after it has been fulfilled once), choose from the drop down menu.

Configure Jenkins for Spot Support

For Jenkins, the major configuration change will be if you have a restrictive firewall, since these instances need to connect back to Jenkins.  The first configuration change to Jenkins is to change your Jenkins URL option in the Configure Jenkins page to be the external URL to the server.  One port that needs to be open is the one that you use to access Jenkins, the other is the JNLP port, which is generally randomly assigned.  To set the JNLP port to something predictable, follow the Connection Mechanism section on this page. Jenkins CLI

Configure AMI for Spot Support

In order for your EC2 instance to know that it is to be a Jenkins agent, it must be preconfigured with start up commands so that it can register itself with Jenkins.  The Jenkins information is passed to the Spot agents via EC2 user-data.  This information includes the name that Jenkins has given the agent, and the configured URL for the Jenkins master node.  

Sample scripts for assisting in configuring an Ubuntu based AMI to work with the Jenkins ec2-plugin and Spot agents are included with the installation of the plugin. 
Config Script:

(jenkins_server)/plugin/ec2/AMI-Scripts/ubuntu-ami-setup.sh

Startup Script:

(jenkins_server)/plugin/ec2/AMI-Scripts/ubuntu-init.py

The config script is run by the user from the EC2 instance with root access.  It installs Java onto the instance and fetches the startup script and sets it to run when the machine starts up.  It can be retrieved from the above url using a command like wget, or copied over using a tool like scp.

wget (jenkins_server)/plugin/ec2/AMI-Scripts/ubuntu-ami-setup.sh

Once the scripts have been downloaded, the script can be run. Running this script will also run the ubuntu-init.py script, so there is no need to run it on its own.

sudo sh ubuntu-ami-setup.sh jenkins_server{:port}

Note: ‘http://” will be prepended to the jenkins_server parameter

The config script then fetches the startup script and sets up the AMI to register itself with a Jenkins master node when it gets started.

After setting up the image, you can save the image using Amazon’s EC2 web console. To do this, right click on your instance from the console and select “Create Image (EBS AMI)”.

In order to set up additional images using other operating systems, you can configure your own startup script based on the startup script above.  This script should perform the following actions when the machine is started up:

# Verify that Java is installed

# Install Java if it is not installed

# Read user data for the EC2 instance. It is available from [http://169.254.169.254/latest/user-data]

# Values are passed in with the format of JENKINS_URL=[Jenkins_Url]&SLAVE_NAME=[Agent_Name]&USER_DATA=[other_user_data]

# Parse the values to retrieve the Jenkins_Url and Agent_Name
# Fetch the agent.jar from the Jenkins master using wget (or something similar)

wget [Jenkins_Url]jnlpJars/agent.jar -O agent.jar
# Register the agent to the Jenkins master node

java -jar agent.jar -jnlpUrl [Jenkins_Url]computer/ [Agent_Name] slave-agent.jnlp

IAM setup

It's possible to create a separate account for jenkins using the Amazon IAM feature. Go to the IAM tab in AWS console and create a user. Then go to user's Permissions tab and use this policy (change username if your user is not called jenkins):

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Stmt1312295543082",
            "Action": [
                "ec2:DescribeSpotInstanceRequests",
                "ec2:CancelSpotInstanceRequests",
                "ec2:GetConsoleOutput",
                "ec2:RequestSpotInstances",
                "ec2:RunInstances",
                "ec2:StartInstances",
                "ec2:StopInstances",
                "ec2:TerminateInstances",
                "ec2:CreateTags",
                "ec2:DeleteTags",
                "ec2:DescribeInstances",
                "ec2:DescribeKeyPairs",
                "ec2:DescribeRegions",
                "ec2:DescribeImages",
                "ec2:DescribeAvailabilityZones",
                "ec2:DescribeSecurityGroups",
                "ec2:DescribeSubnets",
				"iam:ListInstanceProfilesForRole",
				"iam:PassRole",
				"ec2:GetPasswordData"
            ],
            "Effect": "Allow",
            "Resource": "*"
        }
    ]
}

If you want to launch agents with IAM Instance Profile, "iam:PassRole" permission is required.

If you want to launch Windows agents and use the generated Administrator password, the "ec2:GetPasswordData" permission is also required.

Configure plugin via Groovy script

Either automatically upon Jenkins post-initialization or through Jenkins script console, example:

import com.amazonaws.services.ec2.model.InstanceType
import com.cloudbees.jenkins.plugins.awscredentials.AWSCredentialsImpl
import com.cloudbees.plugins.credentials.*
import com.cloudbees.plugins.credentials.domains.Domain
import hudson.model.*
import hudson.plugins.ec2.AmazonEC2Cloud
import hudson.plugins.ec2.AMITypeData
import hudson.plugins.ec2.EC2Tag
import hudson.plugins.ec2.SlaveTemplate
import hudson.plugins.ec2.SpotConfiguration
import hudson.plugins.ec2.UnixData
import jenkins.model.Jenkins

// parameters
def SlaveTemplateUsEast1Parameters = [
  ami:                      'ami-AAAAAAAA',
  associatePublicIp:        false,
  connectBySSHProcess:      true,
  connectUsingPublicIp:     false,
  customDeviceMapping:      '',
  deleteRootOnTermination:  true,
  description:              'Jenkins agent EC2 US East 1',
  ebsOptimized:             false,
  iamInstanceProfile:       '',
  idleTerminationMinutes:   '5',
  initScript:               '',
  instanceCapStr:           '2',
  jvmopts:                  '',
  labelString:              'aws.ec2.us.east.jenkins.slave',
  launchTimeoutStr:         '',
  numExecutors:             '1',
  remoteAdmin:              'ec2-user',
  remoteFS:                 '',
  securityGroups:           'sg-11111111',
  stopOnTerminate:          false,
  subnetId:                 'subnet-SSSSSSSS',
  tags:                     new EC2Tag('Name', 'jenkins-slave'),
  tmpDir:                   '',
  type:                     't2.medium',
  useDedicatedTenancy:      false,
  useEphemeralDevices:      true,
  usePrivateDnsName:        true,
  userData:                 '',
  zone:                     'us-east-1a,us-east-1b'
]

def AmazonEC2CloudParameters = [
  cloudName:      'MyCompany',
  credentialsId:  'jenkins-aws-key',
  instanceCapStr: '2',
  privateKey:     '''-----BEGIN RSA PRIVATE KEY-----
MIIEogIBAAKCAQEAkjaolzS/3wlfDfQkaXyuBQ5VaodY9/0tm8BUdcfGmv7MQLUg
G12MNC5MLcR0Oj4NIOJZEZvH3afPmk5UZ1hZWRWupokSXDM3alNFoWzt0SKDTsig
nHrqouojRbTqZG8dlmAOHom3mTzj3OFG7PyLdsAEOcrVXwnqgBn7mDriXz+dLkFY
QctsJHXpm3XBNVaZ/Rmx7vAwFSg3XO0DkjRjR2mXadyh4yQX/bMqr1VEAYBBjGtZ
ri9OXAnoBMduNndeRTQ6i4xA2mBW7zcg4qksHUd+0jKN5t8cVxqOSOcCCztgwaXh
xEa9/SMIS+FR6NOrUQ+w0MxWsspHogWNWif9IQIDAQABAoIBAGLt9r5wY46OsbEo
SubRBJHJNAQjVhBdTtm+lacnp/rBggQaSYIalr3LwaGJ9jZeO9XPMQLYC7CvVamL
bFKMlc/P+t8VicW2pb0cYNWrdXH2zy+kUf/urwnSMFF+6sVP5Y4UqhkBh6d4apjO
GIZLDjYoD/PmiN6IQBGzAufql7ZntgUohYYkHM/ovskZSR6fSKXn91sirlmisfhE
/74kGfJF2+S/+HYtpcCgYkSYs/c0Ibzw8wEnNaCK+I0zn4Ppp53ip3hOiI3+0EVY
qnNisqL5yj8wjj1QFfwkVyWCtr7p0U4A4aDza35rxDKpZW/PcZNRK5pbLQzriqo5
J9DOQJ0CgYEA2HGwf+GjRMoJCcONjHKP8NJ3KoSBFj0ujJAxhIOyxJveMMS5agCH
94yNReZEppV7C/1fpcPb9GL38tfAb6VdGHOlFmq7djgkCKH+F7/PvDJ+u+1G871K
YtvEFlHT6IPUouEfSj+7/eRxZwNEuKkM2x4dOqPXbvKU63HJkxRFdz8CgYEArO89
WARI2+o82V3ldPEZAIfri+4HD0nYW7UY4hbExdyuYTKL619Wt1nr91ubCnpR5/1s
xfesBGYHlqsAuHi4tXCaU9aDyK9j+MnWUkDMvG5RXWzLDmrrfmFlohHc6r7HuVuR
gtVayj8izcZpXew6Vo3ENRdvfxCzT2V7OPnG058CgYAOb465CMCN7vepWgyPyHhH
NJJUGKBPbmczYs6aqvn6WPb5w7UmF8D5xrsJZXFAtwmM5CpD8QszgoJNBQzFpX7P
Ca+CDj5QhTAKD1vWE6n0QF3phMrNqNtUOpoabvy2Lky5TFB88EFGjrzthO9JbaT4
3EpQxeqxcKZ0CZPLJnf3mwKBgFu46IhufVZm/q8rpjBIUEJ/1Ob68LOjLyY0/2Wr
PeLUEYlsDdphTtUg1I71/12nUxoAyFiX7JzIoO3A9TjijtVtS+17sZoXrKagJxSp
We33dSBgO7MB8rWtYwJ7BvlbBwPBFYSXNPhgVE1gFzLBwI930cF3FKQIb5KE+L5X
fKVxAoGAcvNM9HpgtR3ngP7xWkeOWVkV6NDc2GbuYptbAMM7lY2DzG2Dbq1ru6iJ
n5CNoNomPrHA05Zx2e+DbmrDbJVowSlX5xJKbc3ttYsBZlqYdZmWllpG1np8snwd
I2vmggm6Uubt0s433SbMwgXonolPH0N7s8VdzVf0/moMUujYcE0=
-----END RSA PRIVATE KEY-----''',
  region: 'us-east-1',
  useInstanceProfileForCredentials: false
]

def AWSCredentialsImplParameters = [
  id:           'jenkins-aws-key',
  description:  'Jenkins AWS IAM key',
  accessKey:    '01234567890123456789',
  secretKey:    '01345645657987987987987987987987987987'
]

// https://github.com/jenkinsci/aws-credentials-plugin/blob/aws-credentials-1.23/src/main/java/com/cloudbees/jenkins/plugins/awscredentials/AWSCredentialsImpl.java
AWSCredentialsImpl aWSCredentialsImpl = new AWSCredentialsImpl(
  CredentialsScope.GLOBAL,
  AWSCredentialsImplParameters.id,
  AWSCredentialsImplParameters.accessKey,
  AWSCredentialsImplParameters.secretKey,
  AWSCredentialsImplParameters.description
)

// https://github.com/jenkinsci/ec2-plugin/blob/ec2-1.38/src/main/java/hudson/plugins/ec2/SlaveTemplate.java
SlaveTemplate slaveTemplateUsEast1 = new SlaveTemplate(
  SlaveTemplateUsEast1Parameters.ami,
  SlaveTemplateUsEast1Parameters.zone,
  null,
  SlaveTemplateUsEast1Parameters.securityGroups,
  SlaveTemplateUsEast1Parameters.remoteFS,
  InstanceType.fromValue(SlaveTemplateUsEast1Parameters.type),
  SlaveTemplateUsEast1Parameters.ebsOptimized,
  SlaveTemplateUsEast1Parameters.labelString,
  Node.Mode.NORMAL,
  SlaveTemplateUsEast1Parameters.description,
  SlaveTemplateUsEast1Parameters.initScript,
  SlaveTemplateUsEast1Parameters.tmpDir,
  SlaveTemplateUsEast1Parameters.userData,
  SlaveTemplateUsEast1Parameters.numExecutors,
  SlaveTemplateUsEast1Parameters.remoteAdmin,
  new UnixData(null, null, null),
  SlaveTemplateUsEast1Parameters.jvmopts,
  SlaveTemplateUsEast1Parameters.stopOnTerminate,
  SlaveTemplateUsEast1Parameters.subnetId,
  [SlaveTemplateUsEast1Parameters.tags],
  SlaveTemplateUsEast1Parameters.idleTerminationMinutes,
  SlaveTemplateUsEast1Parameters.usePrivateDnsName,
  SlaveTemplateUsEast1Parameters.instanceCapStr,
  SlaveTemplateUsEast1Parameters.iamInstanceProfile,
  SlaveTemplateUsEast1Parameters.deleteRootOnTermination,
  SlaveTemplateUsEast1Parameters.useEphemeralDevices,
  SlaveTemplateUsEast1Parameters.useDedicatedTenancy,
  SlaveTemplateUsEast1Parameters.launchTimeoutStr,
  SlaveTemplateUsEast1Parameters.associatePublicIp,
  SlaveTemplateUsEast1Parameters.customDeviceMapping,
  SlaveTemplateUsEast1Parameters.connectBySSHProcess,
  SlaveTemplateUsEast1Parameters.connectUsingPublicIp
)

// https://github.com/jenkinsci/ec2-plugin/blob/ec2-1.38/src/main/java/hudson/plugins/ec2/AmazonEC2Cloud.java
AmazonEC2Cloud amazonEC2Cloud = new AmazonEC2Cloud(
  AmazonEC2CloudParameters.cloudName,
  AmazonEC2CloudParameters.useInstanceProfileForCredentials,
  AmazonEC2CloudParameters.credentialsId,
  AmazonEC2CloudParameters.region,
  AmazonEC2CloudParameters.privateKey,
  AmazonEC2CloudParameters.instanceCapStr,
  [slaveTemplateUsEast1]
)

// get Jenkins instance
Jenkins jenkins = Jenkins.getInstance()

// get credentials domain
def domain = Domain.global()

// get credentials store
def store = jenkins.getExtensionList('com.cloudbees.plugins.credentials.SystemCredentialsProvider')[0].getStore()

// add credential to store
store.addCredentials(domain, aWSCredentialsImpl)

// add cloud configuration to Jenkins
jenkins.clouds.add(amazonEC2Cloud)

// save current Jenkins state to disk
jenkins.save()


Known Issues

Authentication Timeout

If you have issues with authentication timing out as a node is starting (see the stack traces in JENKINS-30284) you can use these System Properties to adjust the number of tries and timeout values:

jenkins.ec2.bootstrapAuthSleepMs (default 30000)
jenkins.ec2.bootstrapAuthTries (default 30)

Amazon Linux build/connectivity issues

If you are using a Amazon Linux AMI and encounter exceptions like java.io.FileNotFoundException: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.201.b09-0.amzn2.x86_64/jre/lib/currency.data (No such file or directory) or Remote call on EC2 [...] failed then chances are that the Amazon Linux is doing some security upgrades in the background and causes the slave to be in an invalid state.

From the documentation of the Amazon Linux it's stated:

Amazon Linux is configured to download and install security updates at launch time.[...] The default setting for repo_upgrade is security. That is, if you don't specify a different value in your user data, by default, Amazon Linux performs the security upgrades at launch for any packages installed at that time

This issue can be solved in different ways:

  1. Update your AMI and execute the security fixes => this will mean however that the issue could eventually come back
  2. Update your AMI and disable auto update of security fixes (see amazon documentation)
  3. Force security upgrade through the "init-script" in the Cloud configuration

Change Log

Located in the sub page here

Developer Information

EC2 plugin build

96 Comments

  1. Unknown User (johntdyer)

    So I am using this plugin, and I have a tag applied to the instances ( 32bit ).  I have one job which I want to run on this instance, which I have checked the "restrict build" box in the config, and taged it correctly.  All of that is working great, however now I am noticing OTHER jobs are running on this instance as well.  Shouldnt all other jobs run on the master ?  How can I make sure NOTHING runs on this slave but jobs that are tagged to do so ? 

    -John

    <edit> 

      I got this working by tagging all my builds, guess this was required.

    1. Unknown User (outspoken)

      Is this the expected behaviour? I have not actually started using it yet but wanted to read as much as possible before giving it a go.

      1. Unknown User (johntdyer)

        Yes, I had to tag all my jobs when using this

  2. Unknown User (josesa)

    This plugin should have an option like in other slaves on usage "leave to tied jobs only" or "use as much as possible"

  3. Unknown User (johntdyer)

    Any idea when this 1.18 is going to be released ?  The wiki is updated but the latest release is still 1.18 ?

    1. Unknown User (johntdyer)

      And of course... it just updated :P

  4. Unknown User (johntdyer)

    Ok, so the "latest version" on this page show 1.18 but Jenkins auto update still things 1.17 is the latest.  Any idea how long it takes for that to be updated? 

  5. Unknown User (eldada)

    Hi all. I just installed this, and when I tried setting it up, it failed getting the Region. Looking at the Jenkins error log, I see it has the following error in many exceptions:

    ...Exception: Unable to execute HTTP request: Connection to http://ec2.amazonaws.com refused

    Looking at the URL, I tried using curl -vv to it and got the following: 

    [root@ip-10-147-175-47 ~]# curl -vv http://ec2.amazonaws.com/
    * About to connect() to ec2.amazonaws.com port 80 (#0)
    *   Trying 72.21.215.33... connected
    * Connected to ec2.amazonaws.com (72.21.215.33) port 80 (#0)
    > GET / HTTP/1.1
    > User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.14.3.0 zlib/1.2.3 libidn/1.18 libssh2/1.4.2
    > Host: ec2.amazonaws.com
    > Accept: */*
    >
    < HTTP/1.1 301 Moved Permanently
    < Location: http://aws.amazon.com/ec2
    < Content-Length: 0
    < Date: Tue, 07 May 2013 12:13:44 GMT
    < Server: AmazonEC2
    <
    * Connection #0 to host ec2.amazonaws.com left intact
    * Closing connection #0

    Does anyone have an idea?

    1. Unknown User (eldada)

      Further investigation shows it's related to the proxy. I'm still failing when I try switching between my proxies, but I still have to use one of them. Is there a special proxy setup to support this plugin?

  6. Unknown User (ximon18)

    If you want to manage the risk associated with placing the access and secret keys in Jenkins you can create an AWS IAM group with the following policies and create a new user in this group and use the access keys of this user instead of those of a more powerful user.

    {
      "Statement": [
        {
          "Action": [
            "ec2:DescribeRegions",
            "ec2:DescribeKeyPairs",
            "ec2:DescribeImages",
            "ec2:StartInstances",
            "ec2:StopInstances",
            "ec2:TerminateInstances"
          ],
          "Effect": "Allow",
          "Resource": [
            "*"
          ]
        }
      ]
    }
    
    1. Unknown User (lucasvo)

      Are you sure these permissions are sufficient? Jenkins fails when trying to list available regions with this user in the current version.

    2. Unknown User (lucasvo)

      Are you sure these permissions are sufficient? Jenkins fails when trying to list available regions with this user in the current version.

  7. Unknown User (zpatten)

    I've been adding missing permissions and I think I've got them all sorted now.  This is what I've been using:

    {
      "Statement": [
        {
          "Action": [
            "ec2:DescribeRegions"
            "ec2:DescribeInstances",
            "ec2:DescribeImages",
            "ec2:DescribeKeyPairs",
            "ec2:DescribeSubnets",
            "ec2:DescribeSecurityGroups",
            "ec2:DescribeAvailabilityZones",
            "ec2:CancelSpotInstanceRequests",
            "ec2:GetConsoleOutput",
            "ec2:RequestSpotInstances",
            "ec2:RunInstances",
            "ec2:StartInstances",
            "ec2:StopInstances",
            "ec2:TerminateInstances",
            "ec2:CreateTags",
            "ec2:DescribeTags"
          ],
          "Effect": "Allow",
          "Resource": "*"
        }
      ]
    }
  8. Unknown User (tomasv)

    Hi

    Are there any plans to add support for Windows EC2 slaves?

    Thanks,
    Tomas

  9. Unknown User (btrebbien)

    I am currently running Jenkins in a multi-product based dev shop and am trying to associate billing charges from EC2 instance usage in AWS back to specific product teams.

    Is there a currently available solution available to provide this mapping? Currently my only visibility is a list of servers with unique tag IDs utilized in AWS over the billing period.

    Thanks,

    Brandon

    1. Unknown User (johntdyer)

      Brandon,

         You can add a labels to the instances and then use AWS Cost Management  to bill each to your respective cost centers.   Hope that helps.

      -John

  10. Unknown User (ochedru)

    Generating a key from the GUI fails with a javascript error (tested with Chrome and IE):

  11. Unknown User (mattfirtioncoveros)

    Has the Windows instance type provisioning ever worked?  The code is there but I can't seem to connect to WinRM.  I've made sure WinRM is up and running and verified I can connect using PS from another instance.

  12. Unknown User (smccarthy)

    I see you have to paste the private rsa key in plain text in the configuration. Isn't this insecure? Is there any way to just pass the location of the private key in the ~/.ssh/ directory ?

  13. Unknown User (yongtzha)

    we are using this plugin. We have a case. After creating jenkins slave  by root we need to su to another user. Let's call the user agent-user. 

    agent-user is used to share NFS files in many instances. 

    Now there is no such option "agent user". So we need to add such command like su agent-user -c 'command'

  14. Unknown User (gils)

    I have configured everything and looks like it is working. When I run the job, it starts a new instance and executes the job on it.

    What I do not understand is when it'll start new instances. According to the above documentation, it should start new instances when it detects that the first instance is overloaded.

    I have created a jenkins job that start a Ubuntu instance and run "stress" command on it that occupies the CPU, Memory and HDDs. I run this job 10 times with different parameters. The instance is using all of its resources but still, the plugin, isn't starting a new instance. 

    I have configured another AMI in the Jenkins configuration and added it to the same Label. Still nothing. Jenkins/EC2 plugin is using only one instance.

    Can someone explain how should it work or what am I doing wrong?

    Thanks

    Gil

    1. Unknown User (gils)

      I wasn't aware that "instance cap" parameter on main cloud configuration and on AMI configuration hadn't stated the same meaning.

      The main cloud configuration means what I was looking for, how many simultaneous instances to start. The one in the AMI configuration means what is the maximum number of instances to start.

      Thanks

      Gil

      1. Unknown User (jogipraveen123)

        Hi Gil,

          How did you get it worked. I ran into same issue. Running couple of jobs with the same label. Whenever I start the job first it able to spin up the instance and able start job. Whenever all executors are busy and new job came up EC2 plugin didn't adding new slave to the cluster.

        What I did:

        Created same label for two AMIs. Updated Main cloud configuration instance cap to 10 and AMI instance cap to 3 for the two AMIS.

        Architecture:

        Master + 4 static slaves (not created with ec2 plugin) + 2 ec2 dynamic slaves running(generated with 2 different labels)

        Can you please help me on this. 

      2. Unknown User (cuongkle)

        I'm also having a similar issue. However, I set the "instance cap" parameter to 50 at the Cloud level and the AMI level. I keep getting this error: "java.lang.Exception: Cloud or AMI instance cap would be exceeded for:"

        Right now, it's not even launching one instance due to this error. Anyone know what I'm missing?

        1. Unknown User (neiltingley)

          Me too! Something has broken in a recent plugin update.

  15. Unknown User (alex_bleep)

    It would be nice to have an option to use EC2 Private IP address (EC2UnixLauncher.java => private String getEC2HostAddress) not only private DNS. 

    Right now it is not possible to access EC2 instance with public IP enabled, when private and public DNS name not visible/accessible, but private IP address is visible and accessible. The EC2 plugin can be configured to use either public DNS or Private DNS for such configuration that make it impossible to connect to instance.

    1. Unknown User (vrenjith)

      Uncheck the option Use Private DNS and appears to be picking the private IP.

  16. Unknown User (cyril)

    I am using v1.33 and in configuration page "Amazon EC2 Crendentials" list is empty, although I have added some AWS credentials and verified connection.

    Running recently installed Jenkins 2.6, almost clean. Am I missing some plugins/dependencies ?

  17. Unknown User (romang_1)

    Hello, 

    I'm using v1.33. I use it for Spot instances creation.  I have Instance Cap setting 50.
    From time to time I get the following exception, when try to create a new Spot instance:

    java.lang.Exception: Cloud or AMI instance cap would be exceeded for: EC2 Spot Builder at org.kohsuke.stapler.HttpResponses.error(HttpResponses.java:83)

    As I see the number of spot instances is much less than 50 ...

    Please advise.

    EC2 plugin version I use is 1.33.Jenkins version is 1.609.3.
    Thanks,Roman

  18. Unknown User (romang_1)

    Hello, 

    I'm using v1.33. I use it for Spot instances creation.  I have Instance Cap setting 50.
    From time to time I get the following exception, when try to create a new Spot instance:

    java.lang.Exception: Cloud or AMI instance cap would be exceeded for: EC2 Spot Builder at org.kohsuke.stapler.HttpResponses.error(HttpResponses.java:83)

    As I see the number of spot instances is much less than 50 ...

    Please advise.

    EC2 plugin version I use is 1.33.Jenkins version is 1.609.3.
    Thanks,Roman

  19. Unknown User (schniedergers)

    So I'm trying to make this work with Windows slaves that have an ssh server enabled (Cygwin ssh service).

    When I select the "Unix" slave type, mostly everything works; except when the slave.jar is being started. 
    E.g. when I set "/tmp" as temp dir, the command run on the slave is "java  -jar /tmp/slave.jar". While Cygwin understands the file path syntax, java on Windows does not, so it tells me "/tmp/slave.jar" is not found.

    Any idea on how to make it work with the existing plugin?

    I guess I could modify the plugin for myself, to do "cd /tmp; java -jar slave.jar" which seems to work better (Cygwin shell understands the "/tmp", and java does not need to load a file with path separators).

  20. Unknown User (vintrojan)

    Hi,

    I am using this plugin to launch EC2 (windows) slaves for a job. My job has 1700 parallel builds and I want to 100 instances to be spawned(Instance cap is 100).

    The plugin is able to launch new EC2 machines. However, the rate of spawning new machines is very slow. A new instance is spawned every 90 seconds. Thus invoking 100 instances takes 9000 seconds=2.5 hours+. As mentioned in the description of this plugin if Jenkins notices that your build cluster is overloaded, it'll start instances using the EC2 API

    The job takes only 1 hour to complete if these 100 instances are already running.

    How can I increase the EC2 spawn rate. Or, how do I decrease the interval of launching new EC2 instances? Kindly help with the same.

    Thanks and Regards

    1. Unknown User (thangapr81)

      Hi Vinay,

      I am Prabhu and i am trying to configure the windows slaves getting WinRM error, would you kindly give me the steps you followed to configure the windows slaves ?

       

      1. Unknown User (carlosrodlop)

        Window AMI is not supported at this moment (see  JENKINS-4995 - Getting issue details... STATUS )

    2. Unknown User (carlosrodlop)

      Explanation: EC2 plugin tries to avoid provisioning extra resources until proven required, so this plugin would not provide a new instance in EC2 (billed per hour) in case Jenkins just had to wait a few seconds more to get some build resources released so the internal logic is based on previous build history and such a guess can be wrong.

      Please note EC2 will announce pricing model change, will bill by the second

      Ref: https://aws.amazon.com/ec2/pricing/on-demand/

  21. Unknown User (freddy_kruger)

    First of all great plugin. We have been using the plugin the past year and i works great.

    We now have a fairly advanced setup and need to modify two AMI ID's (programmatically from a slave building a new AMI), which by the plugin is stored in the jenkins_home/config.xml on the master. The problem is that jenkins-cli.jar and Jenkins REST API do not provide access to the root config.xml. Apparently the only way to modify the config.xml is to log on to the master and manipulate the configuration from there. It would be great if the plugin could extend the jenkins-cli to allow remote manipulation of the AMI ID.

    According to the Jenkins documentation "Plugins installed on Jenkins server can add custom CLI commands". We were hoping you would implement such feature.

    Thanks.

  22. Unknown User (aribiross_1)

    Hi,

    This plugin has been working very well for me in the past and I've been using it to provision new ec2 build nodes at build request. I was running Jenkins 1.631 (and this plugin at 1.29) but now after I upgraded to Jenkins 2.46.2 (and this plugin at 1.36) this plugin doesn't work anymore and the credentials disappeared (I backed up my JENKINS_HOME). Now, when I want to configure the plugin again, every time I add my AWS keypair credentials it won't save it; the pulldown menu always shows - none - no matter what I add in the credentials page.

    1. Unknown User (aribiross_1)

      Replying to myself; I've resolved the issue.

      What I did wrong: Upgraded Jenkins from 1.631 to 2.46.2 in one shot.

      How I resolved it: Upgraded incrementally from 1.631 to 1.651.3 first. Then upgraded to 2.7.1, then 2.19.1, then 2.32.1, etc.

      Hope this was helpful.

  23. Unknown User (gilk)

    Hi

    Is there a way to set a specific elastic ip to the slave?

  24. Unknown User (mike_baranczak)

    I just noticed that when a slave is launched via this plugin, then slave.jar gets copied to /tmp, instead of the directory specified as the "remote FS root". 

    First of all, should this be considered a bug? On most Linux distros, the contents of /tmp get cleaned up on a regular basis. (The default for RedHat/CentOS is to delete files that haven't been accessed in 10 days, with some exceptions.) Putting stuff in /tmp, and expecting it to stick around indefinitely, is a bad practice. Was there a reason for doing it like this?

    Second, is there a way to override this behavior, and force the jar to be stored in the Jenkins home (which is what happens with normal slaves)?

  25. Unknown User (thangapr81)

    I am trying to configure the windows slave, i can see the new slave is spinned up, but keep seeing this message, would you kindly help me with a solution ?

    Waiting for WinRM to come up. Sleeping 10s.
    Connecting to ec2-windows-host.compute-1.amazonaws.com(<<host ip>>) with WinRM as Administrator....
    Waiting for WinRM to come up. Sleeping 10s.

     

    1. Unknown User (dfrank)

      I'm also trying to configure for windows. I see the EC2 instance spin up and I have user data to set up winrm, but I get the same endless loop. Did you have any progress for this, or does anyone know a resolution?

      I am able to connect to the slave via RDP, and with powershell "Test-WsMan" and "Invoke-Command" by passing the credential for the administrator.

      My user data:

      <script>
      winrm quickconfig -q && winrm set winrm/config/client/Auth @{Basic="true"} && winrm set winrm/config/service/Auth @{Basic="true"} && winrm set winrm/config/service @{AllowUnencrypted="true"} && winrm set winrm/config/winrs @{MaxMemoryPerShellMB="1024"} && netsh advfirewall firewall add rule name="WinRM-HTTP" dir=in localport=5985 protocol=TCP action=allow
      </script>
      <persist>true</persist>
      1. Unknown User (dfrank)

        My issue was that I also needed to open the port for 445.

        netsh advfirewall firewall add rule name="SMB-HTTP" dir=in localport=445 protocol=TCP action=allow
        1. Unknown User (robeves63)

          Hi David,

          Are you now able to run windows slaves? I'm having the same looping winrm problem.

          Rob

  26. Unknown User (cpenhaligon)

    Is there any way to specify multiple subnet IDs in the config? I'm having a problem launching nodes if I enter more than one, so am currently restricted to a single availability zone. 

    1. Unknown User (mykola)

      A very important issue, it is possible to start spot instance in cheapest subnet from aws api - https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/spot-fleet-examples.html#fleet-config2
      but it is impossible to do the same thing with this plugin (sad)

      when I try to make several records with the same label in different subnets - always starts the first record

  27. Unknown User (romang_1)

    Hi Chris, I've created several AMIs for this purpose

  28. Unknown User (antonio_wang)

    Hi,

    I am a new one start to use this plugin and came into an issue..

    After I input informations in jenkins plugin, I tried to click "Provision via Test" in Nodes and got below error:

    --------------------------------------------------------------------------------------------------------------------------------------------------

    com.amazonaws.services.ec2.model.AmazonEC2Exception: Value (arn:aws:iam::XXXXXXXXXXXX:role/AXXX-FullAdmin) for parameter iamInstanceProfile.arn is invalid. Invalid IAM Instance Profile ARN (Service: AmazonEC2; Status Code: 400; Error Code: InvalidParameterValue; Request ID: 9387cf47-9137-41c0-9aea-652bc4063a7e)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1639)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1304)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1056)

    So I got back to plugin config:

    And I think currently I input correct IAM Instance Profile as I copied from AWS->Roles→Summary Role ARN:

     

    Can you pls help me?

     

  29. Unknown User (funeeldy)

    Please pick up the latest commits.  We need to use this plugin to create parallel slaves ASAP. thank you.

  30. Unknown User (jg_lgc)

    I'm trying to setup with the block device mapping and having some issues. Is EBS Optimized required for the device mapping? When I enable that I get this error

    EBS-optimized instances are not supported for your requested configuration

    I'm wanting to have a 100 GB EBS root volume, and so have the mapping set to

    /dev/xvda=:100:true:gp2

    1. Unknown User (jg_lgc)

      I figured it out. EBS-optimized is unrelated, and the error was because I'm using t2.medium instance types. To map a custom sized root volume I needed to use device /dev/sda1.

  31. Unknown User (vitaly_il)

    I'm trying to use EC2 plugin with IAM role, assigned to the Jenkins master.

    Launching new slave fails with:

     

    com.amazonaws.services.ec2.model.AmazonEC2Exception: You are not authorized to perform this operation. Encoded authorization failure message: lHuEfanNqJBznhhoWAOq6YC7LpXGrt6Ibhb
    cOuEs5ZA7mvLQ2iSZOewOEERjC6VVwV25yLBoNWf5PO9On2ybgaAobi8tFdOmhsj9cnZh0t7IBk7KZ-KChEb-VOjg7jZsfPh9JI6IZe_hqUqrkKO5gm-YnKO707mUtWtEr7qUWPwrQDrQ4Vi0ZdblOO2__HgElb0TuZaqJeqcJpNv9L2P
    YIZ6OihP5fKZGWSYAVRmaUaM3y_6aIR2QIEOjCuJ1uhZvwrT00x_Ge_kA2a1b2j_rONS9ZtE8UNTP5qMnqMkQYea_yGEQFfHxsJ9liVd03_eIP_-bQ19kGI2ZVdmVumxXKW5F6sRYdW3_wozCBmp2e41e7zsPiaBODojdxwrXN2MVnJQI
    7ikWZW99hQ7ApuJrUWPuK8mX54FWjl_l5SiyLaXpdOLChTP_QrwvNUt1wngiBTG1pcXbdIH0IfxQV11h1RWs0XgRJkPGfESUiZhFWCp_30g8exSdahhRjUCSPcSsQIFR5hUQapbP9zpqW7fragkqna_Q4aEyWf4sr8 (Service: Amaz
    onEC2; Status Code: 403; Error Code: UnauthorizedOperation; Request ID: 2bf1b6c4-4376-4ddf-9c90-ff325979d6a4)

    even I'm able to launch new EC2 using the same params using AWS CLI:

     

    aws ec2 run-instances --image-id ami-XXXXXX  --subnet-id  subnet-XXXX --security-group-ids sg-XXXX  --key-name jenkins-ci-slave --instance-type c4.large

    Any ideas?

    TIA,

    Vitaly

    1. Unknown User (vitaly_il)

      Solved: it was permission issue related to launching AMI with encrypted volumes.

  32. Unknown User (scardon)

    Currently I am not seeing anything in the cloud statistics plugin? Is the EC2 plugin hook up to that?

  33. Unknown User (jeffernz)

    The /usr/bin/userdata script is outdated. openjdk-7-jre package is no longer available to download on Ubuntu Xenial. Changing it to openjdk-8-jre manually fixed the problem.

  34. Unknown User (girardc79)

    A warning to anyone running Windows instances with this plugin:  The current version of this plugin (1.38) is not safe to use as it drops the stored Windows machine passwords whenever the Jenkins master is restarted:
    JENKINS-47985 - Getting issue details... STATUS

    The bug appears to have been introduced to the codeline in the 1.37 release.  There is a fix pending, but it has not been acted on since Nov.  For now, I suggest not upgrading beyond 1.36 if you use Windows instances.

  35. Unknown User (greenkarmet)

    I have a need to bind a specific network interface to an EC2 instance to complete a job (due to a node-locked license).  I don't see a way to bind a network interface to a slave.  Is there a trick to doing this?

    1. Unknown User (mykola)

      Hi Unknown User (greenkarmet),

      you can assign Role to a slave which has a policy with attach network interface permission, after that in init script you need to install awscli package and attach network interface during instance initialization (wink)

      1. Unknown User (greenkarmet)

        Perfect.  Works great.  Thanks.

  36. Unknown User (akoi)

    First-off - fantastic plugin, thanks! 

    I'm using this together with the CloudBees Amazon Web Services Credentials Plugin, using an assume-role to manage the EC2 slave instances. This works initially to communicate with EC2 and manage slaves, but as the credentials provided by the STS for assuming the role are only temporary - between 15mins and 1hour (default) - after the credentials expire, the plugin is no longer able to authenticate with AWS, and therefore fails to manage slaves. It looks like the plugin only assigns the credentials on initialisation, and doesn't request new credentials on expiry, causing this problem. Manually going to the Jenkins → Manage Jenkins → Configure System UI page, and just saving that page seems to re-initialise the credentials, causing the plugin to work until the new credentials expire. 

    Interested to hear if anyone else is experiencing this issue, has a workaround (I looked at a managed script to re-save the configuration and force new credentials, but couldn't seem to get this working), or if there is a plan to address this? Will also post an issue on Github and dive into the code when I get the time.

  37. Unknown User (vitaly_il)

    I have a few types of slaves, but userdata is almost the same.

    Is there a way for using some file content as userdata? 

  38. Unknown User (jflorez)

    I am using version 1.38 of the plugin. I create cloud and template definitions using the API and then I check them by running the script at the end of the comment which shows that the cloud and template configurations were created correctly for the given label. Then I try to run a pipeline that makes use of the template and I get Jenkins doesn’t have label jenkins-docker-agent-useast1-t2small

     

    A restart of Jenkins fixes the problem which makes me think that this is some sort of internal cache issue.

     

    Is there a way of forcing a cache refresh after doing a Jenkins.instance.save()?

     

    def label = "jenkins-docker-agent-useast1-t2small"
    def l = hudson.model.labels.LabelAtom.get(label)
    Jenkins.instance.clouds.each {
      def cloud = it
      println "Cloud name: " + cloud.displayName
      println "can provision label with name " + label + ": " + cloud.canProvision(l)
      it.getTemplates().each {
        def template = it
        println "Template name: "+template.displayName
        it.labelSet.each {
          print "Label " + it.toString() + " matches " + label + ": "
          println it.toString().equalsIgnoreCase(label)
        }
      }
    }
     
    1. Unknown User (jflorez)

      Actually this is not a good solution. When I do this then the pipeline downstream doesn't pick up the newly created template. The solution was to get a reference to the current cloud, create a new ArrayList based on cloud.getTemplates, add the new template to this list, create a new cloud duplicating the values of the old one and passing the new templates list to the constructor, deleting the old cloud from the clouds list, adding the new cloud, and then saving the instance

      1. Unknown User (acer123)

        Can you share your code? This is similar to what I'm trying to accomplish. I'm struggling with writing a script that can execute on a regular (e.g. daily) basis that will:

        access the list of AWS AMIs that have a certain tag (e.g. jenkins-slaves-*)

        Loop through the existing configuration and add any AMIs as new slaves if they are not yet configured in Jenkins.

        Modify configurations of existing slaves if they are currently configured in Jenkins (e.g. AMI ID has changed).

        Although I am very competent in bash and python, I've never created a Groovy script and am struggling with some of these tasks so any hints you can offer would be greatly appreciated.

  39. Unknown User (patrickceg)

    The scripts and documentation regarding "SLAVE_NAME" have been invalidated by commit ae7e53ab99028fd8ede1a3e41907812c2edb0eec 

    SLAVE_NAME is still referred to in the ubuntu-init.py and this article, but there are no other references to that in the code, and the userdata seems to now be just a text field that defaults to empty string. (I tried aimlessly for an hour looking at the code and experiment if there was some secret I was missing.)

    Since the slave name is no longer exposed in userdata, what are you supposed to do to replace slaveName when you call the following?

     

    java -jar slave.jar -jnlpUrl [Jenkins_Url]computer/ slaveName slave-agent.jnlp
    
    
    

    I'm happy to help out fixing either the scripts or this document if it will prevent someone else from banging their head for 3 hours.

  40. Unknown User (stevejr)

    Hi,

    I am using the plugin (v1.39) and am struggling with it being able to create more than 1 EC2 instance. I have configured the global instance cap as empty and then instance cap for the AMI to be 5 and set the number of executors to 1. I have set a label value and also added some tags.

    When I execute a build that keeps the execution running on the node i.e. by having an input step, and then run another build that requires the same AWS instance type the plugin does not start another AWS instance but instead just waits for the first job to finish.

    I have looked at the code and there seems to be a flag available to force a new instance to be created but I am unable to set that anywhere in the plugin configuration but I should not have to as I want the number of instances to scale up to the instance cap I have set.

    Any help would be greatly appreciated.

    Thanks.

  41. Unknown User (callump)

    Hi,

    Is this documentation out of date? My scenario is this:

    • I want to run the slave on a docker container on the EC2 instance spawned by this plugin (No, I don't want to use ECS).
    • To achieve the above, I need to use a custom init script with


    The problem is that nothing is passed through via user data (like it states should be on this page) so I can't get the agent name or the Jenkins URL. The URL isn't too much of a problem because I can autodiscover it. Same with the agent name.

     

    I do need the agent secret though, which would be a nice addition.

     

    Thanks

    ---

    Edit: As Patrick Santos pointed out above, this was removed by https://github.com/jenkinsci/ec2-plugin/commit/ae7e53ab99028fd8ede1a3e41907812c2edb0eec. Seems weird to take away that functionality.

     

    1. Unknown User (patrickceg)

      Currently I get the slave name by this hackery by reverse-enginnering the name of "description (instance id)", and use a "user" API key instead of a secret, since as you mentioned, the secret isn't passed easily anywhere from the code that I could find.

       
      #! /bin/bash
      # Stop on error
      set -e
      # Grab user data - see https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html#instancedata-add-user-data
      AWS_INSTANCE_ID=$(curl http://169.254.169.254/latest/meta-data/instance-id)
      # Replace with whatever you put as the 'description' for the Jenkins AMI config, as this is parsed into the name of the Jenkins slave
      JENKINS_AMI_DESCRIPTION=mybuilder
      SLAVE_NAME="${JENKINS_AMI_DESCRIPTION} ($AWS_INSTANCE_ID)"
      echo Slave name is "${SLAVE_NAME}"
      sudo yum install -y perl-URI
      SLAVE_URI=$(perl -MURI::Escape -e 'print uri_escape($ARGV[0])' "$SLAVE_NAME")

       

      Instead of the secret I use a user authentication token as environment variable API_TOKEN, so my launch command then looks like this.

       
      java -jar agent.jar -jnlpUrl "${JENKINS_URL}/computer/${SLAVE_URI}/slave-agent.jnlp" -jnlpCredentials $API_TOKEN -workDir "${AGENT_WORKDIR}"

      Where JENKINS_URL is the the URL to access the Jenkins master, SLAVE_URI is the URL-encoded slave name, API_TOKEN is an authentication token for my Jenkins "user" that is dedicated for initializing the node, and AGENT_WORKDIR is some folder I want the slave to use.

       

  42. Unknown User (__abhi__)

    Hey, 

     

    I am seeing some inconsistent behavior with this plugin (version 1.39). 

    When there are no workers up and I try to run a job, a worker is brought online. 

    But when the only workers that is up has been marked offline due to the disk space being full, the job just sits in the cue, and no worker is brought online.

     

    Alos I am curious to know if I can boot an EC2 worker from the CLI or the API using the EC2 plugin? 

    What I would like to do is poll the number of executors available and start to scale up when there is a certain percentage of free executors like 20% of total for example. The main reason to do this because an EC2 instance take a long time to come up and I want the Jobs to not have to wait. 

     

    Thank you

  43. Unknown User (robeves63)

    Is this plugin currently working with Windows instances? I'm having issues with the looping WinRM 10 seconds sleep.

  44. Unknown User (nissar)

    Does EC2 plugin support AWS Workspaces (https://aws.amazon.com/workspaces/) ?

    Can we create on-demand aws workspaces using EC2 plugin ?

    If there is any documentation available on the same, please share it.

  45. Unknown User (antoine_ch)

    Hello, we use your plugin and it works well. Thanks you for that.

    But we wanted to reuse a little more the EC2 instances created, can we prevent the EC2 instance to terminate at the end of its use of a node ? maybe the EC2 can stick around for a copple of minuts, to be reused when another pipeline want a node?

    1. Unknown User (vitaly_il)

      sure - it's default behaviour.

      you have two params to play with:

      • idle time - how many minutes EC2 instance will wait before shutdown
      • shutdown/terminate - define policy for EC2 stopping
      1. Unknown User (antoine_ch)

        Thx for your repply, so as you said in your doc, to confirm, I need to configure this with a groovy script ? 

        1. Unknown User (vitaly_il)

          No, just in the plugin settings menu.

  46. Unknown User (nvedia)

    I am using this plugin to spawn dynamic Jenkins Windows EC2 instances for doing automation tests (GUI)
    But the problem is that when I login to EC2 instances, I do not see the browser screenshots.
    I understand since we are using WinRM as Slaves, its difficult to see the GUI. Any other options to see the GUI tests initiated by Jenkins on Windows Ec2?
    Do we have any plugin to provision dynamic Slaves using JNLP?

  47. Unknown User (johnrothermel)

    The plugin seems to require an ssh private key but I don't see where to specify the corresponding EC2 Key pair to attach to instances at launch time.

    Is there a way to specify? How does the public key get added to newly provisioned slaves?

    1. Unknown User (johnrothermel)

      Figured it out. Had to read a separate doc that informed me to create a new keypair named jenkins

  48. Unknown User (prashantbhatiain)

    Hi,

         We are using this plugin for launching the Windows Slaves ,they get launched alright ,but the actual builds running on these slaves becomes painfully slow,for example the svn checkout which takes around 30 seconds through on static slave now takes more than 15 minutes,this may be due the WinRM service as transport,which may be slow.Do we have the option to switch to JNLP connection ?

    Using Winrm service is making this plugin useless for windows or  Im doing something wrong?

    Thanks 

     

    1. Unknown User (sijoey)

      Unknown User (prashantbhatiain) Hi Prashant,  I am having the same issue with running Windows with this plugin.  Were you able to resolve your issue with it?  I tried warming the volume and still not change.  I also noticed that jobs run much faster the second time.

  49. Unknown User (gdupin)

    Hi,

    Is it possible to not have to put the SSH private key in plain text but instead to refer to a SSH key from Jenkins Credentials ?

    I am currently using the Jenkins Configuration-as-Code plugin and having to put the entire key in my YAML file is a bit problematic as this configuration file will be in GIT... I think that a reference to an existing Jenkins Credentials key would help here.

    Do you think it is feasible ?

    Thanks

    1. Unknown User (batmat)

      Unknown User (gdupin) you probably should rather send that kind of requests on the users ML (https://jenkins.io/mailing-lists/) for much more audience. Or, even a JIRA ticket would likely make sense here for this pretty clear (and IMO useful) use case.

      I agree with you it would be really nice to integrate with the credentials plugin. I actually handled that case very recently in Jenkins Essentials. The configuration-as-code plugin supports variables, and can automatically source them from various locations, included Hashicorp Vault, or Docker Secrets. I used the latter in https://github.com/jenkins-infra/evergreen/blob/master/distribution/environments/aws-ec2-cloud/config/as-code/ec2-cloud.yaml, and passed it under /run/secrets/PRIVATE_KEY to make it available at startup, in https://github.com/jenkins-infra/evergreen/blob/master/distribution/environments/aws-ec2-cloud/CloudFormation/cloudformation-template.json#L59.

      Hope this helps.

  50. Unknown User (gdupin)

    Unknown User (batmat) ok thanks for the advice and I will open a JIRA ticket to propose it.

    Thank you for the examples ! One point that is not clear for me tough : the content of the "privateKey" parameter in your example seems to be directly the key whereas when I tested it on my jenkins instance, the content apparently needs to be the encrypted form of the key (when I look at the config.xml file, it is something like this : "<privateKey>{AQAA...w==}</privateKey>".

    And when I set it in my jenkins.yml for JCasC, I had to put this encrypted form instead of the real key content. It is not very clear for me if I must deal with keys directly or with their "encrypted-by-master" form... Because in this case, it will be difficult to move the credentials from one instance to another (I will have to move also the "secrets" Jenkins master directory...)

    1. Unknown User (gdupin)

      Here is the JIRA ticket :  JENKINS-52547 - Getting issue details... STATUS

      UPDATE : for my JCasC configuration, both are working : plain text and master-encrypted private key (although I am not sure if it is normal that the encrypted-form works too...)

  51. Unknown User (five_asterisk)

    An instance of the slave is unstable and I am in trouble.
    How do I pass "-text" to slave.jar?

  52. Unknown User (zakabluk)

    I'm trying use plugin for launch selenium tests on Windows server via winrm protocol. All screenshots has size 1036*660 to 1920*1080. I did many attempts for resize a page resolutions, but all attempts has fail. 

    I changed register. I added into init script:

    C:\Users\jenkins>powershell -command "Set-DisplayResolution -Width 1920 -Height 1080 -Force" 
    setres will now attempt to apply the following display settings: 
      Width:          1920
      Height:         1080
    The settings passed in could not be applied to the graphics device.
    init script ran successfully

    There are screens has good resolutions if I will be do rdp connect to windows server before launch my tests.

    Therefore, maybe reason is in display driver (Hyper-V Video), Anybody, maybe do you have any experience for solved my issue?

    1. Unknown User (zakabluk)

      Hi! I found out an answer on my question. WinRm is open session in console 0(sessionid 0). For this session we can't changed anything. I'm trying investigate more.

      p.s. Azure VM Agents plugin has same problem. Ssh connection is using same sessionid 0 for work.

      1. Unknown User (zakabluk)

        Those plugins can't be used  for Selenium tests on Windows when display resolutions is instead 1036*660. Because plugin is working in noninteractive session 0 when use WinRm or ssh as methods for connect to slave.  

        And we can't influence of this situation.

        Links about problem:

        https://docs.microsoft.com/en-us/windows/desktop/api/processthreadsapi/nf-processthreadsapi-createprocessasusera


        http://rzander.azurewebsites.net/create-a-process-as-loggedon-user/

  53. Unknown User (apetrenko_ta)

    Hello,

    I have a problem running jenkins worker with SPOT instance with jenkins amazon EC2 plugin version 1.40.1. 

    Jenkins Amazon EC2 Plugin version 1.39 is working well 

    On-Demand instance with the same settings is starting well (1.40.1).

    log on demand:

    Oct 18, 2018 10:40:55 PM hudson.plugins.ec2.EC2Cloud
    INFO: Launching instance: i-0f79d897f46bcad22
    Oct 18, 2018 10:40:55 PM hudson.plugins.ec2.EC2Cloud
    INFO: bootstrap()
    Oct 18, 2018 10:40:55 PM hudson.plugins.ec2.EC2Cloud
    INFO: Getting keypair...
    Oct 18, 2018 10:40:55 PM hudson.plugins.ec2.EC2Cloud
    INFO: Using key: Jenkins
    25:9f:b6:d8:f9:e2:a7:d8:72:f4:37:b0:f6:d8:4d:0e
    -----BEGIN RSA PRIVATE KEY-----
    MIIEpAIBAAKCAQEAs2RqjBAQKnw8I7grFSlBx21Chh4U34dsn9QhX4HUqbNcFsaV
    d3FbnLaDpsYPjRbXfvapnaIOiTBvCquDiluzP4rww/SoDUg5XpiKih+wMBpgsZb
    Oct 18, 2018 10:40:55 PM hudson.plugins.ec2.EC2Cloud
    INFO: Authenticating as ubuntu
    Oct 18, 2018 10:40:55 PM hudson.plugins.ec2.EC2Cloud
    INFO: Connecting to 10.78.44.158 on port 22, with timeout 10000.
    
    Oct 18, 2018 10:41:05 PM hudson.plugins.ec2.EC2Cloud
    INFO: Failed to connect via ssh: The kexTimeout (10000 ms) expired.
    Oct 18, 2018 10:41:05 PM hudson.plugins.ec2.EC2Cloud
    INFO: Waiting for SSH to come up. Sleeping 5.
    
    Oct 18, 2018 10:41:10 PM hudson.plugins.ec2.EC2Cloud
    INFO: Connecting to 10.78.44.158 on port 22, with timeout 10000.
    
    Oct 18, 2018 10:41:20 PM hudson.plugins.ec2.EC2Cloud
    INFO: Failed to connect via ssh: The kexTimeout (10000 ms) expired.

    ....


    SPOT instance (1.40.1): 

    only changes: 

    set "Use spot instance"
    The current Spot price for a T2Large in the us-west-2a availability zone is $0.127

    Spot Max Bid Price: 0.2


    Log with spot instance:

    Oct 18, 2018 10:44:06 PM hudson.plugins.ec2.EC2Cloud
    INFO: Launching instance: null
    Oct 18, 2018 10:44:06 PM hudson.plugins.ec2.EC2Cloud
    INFO: bootstrap()
    Oct 18, 2018 10:44:06 PM hudson.plugins.ec2.EC2Cloud
    INFO: Getting keypair...
    Oct 18, 2018 10:44:06 PM hudson.plugins.ec2.EC2Cloud
    INFO: Using key: Jenkins
    25:9f:b6:d8:f9:e2:a7:d8:72:f4:37:b0:f6:d8:4d:0e
    -----BEGIN RSA PRIVATE KEY-----
    MIIEpAIBAAKCAQEAs2RqjBAQKnw8I7grFSlBx21Chh4U34dsn9QhX4HUqbNcFsaV
    d3FbnLaDpsYPjRbXfvapnaIOiTBvCquDiluzP4rww/SoDUg5XpiKih+wMBpgsZb
    Oct 18, 2018 10:44:06 PM hudson.plugins.ec2.EC2Cloud
    INFO: Authenticating as ubuntu
    ERROR: Unexpected error in launching an agent. This is probably a bug in Jenkins
    java.lang.NullPointerException
    	at hudson.plugins.ec2.ssh.EC2UnixLauncher.getEC2HostAddress(EC2UnixLauncher.java:368)
    	at hudson.plugins.ec2.ssh.EC2UnixLauncher.connectToSsh(EC2UnixLauncher.java:318)
    	at hudson.plugins.ec2.ssh.EC2UnixLauncher.bootstrap(EC2UnixLauncher.java:282)
    	at hudson.plugins.ec2.ssh.EC2UnixLauncher.launchScript(EC2UnixLauncher.java:130)
    	at hudson.plugins.ec2.EC2ComputerLauncher.launch(EC2ComputerLauncher.java:48)
    	at hudson.slaves.SlaveComputer$1.call(SlaveComputer.java:294)
    	at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
    	at jenkins.security.ImpersonatingExecutorService$2.call(ImpersonatingExecutorService.java:71)
    	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    	at java.lang.Thread.run(Thread.java:748)
    



    Jenkins Version 2.147

    Jenkins Amazon EC2 Plugin version: 1.40.1

    Jenkins Amazon EC2 Plugin version 1.39 is working well 




  54. Unknown User (apetrenko_ta)

    Hello,

    Is there any way to request spot instance for specific duration?

  55. Unknown User (ranjith_322)

    Ondemand Slave volumes are not tagged, is there any option i can Tag the volumes ? 

  56. Unknown User (nforbes)

    Hello,

    We have recently started using this Jenkins plugin.  Whilst perusing the source code we noticed that the contents of the Jenkins private key is added as a header.  https://github.com/jenkinsci/ec2-plugin/blob/17ad3043e0e049f43c4fd5f44167b15fab3bbfcc/src/main/java/hudson/plugins/ec2/EC2Cloud.java#L909  Is anyone able to explain why this is needed, because it looks like a large security risk?  

  57. Unknown User (chippyash)

    Hello,


    Just followed (barely) the article on how to install Jenkins on AWS at https://d1.awsstatic.com/Projects/P5505030/aws-project_Jenkins-build-server.pdf  It's clearly outdated but managed to figure out the wrong bits. Right up to the point 11 in 'Configure Jenkins'  where the doc basically bombs out.  Never mind, I'll look at the plugin docs (at Amazon EC2 Plugin) .  That's where we hit a problem.  They bear no resemblance to the screen I am seeing in Jenkins V2.175 that has just installed on my EC2 instance.  

    Is there an updated page that covers current install/configuration process?

    As an aside,  I'd remark that the wiki page for the plugin is presupposing a huge amount of prior domain knowledge in order to make sense of it.  Having been a dev for only 30 odd years and having some AWS experience and a whole lot of Jenkins experience, it still is not making a lot of sense.  Happy to help in re-scripting the "destructions" if I can get help in understanding what you need to do in the first place.

  58. Unknown User (johnjeffers)

    I'm having a problem where the EC2 plugin is spinning up instances just fine, but is not terminating them. This quickly leads to a state where it just throws warnings in the Jenkins log about 0 capacity for new instances, and is unable to do anything until I manually terminate the EC2 instances it created, which makes it fairly unusable because it requires so much babysitting.

    I have confirmed that the Jenkins master has the appropriate IAM permissions to be able to terminate. In fact, I can see times when it does terminate, so the problem is intermittent.

    May 2nd 2019, 19:24:39.617INFO: Removed EC2 instance from jenkins master: i-06e80c483fca05722
    May 2nd 2019, 19:24:39.616INFO: Terminated EC2 instance (terminated): i-06e80c483fca05722
    May 2nd 2019, 19:24:39.277INFO: EC2 instance idle time expired: i-06e80c483fca05722

    I suspect what may be happening is that the EC2 plugin sometimes loses track of the instances somehow, but I'm not seeing anything obvious in the Jenkins master logs. Any suggestions on where to look?

  59. Unknown User (chippyash)

    So following on from earlier message, I now have it to the point were the plugin does indeed fire up the EC2 instances, but then fails to connect to them. Here's how I fixed it

    All the instructions assume you are using an Ubuntu instance as your slave.  We use AWS Amazon Linux 2, as that is what our production servers run on.

    The key to getting it to work is to use the public key provided when you set up your IAM account with keys.  Use the PEM file it provides and generate an open ssh public key from it.  Add that to authorized_keys on your slaves.  Add the private key as per the documented instructions into Jenkins. Generate your public key from it.  This is difficult in Windoze, but if you have access to a Linux machine (perhaps a VM), copy the pem file to it and then run 

    ssh-keygen -f <name-of-your-private>.pem -y

    This will output the public key string that you can then copy-pasta to the slave node that you are building the image for.

    I'm not enough of a Windoze type to know if PuttyGen can actually do what is required. I tried several times with it and failed, but maybe someone else knows better.


    Hope it helps

  60. Unknown User (rogeriocolferai)

    Hi,

    Is there any way to request spot instance for specific duration?

    I would like to use the option --block-duration-minutes to the request.

    https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/spot-requests.html#fixed-duration-spot-instances

    Thanks.