Skip to end of metadata
Go to start of metadata

Plugin Information

View jenkins-cloudformation-plugin on the plugin site for more information.

This plugin is up for adoption. Want to help improve this plugin? Click here to learn more!

A plugin that allows for the creation of cloud formation stacks before running the build and the deletion of them after the build is completed.

This plugin gives Jenkins the ability to spawn Amazon Cloud Formation stacks before running the build and stopping it at the end.

This is particularly useful for testing; where you can create your application stacks in the Amazon cloud; deploy your application to them, run tests against it and finally take all the stacks down.

If the stack takes parameters, you can enter them as a semi-colon separated list of value=pair. For backward compatibility you can also use comma as separator for parameters. This however prevents you from passing comma separated lists as values for parameters. If you need to pass lists as values for parameters then you should separate your parameters with a semi-colon like this:

 param1=value1;param2=v2,v3,v4,v5;param3=v3,v4;param4=value4

If the stack that is created has outputs, they will be added as variables to the build environment so you can use them during the build just as any other environment variable like $WORKSPACE. The name of the variables will be the name of the output item defined in the stack json document prefixed with the name of the stack that produced it (ie. stack1_outputName).

This has the useful side effect of allowing for the chaining of several stacks where the ones further down can be fed the outputs of the previous stacks as parameters. In this way, if you have a stack that creates an instance and outputs its IP address you can potentially pass the IP address into another stack as a parameter and use it in whatever way you need.

The timeout value in the configuration refers to the maximum amount of time to wait before giving up on the stack creation. Keep in mind that if this threshold is reached the plugin will make no effort to delete the stack. Future version may change this behavior.

The stacks order can be re-arranged by dragging them. WARNING: If you are chaining stacks the order is relevant and therefore you should revise the parameters configuration of each of the stacks.

If an error occurs during the creation of the stack, this plugin will mark the build as failed and all previously created stacks will be deleted (Assuming they were flagged to be deleted at the end of the job)

As of version 0.8 a Build Notifier has been added to delete stacks. This feature allows jobs to kill stacks that were created in previous jobs. Here is a snapshot of the configuration for notifier:

13 Comments

  1. If you want to manage the risk associated with placing the access and secret keys in Jenkins you can create an AWS IAM group with the following policies and create a new user in this group and use the access keys of this user instead of those of a more powerful user.

    Stack Management:

    {
      "Statement": [
        {
          "Action": [
            "cloudformation:CreateStack",
            "cloudformation:DeleteStack",
            "cloudformation:DescribeStacks",
            "cloudformation:DescribeStackEvents"
          ],
          "Effect": "Allow",
          "Resource": [
            "*"
          ]
        }
      ]
    }
    

    Instance Management:

    {
      "Statement": [
        {
          "Action": [
            "ec2:RunInstances",
            "ec2:CreateTags",
            "ec2:TerminateInstance",
            "ec2:DescribeInstances"
          ],
          "Effect": "Allow",
          "Resource": [
            "*"
          ]
        }
      ]
    }
    
  2. Is there any way to debug what output gets saved as environment variables? I'm struggling to get the output of my first stack as an input parameter to my 2nd stack.

    My first stack called: BASEVPC

    My output called: VPCID (And I can see it at the cloudformation output page)

    That's what I'm trying to give to my 2nd stack as parameter:

    VpcID=$

    Unknown macro: { BASEVPC_VPCID }

    (I also tried VpcID=$BASEVPC_VPCID )

    "Parameter value $

    Unknown macro: {BASEVPC_VPCID}

    for parameter name VpcID does not exist."

    1. Hi Adam, did you get this working?

      I'm also not seeing the outputs.  In order to debug I've added a shell build step:

      printenv > allenv.txt

      But I can't see any CloudFormation related variables in there.

      1. Hi Mike,

        Yes, my working solution:

        1 job

        1 Build step - AWS Cloud Formation

        Multiple Stack configuration next to each other.

        The first stack:

        Stack name: Init

        The CF script has outputs called VpcCidr, PrefixName

        Second Stack:

        Cloud Formation parameters: VpcCidr=Init_VpcCidr;PrefixName=Init_PrefixName;

        As far as I remember, nothing else is set.

  3. I'm trying to get this work, but I need stack output paraeters in "Execute Shell" step after CloudFormation.
    It seems that output value variables not really in the environment.
    Should they be there?
    How can I access them?

    Thanks in advance

  4. There is some problem with concurrent execution of job, configured with this pugin.

    For build number n log says:
    AWS Cloud Formation is waiting for a checkpoint on Jobname#(n-1)
    and the build stuck untill the previous build finished.
    Any ideas how to solve or workaround this issue?

    Thanks in advance

  5. I got some problem during I use the plugin. I think the problem is Great Fire Wall stop the connection between Shanghai China and AWS.
    So I setup a proxy for my Jenkins master like this https://wiki.jenkins-ci.org/display/JENKINS/JenkinsBehindProxy
    But the problem not solve still. I don’t know whether this plugin will connect to AWS via Jenkins master’s proxy. How can I make the plugin establish connection via a proxy ?
    This is error information:
     
    Determining to create or update Cloud Formation stack: CollineCItest158
    Stack not found: CollineCItest158. Error was: java.net.UnknownHostException: cloudformation.ap-southeast-1.amazonaws.com
    Creating Cloud Formation stack: CollineCItest158
    Failed to create stack: CollineCItest158. Error was: java.net.UnknownHostException: cloudformation.ap-southeast-1.amazonaws.com
    ERROR: Processing failed due to a bug in the code. Please report this to jenkinsci-users@googlegroups.com
    com.amazonaws.AmazonClientException: Unable to execute HTTP request: cloudformation.ap-southeast-1.amazonaws.com
            at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:472)
            at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:296)
            at com.amazonaws.services.cloudformation.AmazonCloudFormationClient.invoke(AmazonCloudFormationClient.java:1273)
            at com.amazonaws.services.cloudformation.AmazonCloudFormationClient.deleteStack(AmazonCloudFormationClient.java:1119)
            at com.syncapse.jenkinsci.plugins.awscloudformationwrapper.CloudFormation.delete(CloudFormation.java:169)
            at com.syncapse.jenkinsci.plugins.awscloudformationwrapper.CloudFormationBuildWrapper.doTearDown(CloudFormationBuildWrapper.java:113)
            at com.syncapse.jenkinsci.plugins.awscloudformationwrapper.CloudFormationBuildWrapper.setUp(CloudFormationBuildWrapper.java:87)
            at hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.doRun(MavenModuleSetBuild.java:653)
            at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:537)
            at hudson.model.Run.execute(Run.java:1744)
            at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:529)
            at hudson.model.ResourceController.execute(ResourceController.java:98)
            at hudson.model.Executor.run(Executor.java:374)
    Caused by: java.net.UnknownHostException: cloudformation.ap-southeast-1.amazonaws.com
            at java.net.InetAddress.getAllByName0(InetAddress.java:1243)
            at java.net.InetAddress.getAllByName(InetAddress.java:1155)
            at java.net.InetAddress.getAllByName(InetAddress.java:1091)
            at org.apache.http.impl.conn.SystemDefaultDnsResolver.resolve(SystemDefaultDnsResolver.java:44)
            at org.apache.http.impl.conn.DefaultClientConnectionOperator.resolveHostname(DefaultClientConnectionOperator.java:259)
            at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:159)
            at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:304)
            at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:611)
            at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:446)
            at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:863)
            at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
            at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:57)
            at com.amazonaws.http.AmazonHttpClient.executeOneRequest(AmazonHttpClient.java:686)
            at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:461)
            ... 12 more
    project=hudson.maven.MavenModuleSet@5f329723JENKINS_TEST_COLLINE_SYSTEM_TEST_STUB
    project.getModules()=[hudson.maven.MavenModule@1632c4edcoselcoselcosel]
    project.getRootModule()=hudson.maven.MavenModule@1632c4edcoselcoselcosel
    FATAL: Unable to execute HTTP request: cloudformation.ap-southeast-1.amazonaws.com
    com.amazonaws.AmazonClientException: Unable to execute HTTP request: cloudformation.ap-southeast-1.amazonaws.com
            at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:472)
            at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:296)
            at com.amazonaws.services.cloudformation.AmazonCloudFormationClient.invoke(AmazonCloudFormationClient.java:1273)
            at com.amazonaws.services.cloudformation.AmazonCloudFormationClient.deleteStack(AmazonCloudFormationClient.java:1119)
            at com.syncapse.jenkinsci.plugins.awscloudformationwrapper.CloudFormation.delete(CloudFormation.java:169)
            at com.syncapse.jenkinsci.plugins.awscloudformationwrapper.CloudFormationBuildWrapper.doTearDown(CloudFormationBuildWrapper.java:113)
            at com.syncapse.jenkinsci.plugins.awscloudformationwrapper.CloudFormationBuildWrapper.setUp(CloudFormationBuildWrapper.java:87)
            at hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.doRun(MavenModuleSetBuild.java:653)
            at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:537)
            at hudson.model.Run.execute(Run.java:1744)
            at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:529)
            at hudson.model.ResourceController.execute(ResourceController.java:98)
            at hudson.model.Executor.run(Executor.java:374)
    Caused by: java.net.UnknownHostException: cloudformation.ap-southeast-1.amazonaws.com
            at java.net.InetAddress.getAllByName0(InetAddress.java:1243)
            at java.net.InetAddress.getAllByName(InetAddress.java:1155)
            at java.net.InetAddress.getAllByName(InetAddress.java:1091)
            at org.apache.http.impl.conn.SystemDefaultDnsResolver.resolve(SystemDefaultDnsResolver.java:44)
            at org.apache.http.impl.conn.DefaultClientConnectionOperator.resolveHostname(DefaultClientConnectionOperator.java:259)
            at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:159)
            at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:304)
            at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:611)
            at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:446)
            at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:863)
            at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
            at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:57)
            at com.amazonaws.http.AmazonHttpClient.executeOneRequest(AmazonHttpClient.java:686)
            at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:461)
            ... 12 more
    Finished: FAILURE
     

  6. Can someone please clarify me whether this plugin can be used to create, update and delete Stack in AWS Beanstalk? If so, please update me how can i do this from Jenkins with simple illustration. Thanks in advance.

    1. Hello. This plugin will create a stack in AWS CloudFormation. You can use AWS CloudFormation to create a stack in AWS Elastic Beanstalk as a resource. Perhaps these snippet examples will help: http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/quickref-elasticbeanstalk.html. Additionally, if you go here http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-beanstalk.html you will find more information about using AWS CloudFormation to create AWS Elastic Beanstalk resources. I hope that helps.

      Best regards,

      --Nathan

  7. Hi,

    can some one please suggest how to use the parameters field in the plugin and how we reference it in the json template

    I am trying to use the parameter field to create the user defined instance and use the same in json template to create the stack

    Please suggest, appreciate your response :)

  8. Hi,

    I am using Jenkins with AWS CloudFormation plugin to create stacks.

    My current setup in Jenkins:

    Jenkins job 1 - create cloudformation stack1 with build Trigger with Poll SCM

    Jenkins job 2 - create cloudformation stack2 with build after job 1

    Jenkins job 3 - create cloudformation stack3 with build after job 2

    Issue:

    Jenkins job to create a cloudformation stack failed with following error on failed job console output:

    Reason: Detailed Message: No updates are to be performed. (Service: AmazonCloudFormation; Status Code: 400; Error Code: ValidationError;)

    Desired result

    How to get all jobs run successfully. When there is a change on jenkins job 1 and job 3 but no change on job 2 or likewise.

  9. Hi, Is there any way to pass: CAPABILITY_NAMED_IAM

    https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/using-iam-template.html#using-iam-capabilities

    I am using jenkins cloudformation plugin: 1.2 version

     

    any help is appreciated!!

    1. Have the same issue need to pass the parameter to be able to created an ECS deployment. For now I think I have to create the IAM resources seperately (ie manually deploy a template that has them) a checkbox on the plugin that passes --capabilities CAPABILITY_NAMED_IAM to the base command this is running on is all that would be necessary. Trouble is this project is orphaned. A pity, because it's such a simple fix.