Skip to end of metadata
Go to start of metadata

Plugin Information

View Puppet Enterprise Pipeline on the plugin site for more information.

Introduction

This plugin adds Jenkins Pipeline steps for Puppet Enterprise. The provided steps make it easy to interface with Puppet Enterprise services such as the code management service and orchestrator service in order to use Puppet to perform some or all of the deployment tasks in a continuous delivery pipeline.

Features

A Pipeline project can use the provided groovy methods to deploy Puppet code to Puppet Enterprise servers, create Puppet orchestrator jobs, set Hiera key/value pairs, and query the infrastructure with the Puppet Query Language (PQL).

Puppet Enterprise RBAC access tokens are used to authenticate with the Puppet Enterprise APIs, so Puppet itself doesn't have to be configured on the Jenkins server.

node {
    puppet.credentials 'pe-access-token'
    puppet.hiera 'development' key: 'app-build-version', value: 'v1.0.1'
    puppet.codeDeploy 'production'
    puppet.job 'production', application: 'App[instance]', noop: true, concurrency: 10
}

Experimental Hiera Feature

The Hiera feature of this plugin is currently an experimental feature. This is experimental because the values are stored in an XML file on the Jenkins server. There is no audit history of the data and therefore no way to replicate past values. Also, if the file is lost due to, for example, disk failure, the current values are lost. Only use this if you trust your Jenkins server backups and don't care about audit history.

Security should also be considered when using the Hiera feature. Currently, there are no access restrictions on which key/value pairs can be modified by Jenkins Pipeline jobs. Anyone with access to a Pipeline script (including commit access to a Jenkinsfile in a repository) can set any key/value pair they wish. It is recommended this feature only be used if you trust everyone with write access to a Pipeline script.

Configuration

Puppet Server Address

Go to Jenkins > Manage Jenkins > Puppet Enterprise. Fill out the DNS address of the Puppet Enterprise Server. Note, if the Puppet agent is installed on the Jenkins server, it will be used to configure the Puppet Enterprise Server address.

The Puppet Enterprise Server CA certificate is automatically pulled from the Puppet Server's CA API. External CA's are not currently supported.

Access Token Credentials

This plugin uses the Plain Credentials plugin to store access tokens.

First, create a new RBAC access token in Puppet Enterprise. Follow the instructions for generating a token for use by a service. Remember to give the token a long lifetime.

Second, create new credentials in Jenkins. For Kind specify Secret text. Set the Secret value to the token printed out by the first step. It is recommended to give the token an easy-to-identify ID by clicking on the Advanced button. That will make it easier to identify which token is being used from the Jenkins Pipeline scripts.

User Access Token Permissions

The following permissions are needed for the pipeline plugin to successfully run the 'codeDeploy', 'job' and 'query' steps. The example below illustrates a new Puppet user role named 'Jenkins' with the appropriate permissions deploy code, create an orchestration job, run agents, and query the Puppet DB.

Hiera

If you wish to use the puppet.hiera method, you will need to configure the Puppet Server to look at the Hiera store in Jenkins for possible key matches.
Use the hiera-http backend to perform key lookups.

The following example hiera.yaml configuration will first lookup values in the local yaml data store, then if nothing is found, will perform a lookup against the Jenkins server's Hiera data store managed by this plugin.
In this configuration, Hiera will first look for a key in the scope matching the node's certname. If nothing is found, it will look for the key in the node's environment. You can substitute/add any Facter fact here to list of paths.

:backends:
  - yaml
  - http

:http:
  :host: jenkins.example.com
  :port: 8080
  :output: json
  :use_auth: true
  :auth_user: <user>
  :auth_pass: <pass>
  :cache_timeout: 10
  :failure: graceful
  :paths:
    - /hiera/lookup?scope=%{clientcert}&key=%{key}
    - /hiera/lookup?scope=%{environment}&key=%{key}

Hiera lookup authentication

If Jenkins' Global Security is configured to allow unauthenticated read-only access, the 'use_auth', 'auth_pass', and 'auth_user' parameters are unnecessary in the hiera.yaml file. Otherwise, create a local Jenkins user that has Overall/Read permissions use that user's credentials for the hiera.yaml configuration.

Hiera Data Store permission

If Jenkins' Global Security is configured to use matrix authorization, any user with the Hiera/View permission is allowed to view the Hiera Data Store page and any user with the Hiera/Delete permission can delete scopes and keys.
Note, these permissions have no effect on the ability to lookup specificy Hiera keys using the /hiera/lookup endpoint.

Pipeline Steps

puppet.credentials

The puppet.credentials method sets the Puppet Enterprise RBAC token to be used for all other Puppet pipeline step methods.

groovy script invocation: puppet.credentials 'jenkins-credential'

Example

puppet.credentials 'pe-access-token' 

puppet.query

The puppet.query method queries PuppetDB using the PQL query language. To learn more about PQL, go here: https://docs.puppet.com/puppetdb/4.3/api/query/v4/pql.html

This method returns an ArrayList object that can be stored in a variable and iterated on.

groovy script invocation: puppet.query 'query'

Parameters

  • credentials - The Jenkins credentials storing the PE RBAC token. String. Required if puppet.credentials not used.

Example

puppet.query 'nodes[certname] { catalog_environment = "staging" }', credentials: 'pe-access-token'
results = puppet.query 'inventory[certname] { trusted.extensions.pp_role = "MyApp" }'

//The following gets production nodes with failed report, extracts just their
// certnames, then groups them by three, and finally runs Puppet on each
// group of three.
results = puppet.query 'nodes { latest_report_status = "failed" and catalog_environment = "production"}'
nodes = []
for (Map node : results) {
  nodes.add(node.certname)
}
nodesubgroups = nodes.collate(3) //Break results into groups of 3
for (String certnames : nodesubgroups ) {
  puppet.job 'production', nodes: certnames
}

puppet.codeDeploy

The puppet.codeDeploy method tells Puppet Enterprise to deploy new Puppet code, Hiera data, and modules to a specified Puppet environment. To lean more about code management in Puppet Enterprise, go here: [https://docs.puppet.com/pe/latest/code_mgr.html]

groovy script invocation: puppet.codeDeploy 'environment'

Parameters

  • credentials - The Jenkins credentials storing the PE RBAC token. String. Required if puppet.credentials not used.

Example

  puppet.codeDeploy 'production', credentials: 'pe-access-token'
  puppet.codeDeploy 'staging'

puppet.job

groovy script invocation: puppet.job('environment')

Parameters

  • credentialsId - ID of the Jenkins Secret text credentials. String. Required if puppet.credentials not used
  • concurrency - Level of maximum concurrency when issuing Puppet runs. Defaults to unlimited. Integer.
  • noop - Whether to run Puppet in noop mode. Defaults to false. Boolean

Puppet Enterprise 2015.2 - 2016.3 Parameters

The following parameters should be used with Puppet Enterprise 2015.2 - 2016.3 for definining the job's run target. Note, the target parameter will work with Puppet Enterprise 2016.4+ but has been deprecated.

  • target - Target in environment to deploy to. Can be app, app instance, or app component. Defaults to entire environment. String

Puppet Enterprise 2016.4+ Parameters

The following parameters should be used with Puppet Enterprise 2016.4+ for definining the job's run scope.

  • nodes - An array of nodes to run Puppet on.
  • application - The name of the application to deploy to. Can be all instances or a specific instance. e.g 'MyApp' or 'MyApp[instance-1]'. String.
  • query - The PQL query to determine the list of nodes to run Puppet on. String.

Example

  puppet.job 'staging'
  puppet.job 'production', concurrency: 10, noop: true
  puppet.job 'production', concurrency: 10, noop: true, credentialsId: 'pe-access-token'
  puppet.job 'production', nodes: ['node1.example.com','node2.example.com']
  puppet.job 'production', application: Rgbank
  puppet.job 'production', application: Rgbank[phase-1]
  puppet.job 'production', query: 'nodes { certname ~ "substring" and catalog_environment = "production" }' 

puppet.hiera

groovy script invocation: puppet.hiera

Parameters

  • scope - The scope scope of the data lookup from Hiera. Usually this will be an environment name. Required. String
  • key - The name of the key that Hiera will lookup. Required. String
  • value - The value of the key to be returned to Hiera's lookup call. Required. Can be string, array, or hash

Example

  puppet.hiera scope: 'staging', key: 'app-build-version', value: 'master'
  puppet.hiera scope: 'production', key: 'app-build-version', value: '8f3ea2'
  puppet.hiera scope: 'dc1-us-example', key: 'list-example', value: ['a,'b','c']
  puppet.hiera scope: 'host.example.com', key: 'hash-example', value: ['a':1, 'bool':false, 'c': 'string']

Compatibility

This plugin is compatible with Puppet Enterprise 2016.2+ and Jenkins 1.642.3+

Some step parameters are not only available on newer versions of Puppet Enterprise. Those parameters are labelled as such.

This plugin has not been tested with Jenkins Declarative Pipelines.

Examples

Example Pipeline scripts can be found in the examples directory of the project's GitHub repository: https://github.com/jenkinsci/puppet-enterprise-pipeline-plugin/tree/master/examples

6 Comments

  1. Does this require the user to configure Code Manager (or r10k) OR is it an alternative to those?

    1. Hello, Gaurav. I'm sorry I missed your comment. The module requires you to be using Code Manager in Puppet Enterprise. This plugin uses the Code Manager API to issue the Puppet code deploys.

  2. Hi Carl.  This is just what I was looking for.  One question I have is can you override the puppet master in the config as we have more than one puppet master in ourpipeline and I would like to switch through deployments.

    1. Hello, Mike. Unfortunately no, you cannot. That's becoming a more common request so I'll definitely bump up the priority for that feature. Thanks for the feedback!

  3. Hi Carl,

    We would like to have multiple Puppet Enterprise servers as well.

    It would be great if you could at least add the ability to specify the puppet enterprise server URL to override the "default" setting in the "Manage Jenkins" config.

    However, it would also be cool if you could specify a "list" of puppet servers in "Manage Jenkins", and for each puppetserver, we could set a "default" authentication token (DRY, right?). It feels like the authentication token name would be something you would configure at the Jenkins server level rather than per build.

    ~tommy

  4. Carl,

    The plugin is working well - thanks for your work on it, but we are also running into the same issue needing to use it for multiple Puppet Masters. Any update on this feature?

    Thanks!

    Jon