Due to some maintenance issues, this service has been switched in read-only mode, you can find more information about the why

and how to migrate your plugin documentation in this blogpost

Skip to end of metadata
Go to start of metadata

Plugin Information

View vSphere on the plugin site for more information.

Older versions of this plugin may not be safe to use. Please review the following warnings before using an older version:

This plugin is up for adoption. Want to help improve this plugin? Click here to learn more!

Add VMware vSphere support to Jenkins by making available multiple vSphere Build Steps and supporting the use of Virtual Machines as slaves.

vSphere Cloud Plugin

This plugin adds a way to control Virtual Machines hosted in a VMware vSphere server using Jenkins.  You can configure a Jenkins Slave to use a virtual machine, including an optional snapshot name.  If configured to do so, Jenkins will (optionally) revert, then start the virtual machine as a slave.  If configured to do so, Jenkins will also shutdown and (optionally) revert the virtual machine when all jobs have finished processing.

Additionally, you can manipulate your vSphere VMs & templates using any of the vSphere Build steps provided by this plugin.

Configuration

vSphere Cloud Configuration

The first step is to configure Jenkins to know what vSphere server you will be using. To do this you need to add a new "Cloud" in the Jenkins "Configure System" menu.

You will need to enter a display name for this vSphere cloud, the hostname of your vSphere server, and login credentials.  You can (in the Advanced... settings) also specify limits on the number of VMs to be created by Jenkins.

The "vSphere Host" is the name of the VMware vCenter service endpoint; it usually has a format of https://host-name-or-IP-address

The credentials specify the username and password used to log in to the vSphere Host.  If you do now have existing credentials defined for this within Jenkins will will need to "Add" them and then refresh the page.

The "Test Connection" button will test to see if your vSphere is accessible with the specified host name, user name and password.

The user entered when defining the cloud will need to have the following vsphere permissions:

Virtual machine*.Provisioning.*Clone virtual machine on the virtual machine you are cloning.

Virtual machine *.Inventory.*Create from existing on the datacenter or virtual machine folder.

Virtual machine*.Configuration.*Add new disk on the datacenter or virtual machine folder.

Resource*.*Assign virtual machine to resource pool on the destination host, cluster, or resource pool.

Datastore*.*Allocate space on the destination datastore or datastore folder.

Network*.*Assign network on the network to which the virtual machine will be assigned.

Virtual machine*.Provisioning.*Customize on the virtual machine or virtual machine folder if you are customizing the guest operating system.

Virtual machine*.Provisioning.*Read customization specifications on the root vCenter Server if you are customizing the guest operating system.

VM Clone Configuration

There are two ways to define the creation of a clone:

  • Static configuration: You can define the clone creation configuration globally. It can later be referenced in the Jenkins job configuration by using a "label". This configuration is done in the global Jenkins configuration.
    • This method is recommended for build jobs that "do not care about vSphere", e.g. basic compilation/test jobs which simply need a Jenkins slave to run on.
  • Per job configuration: You can define the clone creation configuration individually in each job. This configuration is done in the Jenkins job configuration.
    • This method is recommended for jobs which "know" that they are using vSphere to control a specific VM.

It is not uncommon to use both strategies, e.g. per-job configuration to create VM templates for later use in the global configuration.

Static configuration of clone creation

After adding a cloud you can configure Jenkins to create clones "on demand" by clicking on "Add vSphere Template" and then expanding the "Advanced..." section.  You may add as many templates that you wish.  Each template defines a means of creating a new VM to be used as a Jenkins slave which will automatically be used to clone new slaves as necessary.  You can control which templates get used by specifying labels on the templates and "Restrict where build can run" on the jobs.

 

.

    • Enter a "Clone Name Prefix", that name will be used when creating new clones. The new clone will be named: <clone_prefix><UUID>, where UUID is some dynamically created UUID added to the prefix.  Prefixes MUST be sufficiently unique that any given name can be uniquely matched to a single prefix.

    • The "Master Image Name" is the name of the master image which will be used to make the new clones.

    • Use snapshot: (optional) If ticked, the new clone will be based on a Snapshot within the master image instead of the "live" image.  Ticking this enables the following:
      • Snapshot Name: The name of the snapshot to use, e.g. "Latest".
      • Linked Clone: A linked clone is where the disks of the clone are defined as a delta from the template's disks, meaning that storage space if only required on vSphere for the differences, rather than each clone requiring its own (full) copy of the template's disks.  Recommended for short-lived VMs.
    • Enter Cluster, Resource Pool, Datastore, Folder, Customization Specification as required, this are settings how the clone will be created in vSphere. Enter "Resources" as default for "Resource Pool" if you haven't explicitly defined resource pools in vCenter.
    • Labels: You can use these labels to configure a job where it should be built. Use the label in the box "Restrict where this project can be run" in the job configuration.  This is how Jenkins decides which template to use.
    • Force VM Launch: Launches the virtual machine when necessary.
    • Wait for VMTools: Useful if the virtual machine has VMTools installed; when enabled, Jenkins will wait until VMTools is running before continuing with the connection allowing a lower "Delay between launch and boot complete" value without sacrificing reliability.
    • Delay between launch and boot complete: Number of seconds to delay after starting the virtual machine (or after waiting for VMTools) before assuming the slave is operational.
    • Disconnect after Limited Builds: Will force the slave agent to disconnect after the specified number of builds have been performed, triggering the disconnect action.
      Note: Currently, the "Disconnect After Limited Builds" configuration parameter is not used.
    • GuestInfo Properties: you can use "guestinfos" to provide properties (e.g. the URL to the Jenkins Master and the JNLP "secret") to the clone. This is especially useful if you chose "Java Web Start" (JNLP) as launch method, e.g. for a Windows VM.  See below for further details and example.
    • Retention Strategy: Whether to use each VM once before disposing of it (the "Run-Once" strategy) or to allow the VM to run multiple builds and only be disposed of once the VM has remained unused for too long (the "Keep-Until-Idle" strategy).  If your builds require a fresh VM then use "Run-Once", otherwise the "Keep-Until-Idle" will be more efficient.

GuestInfo properties (mentioned above) allow you to pass information from Jenkins to the newly started VMs.  If you are using the SSH launch method then you may not need this, but if you are using Java Web Start (JNLP) then this will be needed to tell your slave process where the Jenkins server is, what slave it is, the JNLP "secret" etc.  For example, to pass the data necessary for the newly-started VM to start a JNLP slave process and connect back to Jenkins, you could set the following GuestInfo properties:

You would then need to ensure that, after bootup, your VM would automatically use the vmware tools to reach each of these properties (e.g. vmtoolsd --cmd "info-get guestinfo.SLAVE_JNLP_URL" to request the jnlp URL) before starting the slave.jar with those arguments.

All the rest of the configuration variables are the same as when you define a Static Slave.   Please see below for information on setting those configuration parameters.

Per job configuration of clone creation

Add a new "vSphere Build Step" in the job configuration and select the desired action.

Build Steps

Build steps can be used to interact directly with your vSphere instances.  Typical use cases have been added as build steps.  If you would like to see more functionality in this plugin, please open an enhancement request ticket (and, ideally, implement the addition yourself and raise a pull request as well).

Clone VM from Template or VM

This build step will clone an existing Template or VM to a new VM. Linked clones are optional.  Cluster, Resource Pool, and Datastore can be specified.

Deploy VM from Template

This build step will create a VM from the specified template.  The template must have at least one snapshot before it can be cloned.  A linked clone may optionally be chosen.  The new VM will be placed in the same folder and storage device as the original template, and will use the specified ResourcePool and Cluster.

Convert VM to Template

This build step will mark the specified VM as a template. The VM must be powered down first, or the user must use the "force" option.

Convert Template to VM

This build step will convert a template into the VM. 

Delete VM

This build step will delete the specified VM.  At this time, templates will not be deleted by this build step.

As an additional precaution against accidental deletion, if the name or description associated with the vSphere Cloud changes after the Job has been saved, the job will need to be re-saved before it will function correctly.

WARNING: THIS IS A DESTRUCTIVE OPERATION THAT WILL DELETE THE VIRTUAL MACHINE WITHOUT ADDITIONAL CONFIRMATION.  IT CANNOT BE UNDONE.

Delete Snapshot

This build step will delete a snapshot, and optional allow consolidation of the disk.

Power-On / Resume VM

This build step will power on the specified VM and store the IP address in the "VSPHERE_IP" environment variable.   The build step will wait as long as specified in the "timeout" field (max 3600 seconds).  This build step will resume suspended VMs as well.

Power-Off VM

This build step will power off the specified VM.  There is an optional "Graceful shutdown", to attempt a shutdown via VMware Tools, if it's installed.

Reconfigure VM

This build step will allow selectively reconfiguration of the VM.  Available at this time are the following reconfigurations:

  • Edit CPU - change the number of cores and/or sockets.
  • Edit Disk - add a new disk
  • Edit RAM - change the amount of RAM
  • Edit NIC - Add, edit or remove a NIC interface, specify label, MAC address and port group.
  • Edit Notes - Add-to/replace the text "Notes" for the VM.
Rename Snapshot

This build step allows renaming of a snapshot.

Rename VM

This build step will allow renaming the VM

Suspend VM

This build step will suspend the specified VM. 

Revert to Snapshot

This build step will revert the specified VM to the specified snapshot.

Take Snapshot

This build step will take a snapshot of the specified VM using the specified Snapshot name, description, and whether or not to include memory in the snapshot. 

Delete a Snapshot

This build step will delete a specified snapshot from a specified VM. It will optionally consolidate all of the VM's disks.

WARNING: THIS IS A DESTRUCTIVE OPERATION THAT WILL DELETE THE SNAPSHOT WITHOUT ADDITIONAL CONFIRMATION.  IT CANNOT BE UNDONE.

Additionally, you can manipulate your vSphere VMs & templates using any of the vSphere Build steps provided by this plugin.

Virtual Machine configuration

Most of the vSphere plugin "cloud" functionality centers around the cloning of VMs and/or Templates with the objective of having those VMs appear as slave (agent) nodes on Jenkins and then be available to perform builds.  In order for this to happen, the VMs either have to connect to Jenkins, or have to allow Jenkins to connect to them, just like any other Jenkins slave (agent) machine (see Distributed builds for more details).  Failure to set up the VM correctly will mean that, while the plugin is able to start and stop the VMs, they won't be any use to Jenkins and won't run any builds.

There are two common methods by which this can be achieved, and both require the VM/Template to have been configured with the requisite software on it.

  • Launch slave agents via SSH

After the VM has booted up (and after the VMware tools have been detected), the Jenkins master will connect to the slave.

This requires that you configure your VM to run sshd, that it has Java installed, and that Jenkins has network connectivity to the VM's first IP address.  This means that you won't be able to get away with using a host-only network on vSphere for your VMs; you'll need them to have IP addresses that Jenkins can ping.

See the SSH Slaves plugin for further details on how to configure this.

  • Launch agent via Java Web Start

The Jenkins master does not connect to the slave; it relies on the slave connecting to the master.

This requires that you configure your VM to start the Jenkins slave process such that it to connects to the master. This means that you can use a non-routable network for your VMs (they only need NAT access to the Jenkins master's IP address; the Jenkins master doesn't need to be able to connect to them) but, unless you wish to hard-code knowledge of your Jenkins master into your slave VM, you should use the GuestInfo properties to pass the relevant information to the slave, and have the slave automatically read that information on startup prior to connecting to Jenkins.  Exactly how you do this is up to you.  The help for the "GuestInfo properties" section provides details of how to read the properties at runtime.

e.g. If you're using a *nix OS with upstart, you could run the slave from upstart by having a script read the GuestInfo properties, download slave.jar, then runs java -jar slave.jar with the appropriate arguments.

e.g. If you're using Windows, you could have the machine auto-login to itself, start a desktop session and have a .CMD script in the StartUp folder that achieves the same thing.  This example JenkinsSlaveStartup.cmd script requires 3 GuestInfo properties to be set: Property Name "JNLPURL" should be set to value "${JENKINS_URL}computer/${NODE_NAME}/slave-agent.jnlp", "SLAVE_HOME" should be set to "${remoteFS}" and "SLAVE_SECRET" should be set to "${JNLP_SECRET}" (all without the quotes).

Debugging

To see what's going on "under the hood", you'll need to see the logs generated by the code.  To do that, you're best advised to go to Manage Jenkins → System Log and define a logger called "vSphere".  In that, you'll then need to add all the individual loggers that the plugin uses ... unfortunately, this is not presented in a nice clean fashion (for historical reasons that are expensive to resolve).  The vSphere plugin code is not all under one nice neat package; you'll need to have a Logger for "vsphere-cloud" and a Logger for every Java class or package in the source code, while also taking care not to accidentally include anything that isn't vSphere (e.g. the package org.jenkinsci.plugins.workflow includes lots of general Jenkins logs, so it's necessary to specify org.jenkinsci.plugins.workflow.vSphereStep instead) and set the Log level for each of these to "ALL".

Once done, you'll end up with a log definition that looks something like this:

Once you've got that, save it, and then you'll see any future activity in that system log - these logs are likely to reveal more detail than shows up in any build logs, and may help explain why slave node VMs aren't coming online as expected etc.

Notes & Compatibility

  • In some cases, starting a VM might cause vSphere to prompt the user to answer a question regarding the copying or moving of a VM.  This plugin will use the default answer, which is "I copied it."
  • A great deal of thanks to the authors of the Lab Manager Plugin; their code was heavily copied to make this plugin.

This plugin is supposed to work in the following environments:

  • A licensed vCenter 4.0 and higher.  An evaluation license for vCenter should work.
  • A licensed ESX 4.0 or higher, or ESXi 4.0 or higher stand alone host. For some actions like start/stop of a VM the ESX(i) is sufficient, for other actions you need the vCenter management application.
  • NOT TESTED, but should work in theory: VMware Server

This plugin will NOT work with the following VMWare products:

  • VMWare Player, Fusion, or Workstation (no remote APIs)
  • The free ESXi server (remote APIs only work for the vSphere Client)

Change Log

Version 2.21 (Sept 24th, 2019):
Version 2.20 (July 8th, 2019):
  • Bugfix: Cope with ssh-slaves version 1.30 onwards (PR-104).
  • Misc code tidying (PR-102).
Version 2.19 (Feb 26th, 2019):
  • Enhancement: Graceful shutdown timeout is now customizable (PR-97).
  • Correct button text on Folder configuration (PR-99).
  • Improve appearance of slave WebUI page (PR-101).
Version 2.18 (July 23rd, 2018):
  • Enhancement: Added "Reconnect and Revert" VM slave option (PR-90).
  • Enhancement: Improved localization support + code tidy-up for cloud agent retention (PR-91).
  • Enhancement: Don't hard-code a '-' into the VM name (PR-92, PR-93)
  • Enhancement: Implement "Reconfigure Notes" build step (PR-94).
  • Enhancement: Show VM information (including any Notes) on Jenkins' .../computer/... page (PR-95).
Version 2.17 (March 26th, 2018):
  • Fix security issue: Enable SSL/TLS certificate validation by default.
  • Fix security issue: Properly implement access control and CSRF protection on form validation related URLs to prevent credentials capturing and denial of service.
Version 2.16 (Sep 7th, 2017)
Version 2.15 (Jan 2nd, 2017)
  • Folder parameter (PR-61)
  • Folder configuration support (PR-62)
  • Add new parameter ignoreIfNotExists (PR-63)
Version 2.14 (Oct 27th, 2016)
  • Expose customization spec setting (PR-60)
  • JENKINS-39232 Make vSphereCloudLauncher inherit from DelegatingComputerLauncher (PR-59)
  • Re-fix JENKINS-24661 (PR-58)
  • JENKINS-20743 part two (PR-57)
  • Implement JENKINS-20743 (PR-55, PR-56)
  • Implement JENKINS-38269 (PR-54)
  • Enhance JENKINS-22437 (PR-53)
  • Fix JENKINS-24605 and JENKINS-24661 (PR-52)
  • Fix for JENKINS-38249 (PR-51)
  • Bugfix jenkins 36878 (PR-50)
  • Implement JENKINS-22437 (PR-49)
  • Fix JENKINS-36878 and JENKINS-32112 (PR-48)
  • Fix for JENKINS-38030(PR-47)
  • Change version of yavijava that we fetch from 6.0.03 to 6.0.04 (PR-46)
Version 2.13 (June 7th, 2016)
  • Enabled Pipeline support (PR-45)
  • Handle disconnect and temporarily offline (PR-44)
Version 2.12 (Mar 25th, 2016)
Version 2.11 (Feb 24th, 2016)
  • Fixed naming issues with templates (PR-42)
Version 2.10 (Jan 26th, 2016)
  • Slave naming fixes (PR-41)
Version 2.9 (Dec 30th, 2015)
  • Power state after deploy (PR-40)
  • Provided fixes (PR-39)
Version 2.8 (Dec 11th, 2015)
  • Dynamic On-Demand slaves (PR-35)
  • Set DataStore when reconfiguring disks (PR-37)
  • Distributed vSwitch Support (PR-38)
Version 2.7 (Nov, 20th, 2015)
  • Clone with resource pool: PR-33
Version 2.6 (June 10th, 2015)
  • Handle null GuestInfo during PowerOn via PR-31
  • Additional time outs during PowerOn via PR-32
Version 2.5 (May 18th, 2015)
  • HelpDoc: Set correct name for ip address variable via PR-27
  • ClusterFix: Don't assume that the cluster should exist via PR-28
  • SelectableCloud: Cloud should be selectable via PR-29
  • Logging: Log stack trace if exception message is null via PR-30

Version 2.4 (February 26th, 2015)

  •  Disconnect from vSphere after performing actions, contributed by ParagDoke via PR-26

 Version 2.3 (February 3rd, 2015)

  • Add build step for exposing GuestInfo.  Contributed by vorbis via PR-22
  • Get datastores directly from cluster. Contributed by vorbis via PR-23
  • VsphereSelection: Allow dynamic selection of vsphere cloud. Contributed by vorbis via PR-24
  • Optimization contributed by ParagDoke via PR-25
Version 2.2 (January 16th, 2015)
  • Contribute feature: Change/Add disk, by calj via PR-19
Version 2.1 (January 6th, 2015)
  • Fixes contributed by Oleg Nenashev in relation to  JENKINS-25588 
Version 2.0 (December 24th, 2014)
  •  Use Jenkins Credentials Plugin for username/password.  JENKINS-25588  Contributed by Oleg Nenashev
  • Fix for missing DataStores, contributed by calj
Version 1.1.12 (September 26th, 2014)
  • Possible fix for disconnect, shutdown, power on issue
  • Eric added some new VM actions:
    • Clone from template/VM
    • Rename VM
    • Rename Snapshot
    • Added Datastore name to Clone/Deploy
    • Added VM Reconfigure, to adjust CPU, RAM and NIC interfaces
Version 1.1.11 (July 8th, 2014)
  • JENKINS-21647: Allow deployment without a resource pool
  • Added a Graceful Shutdown to the Power Down build step
  • Additional fixes to handle multiple slave shutdowns and job runs on powered down slaves.
Version 1.1.10 (May 15th, 2014)
  • Modified the previous fix for preventing jobs running on slaves that had limited test runs enabled.  The original fix wasn't robust in terms of Jenkins restarts.
  • NOTE: A cleanup of the code revealed that the plugin was storing information in the slave configuration that it wasn't supposed to.  When this version of the plugin is installed, some old data may have been stored and Jenkins will note this on the Manage Jenkins page.  If the data is in regard to "isStarting" and "isDisconnecting", then it should be fine to delete that old data with no adverse effects.
Version 1.1.9 (May 12th, 2014)
  • Fixed problem where VM restarts due to limited test runs would result in the next build failing
Version 1.1.6 (April 25th, 2014)
  • Pull 12: from hrabbouh.  Added new disconnect options and an error case where disconnects were being called too many times.
Version 1.1.5 (March 13th, 2014)
  • JENKINS-22025: "VM cannot be started" repeatedly with vSphere plugin
  • JENKINS-21312: Vsphere plugin keeps reseting the slave when slave is configured for on demand start
Version 1.1.4 (January 30th, 2014)
  • When configuring a job, the "ServerName" drop-down now properly defaults to the saved value. (JENKINS-21580).
Version 1.1.3 (January 7th, 2014)
  • Add "Delete a Snapshot" build step (JENKINS-20793).
  • Update vijava dependency to 5.1.
Version 1.1.2 (November 26th, 2013)
  • Fixed a race-case type issue where Jenkins would disconnect a slave that was in the process of trying to connect.
Version 1.1.1 (November 6th, 2013)
  • Fixed an issue where certain exceptions, including those thrown during the initial connection to vSphere, were not logged to the console.
  • Adding "Resume" to "Power-On" build step title.
Version 1.1.0 (October 4th, 2013)
  • Condensed all vSphere build steps into a single "vSphere Build Step" container
  • Added more build steps; Made existing build steps more granular (JENKINS-19702)
    • Convert VM to a template
    • Convert template to a VM
    • Delete VM
    • Deploy VM from template
    • Power On VM
    • Power Off VM
    • Revert to Snapshot
    • Suspend VM
    • Take Snapshot
  • Deployment log now prints the name of created VM (JENKINS-19436)
  • Added configurable timeout to Power On Build Step
  • More code cleanup, restructuring, and refactoring
  • Due to the restructuring of classes, this plugin's job configurations won't survive the upgrade from 1.0.x to 1.1.0 or above.
Version 1.0.2 (August 28, 2013)
  • Massive code cleanup, restructure, and refactoring.
  • Help text has been overhauled to work better with Jenkins standards (localizable, linkable).
Version 1.0.1 (August 22, 2013)
  • Added build step functionality. The following build steps are now available: Create VM from Template, Convert VM to template, Convert template to VM, and Delete VM.
  • Minor behind-the-scenes code cleanup and re-factoring. 
Version 0.10 (May 25, 2012)
  • Fixed JENKINS-13722
  • Undid all the slave launching logic - the logic plus JENKINS-13735 was resulting in slaves that would never start.  Slave STILL never start in some cases, but once the Jenkins bug is fixed, it should begin working better.
Version 0.9 (May 7, 2012)
  • Overhaul of the slave launching logic.  Multiple slaves will be launched if unique.
  • Fixed JENKINS-17323 Log lines should print with new lines.
Version 0.8 (May 4, 2012)
  • Fixed JENKINS-13675 Fixed an NPE in some race cases involving slave startups
Version 0.7 (May 2, 2012)
  • Re-release due to problems in the release process.
Version 0.6 (release process failed)
  • Fixed JENKINS-12241 Test Connection fails if vSphere host contains trailing slash ('/') character
  • Fixed JENKINS-12163 Cleaning VM before start next job in queue.
  • Fixed JENKINS-13537 Not perform "revert snapshot" for all vmwares of a label
Version 0.5 (Dec 8, 2011)
  • Added "Reset" to the disconnect behavior.  Will do a Reset VM when the agent disconnects.
  • Added "Nothing" to the disconnect behavior.  Nothing at all will happen when the agent disconnects.
  • Updated the VM Java API to the latest version.
Version 0.4 (Oct 8, 2011)
  • Initial Release

141 Comments

  1. Unknown User (ftaysse)

    Really handy when you need to build on a specific machine, great plugin.

    A welcome addition would be the possibility to choose a "What to do when the slave is disconnected?" -> "Do nothing" option, so that the slave VM does not shutdown whenever jenkins is restarting (eg: during jenkins plugin installation)

    1. Unknown User (jswager)

      Reset and Nothing are two idle actions being added to the next version.

  2. Unknown User (cforce)

    Great work!

    We are hardly missing one feature.

    Normally most application ar n-tier systems, so you would need several machines to be setup for a jenkins job. E.g The project is compiled and then db server, app server and web server are reverted to its know default state. Then the aplliucation is deployed in the app server, and the test are run coresponding to the data which is existing in the database machine snapshot which was restored. The test cases of the job's junit test are now running against the web server.

    THe feature is needed would be to be able to add multiple vsphere slave configuration per job.

    Because this slave machine configurations could be reused across all jobs, the config should be handled in the slave configuration section outside of jobs. Jobs shall only be able to use this slaves (including snapshot restore config nad startup options), how they are configured as slave itself. So just check the slave to be "used" as part of a pre or post action.

    1. Unknown User (jswager)

      I'm not sure what you are asking for.  Are you asking that each slave (vSphere Cloud) really be MORE than one VM, so that when the actual slave VM is started, so are additional ones?  In other words, you want to have a job (or test) that requires multiple slave machines?

    2. Unknown User (ftaysse)

      Running integration tests this way - reverting the three application server, db server, web server machines to a snapshot and running the integration test on the web server - can be problematic.
      You can ensure a single job is running on the jenkins slave (eg. "web server" machine), through plugin or a single executor configuration, but you cannot ensure other machines (eg. DB server) - that are not slaves of jenkins - are not used by anyone else during the test
      Strictly speaking, running integration tests by reverting a single machine containing all three servers is not clean either, but it's much more easy to ensure a single client can access the system under test during the run (eg: single executor slave + firewall).

      The scenario you describe would also require that the job can control the "revert snapshot" action (if the integration test job is started and the slave is alerady online, no revert is made, and the system is not properly set up for the test).
      That could be a nice feature, but this may have some side effects that need some careful thought... (how do we know the snapshot exist if the job can run on multiple slaves, how do we handle sumiltaneous jobs making or reverting snapshots on the smae slave, etc...)

      However, it is true that the ability to start multiple machines and/or start a vApp (multiple machines group defined in vsphere) on the slave configuration can be interesting.
      The perspective of delegating the snapshot control to the job could lead to some interesting scenario, particularly for the "deploy" jobs, where you could automatically roll-back should the job fail

      PS: thanks for the update jswagger, no more impromptu shutdowns of my slave server when restarting Jenkins, my sys-admin is now a happy man :)

      1. Unknown User (jswager)

        I completely agree here - it's pretty problematic to try to control two VMs in the context of one slave and one VM.  Not to mention that its a pretty limited scenario, only good for a few things.

        A much better approach would be general feature added to the Jenkins core.  An approach where jobs are "teamed" together.  Each job targets a single slave.  But all the "teamed" jobs have to be simultaneously startable before any one starts.  That way, one job can target the webserver and do the deployment and testing, another job targets the database server and setting it up properly for the test run.  There would need to be some form of communication/synchronization between the jobs, a way to have one job wait until the other job has reached a certain point - much like synchronization is for mutithreaded applications.

        This approach would work regardless of slave type - whether it be a VM with revert capabilities or any other type of Jenkins slave.

  3. Unknown User (sylvainc)

    Hi, it's a very useful plugin. But I have a little comment for a specific case. In fact I want to revert to a snapshot before each job execution but if several jobs try to run on the VMWare at the same time then the VMWare can't revert to the snapshot. Do you have a solution, thank you ?

    1. Unknown User (jswager)

      At the moment, I don't have a solution incorporated into the plugin.  But I'm working on it.  In the meantime, here's a workaround:

      1) Configure the slave with a disconnect type of Revert or Reset.
      2) In your test job, mark the slave temporarily offline.  This will prevent further jobs from targeting the slave. It would be something like this: 
      curl -d "offlineMessage=back_in_a_moment&json=%7B%22offlineMessage%22%3A+%22back_in_a_moment%22%7D&Submit=Mark+this+node+temporarily+offline" http://JENKINS_HOST/computer/NODE_TO_DISCONNECT/toggleOffline

      3) As a final step in your job, start a delayed shutdown of the VM.  If it's Windows, something like "shutdown /s /t 

      Unknown macro: {number of seconds}

      " will do it.  The trick is getting enough delay for the slave to completely finish the job (such as archive file uploading and such) before the VM shuts itself down.

      I haven't tried this workaround, so it might need additional tweaking. 

      1. Unknown User (buraianto)

        I also am interested in being able to revert to snapshots with each job execution. Have you had any time to make progress toward a solution? I am currently use this plugin and I appreciate the work you've already done on it.

        1. Unknown User (jswager)

          Not yet.  Did the workaround work for your situation?

          1. Unknown User (sylvainc)

            My solution :

            • Configure the slave with a disconnect type of Shutdown.
            • Create two new jobs :
            • SwitchOffLineNode (Change status of a node from Online to Offline)
              With a string parameter : name = NODE
              With a build step of windows batch type :
              cd %CURL_HOME% && curl \-k \-u USERNAME:USERPASSWORD \-d "offlineMessage=back_in_a_moment&json=%7B%22offlineMessage%22%3A+%22back_in_a_moment%22%7D&Submit=Mark+this+node+temporarily+offline" %JENKINS_URL%computer/%NODE%/toggleOffline > nul 2>&1
              Replace USERNAME & USERPASSWORD
            • SwitchOnLineNode (Change status of a node from OffLine to Online)
              With a string parameter : name = NODE
              With a build step of windows batch type:
              cd %CURL_HOME% && curl \-k \-u USERNAME:USERPASSWORD \-d "Submit=This+node+is+back+online" %JENKINS_URL%computer/%NODE%/toggleOffline > nul 2>&1
              Replace USERNAME & USERPASSWORD
            • Configure your job :
              First build step "windows batch command":
              set json="{\"parameter\": [{\"name\":\"NODE\", \"value\": \"%NODE_NAME%\"}], \"\": \"\"}"
              set url=%JENKINS_URL%/job/SwitchOfflineNode/build?delay=60sec
              cd %CURL_HOME% && curl -k -u USERNAME:USERPASSWORD -X POST %url% --data-urlencode json=%json%
              Replace USERNAME & USERPASSWORD
              This build step switch offline the vmware to stop new build
            • On "Post build task" of your job (download post build plugin)
              Log Text = "Building" (to execute this task all the time like a "finally")
              Script
              set json="{\"parameter\": [{\"name\": \"NODE\", \"value\": \"%NODE_NAME%\"}], \"\": \"\"}"
              set url=%JENKINS_URL%/job/SwitchOnlineNode/build?delay=60sec
              cd %CURL_HOME% && curl -k -u USERNAME:USERPASSWORD -X POST %url% --data-urlencode json=%json%
              shutdown -s -f -t 30
              Replace USERNAME & USERPASSWORD
              This script turns off the vmware and after switches online the vmware to execute new job.

            I use this solution to run "testcomplete" (graphic tests) jobs and I must run tests from a clean machine.

            Sylvain

            1. Unknown User (larsky)

              The workaround work, but we need 2 extra jobs for each real job. It very expensive for us (we have 30+ jobs with VM tests).

              @ jswager Can I hope for fixing https://issues.jenkins-ci.org/browse/JENKINS-12163? I can help with testing.

          2. Unknown User (buraianto)

            I've been using a workaround that seems to be working, though with the caveat that we are only running one build job at a time.

            I have a master job that calls the sub jobs that need to be isolated from each other by a reversion to snapshot. I have set the slave to be disconnected when unused for 1 minute, and in between the sub jobs I run a Window batch command to wait for a few minutes. This gives the slave enough time to become disconnected and revert to snapshot.

            Like I said, I'm only running one job at a time. This workaround wouldn't work if I ran the sub jobs individually and queued up the jobs such that the slave was never disconnected. But it works for now.

  4. Unknown User (prenaud76)

    We are using vCloud Director here and not vSphere. Would you know if this plugin will work for Cloud Director?

    Note that I did not even try it out yet, as I thought someone might know already and would spare me from spending time testing something that is known to not work...

    Tx!!

    1. Unknown User (jswager)

      If I understand things correctly, vCloud Director it just a high level abstraction API that still requires vCenter (part of vSphere) to manipulate VMware based virtual machines.  So somewhere in your vCloud, there should be a vCenter.  And that vCenter is what this plugin will talk to.  So, in theory, it should work.  But since I don't have a vCloud installation, I can't test it to be certain.

    2. Unknown User (jmatcuk)

      Hi Patrick, have you done any testing yet?  I have access to a company vCloud Center but when I try to use the plugin on the url, I receive an exception prior to the login credentials are even checked.Caused by: java.rmi.RemoteException: VI SDK invoke exception:org.dom4j.DocumentException: Error on line 130 of document : Open quote is expected for attribute "

      Unknown macro: {1}

      " associated with an element type "href". Nested exception: Open quote is expected for attribute "

      " associated with an element type "href".
      at com.vmware.vim25.ws.WSClient.invoke(WSClient.java:182)
      at com.vmware.vim25.ws.WSClient.invoke(WSClient.java:124)
      at com.vmware.vim25.ws.VimStub.retrieveServiceContent(VimStub.java:1521)
      at com.vmware.vim25.mo.ServiceInstance.<init>(ServiceInstance.java:85)
      at com.vmware.vim25.mo.ServiceInstance.<init>(ServiceInstance.java:69)
      at org.jenkinsci.plugins.vSphereCloud$DescriptorImpl.doTestConnection(vSphereCloud.java:311)

      My investigation so far is leading me to believe that it is not compatible with vCloud directly. I will need to speak with my admins to determine if I can have access to vSphere to control vCloud through it.

  5. Unknown User (cforce)

    Is vcenter apiWhat version/liencse of vspehere/vcenter do i need?

    Which api is used? - vcenter api or vcloud api

    Does it work on vspehere 5 ?

    Which ports and protocols do my newtorkers have to shoot through firewalls in beetwen jenkins server an vsphere center servers?

    1. Unknown User (jswager)

      This plugin was built using the vSphere 5 API.  The plugin will use the same ports/protocols as the vSphere Client to connect to the vCenter server.  Whatever VMware documents as being the requirements for vSphere Client access will apply to this plugin.  The following list will probably help: http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1012382

      1. Unknown User (cforce)

        Well, i can't open 50 ports or so. I would guess its only port 443 /https needed.

        vSphere Management Assistant

        443

        TCP

        vSphere Management Assistant

        ESX Host

        For SDK traffic

        But being sure would be good to know

        1. Unknown User (jswager)

          In the list, any that have a source of "vSphere Client" and a target of of "vCenter server" and product is vCenter 5.x, that's the ones you'll want.  Which really does just leave 443. 

  6. Unknown User (sylvainc)

    I have 3 identical vmwares with the same label but when I run a job restricted to this label Vsphere plugin makes a revert snapshot action on all vmwares which have this label. Maybe you have an explanation ?

    1. Unknown User (jswager)

      Yes, I do have an explanation.  I have the same problem too, so I can confirm what you are seeing.

      When Jenkins gets a job it queries each slave if it can handle the job.  Jenkins handles label checking on its own, but it the labels pass, then the slave is asked "can you run this".  For the vSphere slaves, they answer Yes if they CAN run, but the slave isn't currently up.  So Jenkins says "start up, please".  The slave begins the start up up work, which can take several minutes.  Jenkins moves onto the next matching slave and goes through the same process.  The result is that every powered down vSphere slave that COULD run the job will be up and running, but only the first one to actually get connected first will get the job.

      This is something I've been trying to solve for some time, but without much luck.  Some of my jobs trigger 7 separate slaves into action.  If you could submit a JIRA issue against this plugin, it would be helpful for tracking purposes.

    2. Unknown User (pjdarton)

      JENKINS-36878 - Getting issue details... STATUS was fixed in 2.15

  7. Unknown User (b_kuerzinger)

    Hi, shutting down the slave works well. But I have problems to start the slave afterwards again and reverting does not work either. Can someone please help me? Thank you.

    1. Unknown User (jswager)

      When you shutdown the slave, do you shut it done from vCenter or disconnect it from Jenkins?  The slave should start (optionally, revert and start) when Jenkins needs the slave.  That will depend on the slaves scheduling and usage.  There is currently an issue with the latest version where issues with slave starting could cause a very long delay before your desired slave actually starts back up (well, never in some cases...)

      Also, do you have a snapshot name specified?  That's the only way a revert will actually work.

  8. Unknown User (mbelosto_12)

    Hi, I have similar problem I think,

    The plugin cannot start any of my machines After upgrade to latest plugin version...

    When I go into the log I can C only the "spinning circle"

    Can someone advice?

    1. Unknown User (jswager)

      I have started hearing reports of this.  The initial reports though suggested that Jenkins would eventually start the slave, but not right away and not in a reasonable amount of time.  Eventually, the slave you wanted to start WOULD start, but Jenkins was busy starting a bunch other slaves first, or attempting to start them.  I believe that this due to a combination of two things: 1) recent changes in the plugin to prevent simultaneous starting of ALL VMs that match a label and 2) odd behavior in Jenkins where a job that can't possible run on a particular slave still triggers the slave to be started.

      If you could submit a JIRA issue for tracking purposes, that would be great.  I'm looking into possible solutions now.  

      1. Unknown User (mbelosto_12)

        How can I downgrade to version 0.5(I think this is the last good working for me)

        It didn't work for me well When I tried to downgrade using Jenkins Plugin manager..

        1. Unknown User (jswager)

          Unfortunately, I'm not sure HOW to downgrade.  I'm not sure where Jenkins stores the older HPI files, or if they even do.  However, the good news is that if you upgrade to the latest vSphere Cloud Plugin (v0.10) AND upgrade to the latest Jenkins, things will begin to work much more smoothly.  Jenkins has had a core defect in the slave retention code that made the vSphere Plugin work pretty poorly.  This has been fixed in 1.468 and higher and GREATLY improves the slave launching.

  9. Unknown User (jglick)

    <dependency>
        <groupId>com.cloudbees.thirdparty</groupId>
        <artifactId>vijava</artifactId>
        <version>5.1-beta</version>
    </dependency>
    
  10. Unknown User (larsky)

    This plugin working with vSphere 5.1 and JWM connection?
    I can't configure it.

    https://wiki.jenkins-ci.org/display/JENKINS/Scripted+Cloud+plugin don't working too.

    1. Unknown User (jswager)

      I haven't tested with vSphere 5.1.  Although if the APIs are unchanged, I would expect it to work.  What is a JWM connection?

  11. Unknown User (larsky)

    I confirm that plugin doesn't work with vSphere 5.1.

    Issue: https://issues.jenkins-ci.org/browse/JENKINS-16535

    1. Unknown User (jswager)

      We've been running the plugin against vSphere 5.1 for the last couple months now, no obvious problems.  Had some issues getting the SSO setup, but that was a vSphere problem.  What problems are you having exactly?

  12. Unknown User (larsky)

    The problem was in my settings. See https://issues.jenkins-ci.org/browse/JENKINS-16535 comments.

    Generally all work. :)

    Will try test and use it.

  13. Unknown User (vsengutt)

    I'm a little new to Jenkins, but how do I get to the vSphere Slave Configuration screen (2nd screenshot) Thanks in advance

    1. Unknown User (jswager)

      Create a new slave (Manage Jenkins->Manager Nodes->New Node) that is of type "Slave virtual computer running under vSphere Cloud".

  14. Unknown User (vsengutt)

    I have an issue when two jobs are queued to run on the same machine. I have the VM to be disconnected after 1 build. It's then shutdown, reverted and started. After the 1st build runs, it triggers the VM to shutdown. However, before the VM can get disconnected, the second build that was queued starts running and it fails. Is anyone else seeing this or is it just me?

    1. Unknown User (jswager)

      Could you please submit a JIRA incident for this?  That way - it can be tracked if a real problem, and if not - a workaround/configuration change can be documented.  But I suspect it's a bug.

  15. Unknown User (miked)

    I'm having problems reverting to a snapshot on startup. The connection tests indicates that everything is fine but the slave doesn't start.
    Jenkins log indicates the following loop :

    Mar 5, 2013 4:23:40 PM org.jenkinsci.plugins.vSphereCloud InternalLog
    INFO: [vsphere] Starting Virtual Machine...
    Mar 5, 2013 4:23:40 PM org.jenkinsci.plugins.vSphereCloud InternalLog
    INFO: [vsphere] Reverting to snapshot:Boxer A
    Mar 5, 2013 4:25:21 PM hudson.slaves.SlaveComputer tryReconnect
    INFO: Attempting to reconnect vsphere
    Mar 5, 2013 4:23:40 PM org.jenkinsci.plugins.vSphereCloud InternalLog
    INFO: [vsphere] Starting Virtual Machine...
    Mar 5, 2013 4:23:40 PM org.jenkinsci.plugins.vSphereCloud InternalLog
    INFO: [vsphere] Reverting to snapshot:Boxer A
    Mar 5, 2013 4:25:21 PM hudson.slaves.SlaveComputer tryReconnect
    INFO: Attempting to reconnect vsphere

    If I leave the snapshot field empty, everything works fine

    What can it be ?

    Thanks

    1. Unknown User (jswager)

      When you specify a snapshot and click "Test VM Connection", it does say "Virtual Machine found successfully"?  If not, that would be the first problem.

      The "attempting to reconnect vsphere" is an odd message.  That's coming from the vijava library; for some reason, it's connection was terminated.  Which might be the cause of the problem.

      Does the vSphere user that Jenkins is using have permissions on that particular VM to revert?

      1. Unknown User (miked)

        When i click "Test VM Connection" everything works fine. it even recognizes the snapshot name, because if i put a wrong one it tells me that it doesn't exist.

        Also, if i leave the snapshot field empty, everything works fine

        The user has permissions to revert snapshots when i test it from the vsphere client,

        Does the API need different permissions ? maybe  authorization problem from the Jenkins machine ?

        1. Unknown User (jswager)

          The same authorization that's used to query the VM and snapshot presence is used to power on, power off, revert, etc.  So as long as the user has permissions to revert as well, then it should work.

          Are there any error messages on the vSphere side when the revert operation is attempted?  Can you even see the revert attempt in the vSphere task list for the VM?

          Another possibility to try: Set the slave to no snapshot.  Get it running and connected.  Then, while connected, set the snapshot to a valid value.  Also, set the "What to do when the slave is disconnected" to "Shutdown and Revert".  Then disconnect the slave (kill the slave agent or hit Disconnect in Jenkins).  See if both the power down AND revert commands are sent to the VM by looking at the vSphere task list.

          At this point, it would be a good idea to submit a JIRA incident and track the rest of this conversation there.

          1. Unknown User (miked)

            Thanks, I tried what you suggested, didn't work, got a NPE and more.

            I documented it in Jira :

            https://issues.jenkins-ci.org/browse/JENKINS-17095

  16. Unknown User (fschott)

    Hi,

    I installed this plugin to my Jenkins instance and configured the cloud in the jenkins settings. The connection tests says 'OK'. But now I'm missing the part for the Slave Configuration? Where is it? Am I blind, I can't see it in the job configurations anywhere. Could you point me to where it should appear?

    1. Unknown User (jswager)

      You need to add a new Slave and check "Slave virtual computer running vSphere Cloud" as it's type, rather than the normal "Dump Slave" type.

  17. Unknown User (portilha)

    Hi,

    This is really a great plugin, i am using it for a time and it runs smooth, but i have a small feature that i would like to found in this plugin that is the possibility to put a build step that manipulates the state of a virtual machine inside the vsphere arquitecture, you already have the API made, I think it would be easy to extend the possibility of include the functionality you already implement to build steps, because i am building an arquitecture of tests where the machine are really small and i don't want to include them as jenkins slaves.

    I know that i can change the state be executing the command line of the vmware, but it would be really great to use the same configuration from the Jenkins Configuration that your plugin already configures to do it, and the easy of configuration would be great.

    Please let me know what you think of my idea, if help is needed, i would be available to help out. I know a few things about Java programming :).

    Thanks for your time. 

    1. Unknown User (jswager)

      Would you expect the build steps (probably Power On, Power Off, Revert - I'm guessing) to execute against the VM slave that the rest of the build steps (Shell, etc) of the same job is running on?  If so - then it won't work.  That particular feature has been requested, but Jenkins has a strict policy: when the slave agent running a job is terminated (which those build steps would do), the job immediately fails.

      It's possible to have the build steps present, but they would have to execute against a slave OTHER than the one running the job.

      1. Unknown User (portilha)

        Hi jswager,

        Thanks for the previous reply, i don't want to run the commands against any VM jenkins slaves, but against other machines in the vsphere that are not slaves, for example, i want to bring a VM up to update some binaries and then bring turn it down (or suspend it). I want to create a job that does this for several machines that jenkins have no knowledge about.

        You said it is possible to have the build steps present, how can i do that?, i didn't find any build steps related to manipulating vm states.

        1. Unknown User (jswager)

          In the present version of the plugin - there are no build steps to handle VM manipulation.  However, there is a contributor that is considering adding some such steps to the plugin.  There were some issues that they were resolving.  If they are able to finish the work, I can update the plugin to have build steps such as Power On/Power Off/Revert.

          1. Unknown User (portilha)

            Can you tell me who is the contributor so i can speak to him to find out if it needs some help, as I said before, I know a little bit of Java and I would like to contribute if needed.

            Thanks,

            1. Unknown User (jswager)

              Taking conversation to email.

        2. Unknown User (cforce)

  18. Unknown User (prashant_katti)

    Hi JS,

    I have an ESX server who has not been configured in cluster mode and whenever i configure using this plugin i m not able to check data. The resource pool parameter is empty as it does not exist.

    Do we have some setting to handle this situation.

    Regards,

    Prashant 

  19. Unknown User (odklizec)

    Hello,

    Could anyone please explain, what exactly means this line in 1.1.0 Change Log:

    Due to the restructuring of classes, this plugin's job configurations won't survive the upgrade to 1.1.0

    Does it mean that all previous Jenkins jobs need to be created from scratch? Or just that the VSphere-related build steps must be configured again? Thank you in advance.

    Cheers,
    Pavel

    1. Unknown User (jswager)

      It means that the vSphere BUILD STEPS - introduced in 1.01 - won't survive the upgrade.  If one of these build steps was referenced in a 1.01 install, then the system was upgraded, the build steps would just disappear.  I think Jenkins might report that there was some "unused data" or something on it's management page and allow you to clean it up.  But essentially, the build steps just disappear.

      The newer format is much more robust and should handle future upgraded much better.  The new version is also much easier on the UI.

      1. Unknown User (odklizec)

        Hi and thank you for the reply. It makes sense now ;)

        Cheers,
        Pavel

  20. Unknown User (dharmsheta)

    Thanks jswager for making recent enhancements. We've been using plugin since quite a while and it has helped us to overcome many challenges to our test infrastructure for provisioning test environments. 

    Recently we'd use case to back up our VMs running under VMware Workstation(PS:you mentioned that plugin doesnt work with VMWare workstation. Appratently it doesn work with  workstation v 8, 9, 10). 

    Preferred method of taking back up is taking a snapshot and copying VMs to backup drives as per needs. So far everything has been workin nicely. However  I do miss one thing is "delete a previous snaphot " or "delete a given snapshot" of vm using jobs build step: an anti action to "take sanpshot" feature in nutshell. I believe this feature make lot of sense for us saving diskspace.  If you think it's a good idea then I can create a ticket on JIRA for this feature and if time permits me, I think of can contributing this feature to plugin. 

    1. Unknown User (jswager)

      I can't take credit for the recent enhancements; those are the fine work of Eric Lordahl (elordahl) a big time contributor to this plugin.  I'm glad that the changes have made it easier to work with.

      I could see use for a "delete snapshot" step in your case.  I would very much recommend creating a JIRA ticket to track this feature request.  Describe in detail what you're thinking of.  And if you want to contribute a change, please feel free to make changes and send them along as a pull request.  

  21. Unknown User (gspaldin)

    When I try using the plugin to Power-On/Resume a VM Jenkins gives me an error, Error: Could not get IP for <machine-name>. I'm not sure what I am doing wrong. Does anyone have any Ideas?

    1. Unknown User (jswager)

      You will most likely get this if VMWare Tools is not installed in the VM.

  22. Unknown User (josesa)

    It would be great if it was possible to use a saved "guest customization spec" when performing the VM deployment from Template.

    You can find these specs in Vcenter > Home > Management > Customization Specifications Manager.

    1. Unknown User (jswager)

      Best way to track this would be to submit a JIRA feature request.

      Customization Specifications are a bit tricky.  I think it's possible to get a list of them, although permissions can be a pain to setup.  The problem that I foresee is that some specifications allow for additional data entry.  Reading those specs, then building a dynamic UI for it would be very difficult.  Interested in adding those code changes yourself?

      1. Unknown User (fschott)

        Hi,

        I would also need such a feature. For my special case, it would be sufficient to be able to use specifications with no interactive input.

        Has a request already been added to Jira? I also would perhaps add this functionallity (well, the non-interactive one mostly) to the plugin, because I really need it for my project.

        1. Unknown User (jswager1)

          No JIRA submitted yet.  That would be the best thing to do.  Even better; feel free to grab the code and submit a code pull.  

    2. Unknown User (pjdarton)

      This was added in 2.15

  23. Unknown User (josesa)

    Another interesting feature would be that instead of defining a new slave static configuration you could define a new slave dynamic configuration (similar to EC2 plugin).

    The purpose would be to have a pool of vms based on the same template that would be created on demand depending on the jobs queue size and destroyed after some inactivity time (basically the "cloud" behavior that seems missing from this plugin).

    1. Unknown User (jswager)

      JIRA feature request would be the best way to track this.

      This issue of dynamically creating slaves has come up several times.  With the addition of the deploy build steps, its theoretically possible to add this.  It would have to be extremely limited and rigidly defined though.  Probably nothing more than a simple clone to a specific folder, data store and host/resource pool.  But then everyone wants something else - customiztion specs, IP address assignments, dynamic hardware changes, network changes.  I notice that the EC2 plugin has a very simple interface; you're given only a choice of the instance - that's it.  This sort of feature would have to be kept pretty simple.  We're not out to rewrite the vSphere Client into Jenkins.

      1. Unknown User (josesa)

        Actually the plugin as it is it almost serves my purpose for direct deployement (linked clone to same datastore). I'm already performing dynamic IP management by using the "IP Pools" defined on Datacenter level and associated with respective network. Setting the VM Template with vAPP options I'm able to add OVF dynamic properties to allocate one of those IPs when the VM starts and it is released back to pool when VM shuts down.

        From within the VM I'm able to access the XML information provided by the OVF environment to setup most of the VM (ip, netmask, gateway, dns), by running custom script that apply host specific commands, but unfortunately not the Hostname same as VM Name because there isn't any field that provides it :(

        Using a single guest customization profile (with dhcp) I was able to easily use the vsphere way of deployment with minimum input (just new vmname) in order to have hostname inside vm correctly aligned with VM name and the rest is automatically set with internal scripts inside vm when it first boots.

        From there to a pool of slaves dynamically with basically have the same name prefix and some order number to differentiate, is a quick step.

        Still this is just a quick overview of my current setup and what I think is missing. I'll open jira request with the specific details for each case.

    2. Unknown User (pjdarton)

      This was added in 2.8

  24. Unknown User (dev009)

    i am facing this problem with vsphere cloud plugin , where i have configured my slaves (vsphere images) to be turned on on demand, the problem is when i submit the job it goes in waiting queue , after a while Jenkins turn on the corresponding slave , now while the process of turning on the slave is in progress a poll operation is performed by jenkins and restarts the slave , so this operation keeps on going endlessly.

    log-Jan 09, 2014 7:30:45 PM INFO hudson.slaves.RetentionStrategy$Demand check
    Launching computer Vistax64_sp1 as it has been in demand for 10 min

    Jan 09, 2014 7:30:45 PM INFO hudson.slaves.RetentionStrategy$Demand check
    Launching computer Vistax64_sp2 as it has been in demand for 10 min

    Jan 09, 2014 7:30:45 PM INFO org.jenkinsci.plugins.vSphereCloud InternalLog
    Vistax86_sp2 Starting Virtual Machine...

    Jan 09, 2014 7:30:45 PM INFO org.jenkinsci.plugins.vSphereCloud InternalLog
    Vistax64_sp1 Starting Virtual Machine...

    http://javadoc.jenkins-ci.org/hudson/slaves/CloudSlaveRetentionStrategy.html

    Retention strategy polls after every two minutes and resets the slave , is there any way to increase this polling period ?

    1. Unknown User (jswager)

      What version of the vSphere plugin and Jenkins are you using?  Submitting a JIRA defect would probably be best, rather than trying to troubleshoot via the comments part of a wiki page.

      1. Unknown User (dev009)

        Jenkins Version - 1.544

        VSphere Plugin - 1.1.2

  25. Unknown User (araza82)

    Hello,

    I am trying to figure out how to use 'deploy VM from template'. The documentation and the error on my jenkins say that the template must have at least one snapshot before it can be cloned. I am not aware of how to take a snapshot of a template in vsphere. Can someone explain how this is done?

    Thanks

    1. Unknown User (jswager)

      Convert the Template to a VM.  Take a snapshot.  Convert back to a Template.  

      Although, I thought this had been fixed.  If "Linked Clone" was disabled, a template shouldn't be required.

      1. Unknown User (araza_splunk)

        Hi jswager,

        Would it be possible to fix this bug?

        1. Unknown User (jswager)

          Can you reproduce the problem even with "Linked Clone" unchecked?  And using the latest vSphere Plugin version?  If so, please submit a JIRA incident; that's the best way to get a defect fixed.

          1. Unknown User (araza_splunk)

            Hi,
            I was able to resolve this issue. Similar to an issue already in the JIRA list.

  26. Unknown User (dev009)

    is vsphere 5.5 supported ??

    1. Unknown User (jswager)

      Yes.  Actively in use on production vSphere 5.5 systems.

  27. Unknown User (dev009)

    i have recently started using Jenkins and have this huge dependency on this plugin .

    i want some clarification on slave configuration part.

    in my environment , i have configured a multi configuration job which run on 30 platforms.

    how i want it to work is , once job execution starts , slave should be reverted and turned on (achieved). 

    Now the issue i am seeing is , i am getting this socket closed error and seeing logs jenkins brings the machine up and stops , this exercise happens many times before the slave actually comes online ... i think there is something wrong with the configuration. 

    i am using this configuration -

    1. Unknown User (jswager1)

      First - I suggest submitting a JIRA incident.  These comment pages are not good for a back and forth conversation.  I would recommend carrying on further conversation there.

      Based on your settings, I would suggest raising the Idle Delay to a larger value - try 60.  See if this makes a difference.  Also, if you start the slave VM yourself, does it correctly connect without intervention?

      1. Unknown User (dev009)

        i was reluctant in filing a report , because i was not sure if it's an issue or some config problem !!

        if you could explain Force VM launch option and it's expected usage !!

        what is difference between slave being disconnect and offline from Jenkins Perspective ?

        what will happen if i keep what to do when the slave is disconnected to do nothing ?

        1. Unknown User (jswager1)

          Doesn't need to be a bug to file a JIRA incident.  You can use it for feature requests or information gathering too.  It just works a lot better than this comment system.

          First - use the help buttons.  The vSphere Plugin has help for individual options.  And those should answer most of your questions.

          Jenkins only knows if a slave is online, offline, or temporarily offline (really online, but not accepting jobs).  Online means the slave can take jobs, offline means no jobs.  A normal Jenkins slave (dumb one) can take the slave online/offline for some types, such as SSH, Windows Service.  But those don't power the machine on or off.  The vSphere Plugin extends the slave idea allowing power on/off and revert.  When Jenkins wants the slave to be online and the VM is off, the vSphere plugin can power the VM on, revert.  When Jenkins takes the slave offline - the vSphere plugin extends that by optionally allowing the VM to be powered down.

          To have the VM do nothing when the slave agent is disconnected, set "What to do when the slave is disconnected" to "Nothing".

          1. Unknown User (dev009)

            in my scenario my machines aka slaves remain shutdown until unless there is a job for them and i don't want to start them manually, what could be the ideal config for it ?

            1. Unknown User (jswager1)

              Just like you have - force start using a schedule with an in-demand delay.

  28. Unknown User (hackndoes)

    Hi, Will you add ability to run batch like operations?

    I need to be able to delete all snapshots that are matched by a regular expression, say

    Test_VM_<some expression for a date> this is very useful when you create snapshots in an automatic way and name it 

    with a pattern, later you can delete all snapshots of the same pattern, or for a specific date, it is unlimited power to the user.

    1. Unknown User (jswager)

      No, there were no plans to implement this feature.  The ability to delete a single snapshot by name seems to have been adequate for the majority of users.

      That said, please feel free to add your feature to the code and submit a pull request with the modified code.

  29. Unknown User (deligeli)

    Is it possible to parameterize the cloud config in System configurations?

    I want to be able in Job to define the parameters for the servers that I need to take a snapshot or revert

    1. Unknown User (jswager)

      The cloud parameter cannot be parameterized, but the VM name can.  Does this issue describe what you are looking for? https://issues.jenkins-ci.org/browse/JENKINS-25291

      1. Unknown User (deligeli)

        I need to use a property file that contains the server access details (credentials) instead of manually inserting them

        Would the above solve it?

        1. Unknown User (jswager)

          I would recommend submitting a JIRA incident.  Offhand, I would say that it's going to be a bit difficult to allow any individual build step to override the vSphere Cloud credentials provided in the Jenkins config.

          1. Unknown User (deligeli)

            Thanks

            i opened one here:

            https://issues.jenkins-ci.org/browse/JENKINS-25380

            It will really be wonderful to have it!

  30. Unknown User (deligeli)

    I am trying to use Jenkins String parameter for vm name.

    In the job, I added a String parameter: vm-name. 

    In the build step, I use it in the name like this:

    $

    Unknown macro: {vm-name}

    I am running on windows. But the build fails. If I try it without a parameter it works fine. I tried also:

    $vm-name and %vm-name% but no success.

    Is it parameterized? 
    Can the VM parameter be parameterized?

    1. Unknown User (jswager)

      The VM name can be a Build Parameter - either one specified as a parameter or one injected through another plugin.  Typically, I use the notation $PARAMETER to specify it.  In your example, $VM-NAME.  

      1. Unknown User (deligeli)

        I tried it, but it doesn't work. Is there a way to debug why it doesn't load it?

        Since I am using windows, I also tried %VM-NAME% so it looks like this:

        1. Unknown User (jswager)

          Using $PARAMETERNAME should work; using %PARAMETERNAME% will not work; the plugin isn't looking for build variables in that way.  I would suggest using a variable name with a "" in it.  Try the simplest case first - just alpha characters.  I've had problems with "" in the past when you try to reference the parameter from any plugin, although "_" seems to work just fine for me.

          Like other Jenkins plugins, a separate log isn't maintained on a per-plugin basis.  All plugins write to the same Jenkins log.  However, from Jenkins logging management screen, you can create a new logger that will capture the "vsphere-cloud" messages.  Viewing that logger after reproducing the problem may give some clue as to what's going on.

          I would recommend opening a JIRA incident.  Carrying on a troubleshooting conversation in the comments of this page is rather inefficient.

  31. Unknown User (deligeli)

    I am trying to use Jenkins String parameter for vm name.

    In the job, I added a String parameter: vm-name. 

    In the build step, I use it in the name like this:

    ${vm-name}

    I am running on windows. But the build fails. If I try it without a parameter it works fine. I tried also:

    $vm-name , %vm-name%

    but no success.
    Is it parameterized? 
    Can the VM parameter be parameterized?

  32. Unknown User (lilaches)

    Hi,

    Is it possible to pass a list of VM's to each action? for example pass a list of 4 VM's that should be turned off?

    1. Unknown User (jswager)

      Not at this time, it's not possible.  However, you can submit a JIRA feature request - or better yet, a pull request with the code modifications.  

  33. Unknown User (nick_a)

    Hi,

    I want to revert to a snapshot and start up the VM.
    When it has finished reverting successfuly; the job fails with this error:
    Connection was broken
    java.io.IOException: Connection aborted: [org.jenkinsci.remoting.nio.NioChannelHub$MonoNioTransport@940df8[name=TESTVM025|mailto:org.jenkinsci.remoting.nio.NioChannelHub$MonoNioTransport@940df8name=TESTVM025]
     at org.jenkinsci.remoting.nio.NioChannelHub$NioTransport.abort(NioChannelHub.java:211)
     at org.jenkinsci.remoting.nio.NioChannelHub.run(NioChannelHub.java:631)
     at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
     at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
     at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
     at java.util.concurrent.FutureTask.run(Unknown Source)
     at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
     at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
     at java.lang.Thread.run(Unknown Source)
    Caused by: java.io.IOException: An established connection was aborted by the software in your host machine
     at sun.nio.ch.SocketDispatcher.read0(Native Method)
     at sun.nio.ch.SocketDispatcher.read(Unknown Source)
     at sun.nio.ch.IOUtil.readIntoNativeBuffer(Unknown Source)
     at sun.nio.ch.IOUtil.read(Unknown Source)
     at sun.nio.ch.SocketChannelImpl.read(Unknown Source)
     at org.jenkinsci.remoting.nio.FifoBuffer$Pointer.receive(FifoBuffer.java:136)
     at org.jenkinsci.remoting.nio.FifoBuffer.receive(FifoBuffer.java:306)
     at org.jenkinsci.remoting.nio.NioChannelHub.run(NioChannelHub.java:564)

    Any ideas on what the problem could be?
    I use $NODE_NAME as VM Name so that it can run on all machines with a certain label.

    Tnx

  34. Unknown User (boris_bondarenko)

    Hello guys,

    At first I would like to thank you for this great plugin. We notice an interesting behavior during our testing with vSphere Cloud plugin (used for automatically

    powering on / off when in demand nodes).

    (we created pools of nodes, each pool with certain labels to match the criteria for different builds, so number of nodes have some set of labels and other number of nodes

    have different set of labels). In the time of need, we needed to disable the connection to vSphere Cloud and do some maintenance on the node and leave it running without the intervention from vSphere Cloud plugin. 

    We switched the "vSphere instance" in the node configuration from our instance to default "Select a vSphere Cloud instance..." hoping that this node will be skipped 

    during the picking of the nodes based on labels.

    After we started a build, which is running on a node satisfying the label expression, the build was hanging sometimes for 40 minutes or longer. After peeking through the log

    we found out that the vSphere Cloud Plugin have been trying to launch the node (although there have number of nodes with the same labels).

    So in short: when the vSphere Cloud instance is not selected in the node configuration and the labeling mechanism is used to run a build on some node matching the label expression,

    vSphere Cloud plugin can take the node to which it cannot connect an will keep trying over again, instead of just picking a different node matching the label.

    Is this something which is a part of vSphere plugin or some core Jenkins functionality?

    Message:

    Launching computer XYZ as it has been in demand for 49 minAug 12, 2015 11:39:46 AM INFO org.jenkinsci.plugins.vSphereCloud InternalLog
    Could not find our vSphere Cloud instance!

    1. Unknown User (jswager)

      I would recommend opening a JIRA issue against vSphere Plugin.  The back-and-forth of troubleshooting and questions is better handled by JIRA than comments on the wiki page.

      1. Unknown User (jazze)

        But be warned that Daniel Beck will probably scold you for opening that JIRA issue. :-)

  35. Unknown User (wellu)

    We got vSphere 6.0 and jenkings 1.6.3.9

    vSphere plugin is version 2.7

    Adding new cloud and testing connection gives me "Connection succesfull", but setting up a job to provision a new virtual machine just gives me 

    javax.servlet.ServletException: java.lang.RuntimeException: Could not find vSphere Cloud instance "null"!
    at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:796)
    at org.kohsuke.stapler.Stapler.invoke(Stapler.java:876)
    at org.kohsuke.stapler.MetaClass$6.doDispatch(MetaClass.java:249)
    at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:53)
    at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:746)
    at org.kohsuke.stapler.Stapler.invoke(Stapler.java:876)
    at org.kohsuke.stapler.MetaClass$6.doDispatch(MetaClass.java:249)
    at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:53)
    at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:746)

    I have tried just the cloud name that the drop down provides me "vcenter" and using a string parameter set to VSPHERE_CLOUD_NAME == vcenter. How can the connection test work, but no cloud name is passed to the job?

    Also the plugin still requires me to have a snapshot present from the template. No matter if linked clone is checked or not. This is a bug.

    1. Unknown User (jswager)

      Please submit a JIRA issue for this.  

  36. Unknown User (mtaylor)

    Can you please add the ability to delete a Template configuration. Currently there is only the option to delete the entire Cloud configuration and only an add button is at the bottom.

    1. Unknown User (pjdarton)

      Done. See JENKINS-38030 - Getting issue details... STATUS

  37. Unknown User (mtaylor)

    Another issue I am seeing is the inability of the plugin to find the snapshot for a template

  38. Unknown User (mtaylor)

    No support for Windows VM's as templates? Did I completely miss this? The only option for connection is SSH please allow the secondary launch method to be a part of this.

    1. Unknown User (thewildscot)

      Any word on this?  We just set this up with no indication that this would only work for Unix machines and, yeh it always defaults to connecting through SSH.  Can we set up software on the machine image for SSH connections and get this to work on a windows machine?

      1. Unknown User (mtaylor)

        No sir. Sorry I'm just getting to this reply. I haven't heard a peep on if or when we will see this ability added. I wish I knew another way or could fix it in a fork/branch myself but I don't know how. We will just have to wait and see. If you figure out an alternative let me know.

        I just do it the old fashioned way where I have a base VM and I manually create templates once there is a reason or need to update the base VM I destroy the temp templates and recreate new ones.  The bummer about this plugin is that it get's all the way through and chokes on the jnlp connection its a shame really.

        thanks and good luck!

    2. Unknown User (malefikh)

      There is a way to connect via SSH to Windows through the plugin, so it can install the slave.jar and make the connection available

      You need to install Cygwin and sshd like it's explained in this link:

      https://wiki.jenkins-ci.org/display/JENKINS/SSH+slaves+and+Cygwin

    3. Unknown User (pjdarton)

      There wasn't ... but support was added in 2.14.  Check out the help for the "GuestInfo Properties" field in the template configuration.

  39. Unknown User (mtaylor)

    Another issue is that when I choose the option of when to utilize the node and I specify to only use the node with the label indicated. Not the Utilize this node as much as possible.  The option to specify the node does not stick after saving.

  40. Unknown User (odklizec)

    Hi guys,

    What's your experience with vSphere plugin v 2.13? I'm experiencing an error (ERROR: no workspace for 'name-of-job') while attempting to restart a VM. I've posted detailed description of the problem >here<. The same steps works with v 2.12.

  41. Unknown User (xtaylor21x)

    Mr. Swagger!

    I cannot thank you enough for the work time and effort you have put into this last update! 2.14 HUGE! Kudos to you man and if I can donate in any way please let me know!

  42. Unknown User (elliottjones)

    See https://issues.jenkins-ci.org/browse/JENKINS-42538 issue with flow jobs and label parameters and latest vsphere plugin.

  43. Unknown User (svraalte)

    Is there any way to read the MAC address after cloning a vm from template? I'd like to use for os installation via pxe afterwards

    1. Unknown User (pjdarton)

      This would require enhancements to the plugin code.  If you create an enhancement request in JIRA, implement it and issue a pull request, it can be included in the next release.  There's a bit of a learning curve to enhancing plugins, but there is existing code that exposes the IP address so obtaining and exposing the MAC should fit.

      ...or run the job that's doing the cloning on a slave that's got access to the vSphere server and run command-line commands to interrogate it.

      ...or don't tick the power-on option on the "Clone" step, use the "Reconfigure" step to set the MAC to one of your own choosing, and then switch it on.

      1. Unknown User (svraalte)

        Unfortunally I'm not a developer, so I opened an issue an will try to figure out how to enhance the code, but I'm not sure if I will succeed

        We will also try to run an script that reads the mac.

        Setting a manuel MAC won't work in our enviroment.

        Thank you for your help

        1. Unknown User (puma34)

          Confirm. Implement this addressType 

  44. Unknown User (lil2006)

    GuestInfo properties (mentioned above) allow you to pass information from Jenkins to the newly started VMs.  If you are using the SSH launch method then you may not need this, but if you are using Java Web Start (JNLP) then this will be needed to tell your slave process where the Jenkins server is, what slave it is, the JNLP "secret" etc.  For example, to pass the data necessary for the newly-started VM to start a JNLP slave process and connect back to Jenkins, you could set the following GuestInfo properties: .... 
    Please elaborate these steps, how to make start of jnlp. ? 

  45. Unknown User (pjdarton)

    How your VM starts the slave.jar process (that will connect back to Jenkins via JNLP) is up to you.

    e.g. When we're using Windows, we configure Windows to auto-login to a specific (test) user and have a scheduled task that triggers "on login" which runs a batch file which interrogates vSphere (via the vSphere client tools "vmtoolsd" that we installed in the VM) to find out the value of the variables we've passed in.  This way, when the plugin starts a clone of our VM template (set up as described earlier), the VM boots up, then logs in, then starts the batch file which reads the GuestInfo parameters and then starts the slave.jar using the necessary command-line arguments for it to successfully connect to Jenkins as a JNLP slave.

    i.e. It's just like you would do on a non-dynamic slave that's connecting via JNLP, except that there's some data which you can't hard-code to constant values when you're dealing with a dynamically-created VM and that's where the GuestInfo functionality is useful.

    If you require more detailed information, I suggest you read up on how to make a static (non-dynamic) VM connect via JNLP - you'll then see that there's some information that cannot be "baked in" to the template (it has to be decided after the VM has been cloned) and also the vSphere documentation on how to use "vmtoolsd" (which is how we pass that information from Jenkins to the guest OS in the newly-cloned VM)..

    1. Unknown User (lil2006)

      Peter, thank you. 
      I'll make investigation based  on your comments. 

    2. Unknown User (lil2006)

      Hi. Peter

      Still have question about ${Node_name} how it's corresponds to node name in Jenkins. And please could you show example of  bat command how use vmtoools

      Thanks

      1. Unknown User (opozot)

        I second that.

        I'm new to this and I'm not sure how to configure Guestinfo.

        Could anyone paste a GuestInfo config for dynamic provisioning?

        Or if I could get answers to the below:

        • when I set 'Param Name: SLAVE_URL | Param Value: ${JENSKINS_URL}computer/${NODE_NAME}/slave-agent.jnlp'
          where do I set the ${JENKINS_URL}? Where do I set the ${NODE_NAME}? In the GuestInfo? If yes, how? 'ParamName: NODE_NAME | ParamValue: hostname'?

        Any pointers would be greatly appreciated!

        1. Unknown User (pjdarton)

          The screenshots above include an example GuestInfo config.  The built-in help for the GuestInfo fields explains where JENKINS_URL, NODE_NAME etc come from.

          1. Unknown User (opozot)

            Thanks for the answer.  I've configured the guestinfo, but it just seems I'm not able to pass the parameters to the newly created VM,

            i.e. e.g. vmtoolsd --cmd "info-get guestinfo.SLAVE_HOME" returns no value found.

            So my slave isn't able to get the needed info. I'll have to troubleshoot that.

            1. Unknown User (pjdarton)

              To troubleshoot, I found it was easiest to manually create a VM, and manually set the guestinfo properties (using the vSphere GUI).

              You can then run vmtoolsd commands on the VM and edit the guestinfo properties, back and forth, until you've found the problem.

              FYI GuestInfo properties are set/stored in the vSphere hypervisor; they are "live" in that you can change them (via the vSphere UI) while the VM is running and they should be immediately readable by the vmtoolsd command on the guest OS.  I did find that only parameters whose names started "guestinfo." would be passed though though.

              Note: There's also a Windows .CMD script available as a working example - search above for "JenkinsSlaveStartup.cmd".  Even if you're not using Windows, the script may be useful as a starting point.

  46. Unknown User (puma34)

    How run two jobs with same label with different vm? 

    1. Unknown User (pjdarton)

      Create a VM, turn it into a template.  Configure Jenkins with a Cloud pointing at your vSphere server.  Create a template entry for that cloud describing how to create a clone of your VM.  Tell Jenkins it's got just 1 executor on that VM so that the VM can only run one job at a time.

      Now tell Jenkins to run both jobs - it'll start two VMs and run each job on different VMs.

      1. Unknown User (puma34)

  47. Unknown User (puma34)

    what is Instance Cap ?

  48. Unknown User (chstclai)

    Does anyone know how to make the snapshot quiesce guest file system?

    1. Unknown User (pjdarton)

      Where I work, we tell the guest OS to do a controlled shutdown and then do the snapshot.  It might be overkill but it works for us.

  49. Unknown User (odklizec)

    Hi folks,

    I'm recently configuring a new vSphere-based VM as a Jenkins slave. Jenkins agent is running, Jenkins master finds slave and even extracts system info from it (OS, mem, HDD, etc...). What bothers me, though, is that after clicking "Test VM connection" button, it returns "Virtual Machine was not found"? I have another slave VM located on the same vSphere server and this one connects fine. So my guess is, that there is still something blocked between Master/Slave/vSphere, but I have not clue what (FW, port, etc)? Any idea what to check and eventually unlock? What exactly the vSphere plugin is doing, after clicking "Test VM connection" button? Thank you in advance for your help.

    1. Unknown User (gmaxera)

      I don't have any "Test VM connection" in my vSphere plugin. The most similar button is "Test connection" and from what I understood, it only checks if it can connects to vSphere API and does anything else.

  50. Unknown User (gmaxera)

    Hi all,

    is it possible to launch an agent into a per-job generated VM ?? 

    More in details, I can create a new VM using the "vSphere buildStep: [ $class: 'Clone', ... ] " ... but that VM doesn't appear as a Jenkins new node.

    I'm wondering if there is a way to add this new VM as a node into Jenkins and run stages on it.

    Thanks.

    Gianluca.


    1. Unknown User (gmaxera)

    2. Unknown User (opozot)

      I'm using Jenkins Swarm plugin for this. When you create a new VM via job you have two options. Use the customization specification in vSphere or a script. I'm using a script since I'm deploying from a template. The template has the swarm-client.jar and a vbs script to launch it in the background during the first login. The path to the script is set in the registry in RunOnce key. So basically when the VM spins up the RunOnce key fires up the swarm-client and it connects to the master of your choice.