Skip to end of metadata
Go to start of metadata

Plugin Information

View Clone Workspace SCM on the plugin site for more information.

This plugin makes it possible to archive the workspace from builds of one project and reuse them as the SCM source for another project.


Usage

Publisher
  • In the configuration for a project whose workspace you want to clone and re-use in other projects, select "Archive for Clone Workspace SCM" in the list of publishers.
  • If desired, specify the files to include in the archive - by default, this will be "*/". Use Ant-style globs.
  • Specify the criteria a build needs to meet in order to be archived.
  • Run a build. If it meets the criteria, its workspace will be archived, until a new build meeting the criteria has run, at which point the old archive will be deleted.
SCM
  • In the configuration for a project which you wish to have re-use another project's workspace, select "Clone Workspace" from the list of possible SCMs.
  • Choose the parent project whose workspace you wish to re-use from the drop-down list - if no projects have the Clone Workspace Publisher enabled, the drop-down will be empty.
  • Choose the parent build criteria you wish to use.
  • Run a build - assuming the parent project has an archived workspace meeting the criteria in question, it'll be expanded and used as the workspace for this build.
  • Additionally, the changelog from the parent project build that archived workspace came from will be re-used as the changelog for this build.

Changelog

Version 0.6 (Sept 6. 2013)
  • Add support for Matrix jobs (JENKINS-6890)
  • Add null check in CloneWorkspaceSCM.createChangeLogParser() to prevent NPE. (JENKINS-13160)
  • Fix Switching archive type of cloned workspace leads to Exception in further jobs (JENKINS-15038)
  • Use full name for jobs
  • Don't only consider jenkins root jobs as eligible parents
Version 0.5 (Aug 17, 2012)
  • Add option to turn off default Ant excludes from workspace archive. (JENKINS-13888)
Version 0.4 (Feb 3, 2012)
Version 0.3 (March 4, 2011)
  • Some minor fixes
Version 0.2 (also March 14, 2010)
  • Clarifying display name text for CloneWorkspacePublisher
Version 0.1 (March 14, 2010)
  • Initial release.

15 Comments

  1. Unknown User (steveharman@gmail.com)

    Has this plugin been tested with Master/Slave setups? I ask because my jobs seem to work as long as both the upstream (publishing) and downstream (SCM) job run on my master... but the downstream jobs seem to hang while unpacking the archived files:

    Started by upstream project "MyProject-Fast" build number 16
    Building remotely on ci-slave-01
    FATAL: hudson.remoting.RequestAbortedException: java.net.SocketException: Connection reset
    hudson.remoting.RequestAbortedException: hudson.remoting.RequestAbortedException: java.net.SocketException: Connection reset
    at hudson.remoting.Request.call(Request.java:137)
    at hudson.remoting.Channel.call(Channel.java:557)
    at hudson.FilePath.act(FilePath.java:742)
    at hudson.FilePath.act(FilePath.java:735)
    at hudson.FilePath.unzip(FilePath.java:415)
    at hudson.FileSystemProvisioner$Default$WorkspaceSnapshotImpl.restoreTo(FileSystemProvisioner.java:227)
    at hudson.plugins.cloneworkspace.CloneWorkspaceSCM$Snapshot.restoreTo(CloneWorkspaceSCM.java:344)
    at hudson.plugins.cloneworkspace.CloneWorkspaceSCM.checkout(CloneWorkspaceSCM.java:126)
    at hudson.model.AbstractProject.checkout(AbstractProject.java:1054)
    at hudson.model.AbstractBuild$AbstractRunner.checkout(AbstractBuild.java:479)
    at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:411)
    at hudson.model.Run.run(Run.java:1248)
    at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
    at hudson.model.ResourceController.execute(ResourceController.java:88)
    at hudson.model.Executor.run(Executor.java:129)
    Caused by: hudson.remoting.RequestAbortedException: java.net.SocketException: Connection reset
    at hudson.remoting.Request.abort(Request.java:257)
    at hudson.remoting.Channel.terminate(Channel.java:608)
    at hudson.remoting.Channel$ReaderThread.run(Channel.java:899)
    Caused by: java.net.SocketException: Connection reset
    at java.net.SocketInputStream.read(Unknown Source)
    at java.io.BufferedInputStream.fill(Unknown Source)
    at java.io.BufferedInputStream.read(Unknown Source)
    at java.io.ObjectInputStream$PeekInputStream.peek(Unknown Source)
    at java.io.ObjectInputStream$BlockDataInputStream.peek(Unknown Source)
    at java.io.ObjectInputStream$BlockDataInputStream.peekByte(Unknown Source)
    at java.io.ObjectInputStream.readObject0(Unknown Source)
    at java.io.ObjectInputStream.readObject(Unknown Source)
    at hudson.remoting.Channel$ReaderThread.run(Channel.java:875)

    Any ideas?

  2. Unknown User (steveharman@gmail.com)

    UPDATE

    I have done some more digging and am starting to think it may have something to do with the size of the files being archived in the workspace - perhaps with how the files are streamed/buffered when being unzipped on the slave? See my additional comments at on the related Issue: http://issues.jenkins-ci.org/browse/JENKINS-6817

  3. Unknown User (capitaine.banane@gmail.com)

    Hello,

    It seems that this plugin fails when the workspace is too big.

    I have tested it with a small workspace and it works fine, but in real life my workspace contains more than 20 000 files and I have this error:

    Started by upstream project "split-part01-checkout" build number 8
    FATAL: null
    java.lang.IllegalArgumentException
    at java.util.zip.ZipInputStream.getUTF8String(Unknown Source)
    at java.util.zip.ZipInputStream.readLOC(Unknown Source)
    at java.util.zip.ZipInputStream.getNextEntry(Unknown Source)
    at hudson.FilePath.unzip(FilePath.java:472)
    at hudson.FilePath.access$100(FilePath.java:162)
    at hudson.FilePath$2.invoke(FilePath.java:420)
    at hudson.FilePath$2.invoke(FilePath.java:418)
    at hudson.FilePath.act(FilePath.java:756)
    at hudson.FilePath.act(FilePath.java:738)
    at hudson.FilePath.unzip(FilePath.java:418)
    at hudson.FileSystemProvisioner$Default$WorkspaceSnapshotImpl.restoreTo(FileSystemProvisioner.java:227)
    at hudson.plugins.cloneworkspace.CloneWorkspaceSCM$Snapshot.restoreTo(CloneWorkspaceSCM.java:344)
    at hudson.plugins.cloneworkspace.CloneWorkspaceSCM.checkout(CloneWorkspaceSCM.java:126)
    at hudson.model.AbstractProject.checkout(AbstractProject.java:1044)
    at hudson.model.AbstractBuild$AbstractRunner.checkout(AbstractBuild.java:479)
    at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:411)
    at hudson.model.Run.run(Run.java:1241)
    at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:306)
    at hudson.model.ResourceController.execute(ResourceController.java:88)
    at hudson.model.Executor.run(Executor.java:122)

    1. I also got the IllegalArgumentException on a big workspace. In my case, I narrowed down the cause to this Java bug: http://bugs.sun.com/view_bug.do?bug_id=4820807

      While this bug seems to be fixed in Java 7, it still would be nice to have a workaround. Anyone else experienced this problem and found a solution (besides renaming any workspace files, as that is not an option for me). A clean solution might be to offer other compression options in the configuration of the clone step.

  4. After upgrading from 0.3 to 0.4 to get the "exclude" capability, a NullPointerException happens in this plugin at the end of our job:

    https://issues.jenkins-ci.org/browse/JENKINS-12835

  5. The 0.4 plugin is working well for me, keep up the great work!

    I do have a question, however... is there a way to tell the plugin to use a known previous build number instead of simply the "latest successful", or "latest stable", etc. builds?  In my situation, I may need to use an arbitrary previous successful build number; I'm currently using a parameter and fetching from the parent's specific build number's workspace, unpacking, etc. as manual pre-build steps... it would be nice if I could use the plugin and just specify a previous build number from the drop-down.

    1. Are you able to configure jobs? I wasn't able to after upgrading to cloneworkspace to 0.4. The page would stop half way and not finish loading. It usually died around "Advanced Project Options" or right before it.  When I downgraded to 0.3 and then restarted, I could then configure jobs again.

      1. Yes, I am able to configure jobs... but, I started fresh on a new server with the 0.4 version of the plugin.  I haven't tried upgrading from a previous release; could be some differences in configuration between 0.3 and 0.4?

  6. Perhaps we're not doing things the way Jenkins intended, but this is our build pipeline:

    1) Compile - checks files out of Perforce, cleans, configures for the target architecture and compiles.  Archives the workspace for susequent steps in the pipeline.

    2) Zip - builds the .ear file, verifies it, builds a complete shippable .zip of all project deliverables.

    3) Quick test - runs a relatively short test suite. 

    2a) Zip is "promoted" if all tests in QuickTest pass.  This delivers the Zip to the System test team (outside of Jenkins).

    Now we didn't want each step to sync the files down from Perforce and compile afresh, the clone-workspace plugin seemed ideal for us. Unfortunately there are a number of problems with it including:

    a) Only the latest workspace from step(1) is maintained.  So if another change triggers a Compile whilst Zip is in progress, it could be that (3) Quick test actually tests that subsequent build and not the one it is supposed to be testing.  This has regularly caused us considerable confusion as the jobs were out-of-step.

    b) It takes so long ... our project is somewhat monolithic, including all binaries required within the workspace.  The workspace.zip archive is about 1.5Gb and creating it takes around 50 minutes using clone-workspace-plugin vs. only 10 minutes for the actual compile, that's unacceptable.  Un-zipping at the start of the steps (2) and (3) is not quite so bad but still 10-20 minutes.  It seems to be something to do with the way data is piped between master and slave.

    The solution we now have working is to replace this plugin by a shell script. This script takes 2 minutes to archive the workspace at the end of job (1) and 2 minutes at to un-archive it at the start of the other steps. I'm attaching it here in case it is useful to someone else, and to illustrate our use case which we would like this plugin to support.

    clone-workspace shell script

    It is simple to use, as follows:

    At the end of job (1) invoke the script using "Execute Shell":

    ${JENKINS_BIN}/clone-workspace PACK "${BUILD_TAG}"
    

    Optional INCLUDE= and EXCLUDE= filters can be used to limit the contents of the archive.  We actually create 2 archives, one is much smaller containing just the information required for the "promote" step (2a) to work.

    Also add a Post-Build action "Trigger parameterized build on other projects" to start job (2) using the parameter:

    UPSTREAM_BUILD_TAG=${BUILD_TAG}
    

    Jobs (2) and (3) are parameterized, accepting UPSTREAM_BUILD_TAG.  The first task that each of these downstream jobs runs is:

    Execute Shell

    ${JENKINS_BIN}/clone-workspace UNPACK "${UPSTREAM_BUILD_TAG}"
    

    The archive dir must exist on the master node and be referenced by the JOBS_ARCHIVE environment variable.
    The last job in the pipeline removes the redundant archive by invoking:

    ${JENKINS_BIN}/clone-workspace CLEAN "${UPSTREAM_BUILD_TAG}"
    

    We also purge all archives from it overnight using a "cron" job, just to be sure.

    Now the downstream jobs don't show the change history;  that was solved by re-instating the clone-workspace-plugin but configured to archive just one file.

  7. Hi,

    I am using this plugin to reuse changelog of an uperstream job for our findbugs job. The findbugs build job used CVS information to determine the changes. I include CVS directories (CVS/*/) explicitily but I always get CVS content ignored in the archived workspace. I wonder if that is default behaviour of the plugin or something I setup incorrect. Can someone suggest?

    My include fileset is like this: nms/*/, CVS/**/*

    Thanks.

    Gene

    1. find out workaround way -- archiving after the build with a script and push back with over written the unwanted one.

      1. Yes that is what I ended up doing with my clone-workspace shell script above.  You therefore need to use this plugin to archive only one file from the upstream workspace, that will be sufficient to preserve the visibility of the change history in the downstream job.

  8. I apologize if this is the wrong place to post this question. I am new to using jenkins. I am currently using this plugin (Clone workspace SCM 0.5) to split my job into 2 jobs. I want my second job to use the same workspace as the first job. In the configuration of job1 i have a post build step that specifies archiving the entire workspace as "Gzipped tar" . However this gives the following error:Archiving workspace
    ERROR: Publisher hudson.plugins.cloneworkspace.CloneWorkspacePublisher aborted due to exception
    java.lang.UnsupportedOperationException
    at hudson.os.PosixAPI$1.getCurrentWorkingDirectory(PosixAPI.java:59)
    at org.jruby.ext.posix.util.ExecIt.run(ExecIt.java:59)
    at org.jruby.ext.posix.util.ExecIt.runAndWait(ExecIt.java:51)
    at org.jruby.ext.posix.JavaLibCHelper.readlink(JavaLibCHelper.java:196)
    at org.jruby.ext.posix.JavaPOSIX.readlink(JavaPOSIX.java:160)
    at hudson.Util.resolveSymlink(Util.java:1067)
    at hudson.Util.resolveSymlink(Util.java:1030)
    at hudson.util.DirScanner$Glob.scan(DirScanner.java:115)
    at hudson.FilePath$1.invoke(FilePath.java:411)
    at hudson.FilePath$1.invoke(FilePath.java:407)
    at hudson.FilePath.act(FilePath.java:839)
    at hudson.FilePath.act(FilePath.java:821)
    at hudson.FilePath.archive(FilePath.java:407)
    at hudson.FilePath.tar(FilePath.java:1808)
    at hudson.plugins.cloneworkspace.CloneWorkspacePublisher.snapshot(CloneWorkspacePublisher.java:231)
    at hudson.plugins.cloneworkspace.CloneWorkspacePublisher.perform(CloneWorkspacePublisher.java:178)
    at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)
    at hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:710)
    at hudson.model.AbstractBuild$AbstractRunner.performAllBuildSteps(AbstractBuild.java:685)
    at hudson.model.Build$RunnerImpl.post2(Build.java:162)
    at hudson.model.AbstractBuild$AbstractRunner.post(AbstractBuild.java:632)
    at hudson.model.Run.run(Run.java:1463)
    at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
    at hudson.model.ResourceController.execute(ResourceController.java:88)
    at hudson.model.Executor.run(Executor.java:239)

    When i change the Archiving method option to "Zipped" then i dont get the above error. However, in this case job2 does not recognize the cloned workspace as it is looking for "workspace.zip" and the plugin as ended up creating a "workspace.tar.gz" even when the Archive method was set to zipped.

    I have read about this in other forums and they recommend using the "Gzipped Tar" archive method due to permissions. I would therefore prefer to use that method. However, if it is easier to make the "zipped" method work...then i guess i will have to use that.
    I am using Jenkins 1.466.1

    Any help on this matter will be appreciated. Thanks in advance!
    -rahul

  9. I'm having trouble with one of the fixes for 0.4:  "Parent project can be specified as a parameter. (issue #9779)"

    How do I specify a parameter?  All I see is the dropdown list...

  10. This plugin works exactly as one hoped for. Awesome Stuff!

    I just wanted to have a view on how we can speed up the clone restoring process. Is there an rsync method we can have inorder to restore the workspace faster? An rsync would generally only update all the files that has a newer date.

    The speed is an issue when in master-slave mode. The restoring from remote servers is slow, though not a deal breaker.

    Any other options to speed things up a bit?

    Cheers!