Child pages
  • Multiple SCMs Plugin
Skip to end of metadata
Go to start of metadata

Deprecated: Users should migrate to https://wiki.jenkins-ci.org/display/JENKINS/Pipeline+Plugin .  Pipeline offers a better way of checking out of multiple SCMs, and is supported by the Jenkins core development team.

Allows a job to check out sources from multiple SCM providers.

Plugin Information

View Multiple SCMs on the plugin site for more information.

Background

This plugin is more of a proof-of-concept than a robust and fully functional component. It does work in my particular build environment, and is meant to serve as a demonstration of what might be possible with more work. It was inspired by JENKINS-7155 requesting multiple repository checkouts for Mercurial similar to what is possible with the Subversion plugin. It's currently implemented as a plugin, but if enough people find it useful, I think the idea would work better in the Jenkins core.

We are a small team and have been using Subversion which has been adequate for our needs. However, I wanted to experiment with distributed VCS systems, so I tried using Mercurial for a medium-sided update to one of our projects and enjoyed it very much. Everything went great until I got to the Jenkins build part. Our projects are small to mid-sized and are usually structured like:

/
  /project-code
  /inhouse-common-library
  /3rd-party-deps
     /lib1
     /lib2

The in-house library is built by an upstream job, but the 3rd-party dependencies are checked out as part of the build. With subversion, we could check each one out individually, and initially I thought this is what I wanted for Mercurial as well. However, quite often we have the entire source for the library checked in, but the build really only needs a single dll or jar. Since Mercurial doesn't support partial clones, it now seems better to leave these in subversion since they're rarely changed and we can check out just the parts needed during a build.

I googled for solutions, but the ones I found all seemed to have limitations compared to the working and adequate subversion solution:

  1. Script the dependency checkouts as part of the build. This would work and would be easy, but changes to third party libraries would not automatically trigger a build or be included in the change log.
  2. Have an upstream job to check out the dependencies. This could work, but it sounded tedious to set up and maintain, and would clutter the job list with lots of superfluous jobs.
  3. Mercurial forests/subrepos. The general feel I got was that the forest extension is not actively supported, and subrepos are not yet supported by the Mercurial plugin. Furthermore, this would require changes to how we structure our code. If we were a bigger shop, this would probably be worthwhile, but for me it seemed like more maintenance.
  4. Just leave everything in subversion and live with it. The easiest of all, but loses all the benefits of DVCS (however minimal for a team our size).

Usage

In the SCM section of the Jenkins job configuration screen, choose 'Multiple SCMs'. You'll then see a drop-down list of the available SCM plugins which can be configured similar to the way build steps are configured for a freestyle job. When chosen, each SCM plugin will present its normal set of configuration options.

Make sure each SCM checkout is done in a separate sub-directory of the workspace to avoid conflicts between the SCM providers. If using Mercurial, this option is hidden in the 'Advanced' section, so don't forget to configure it.

If changing the SCM provider for an existing job, I recommend wiping out the workspace.

Limitations

  • Currently tested only with Mercurial and Subversion plugins, as that is what I use locally.
  • Post-commit type triggers don't currently work (at least for subversion), so it is necessary to configure 'cron' type polling.
  • Repository browser configuration is also not supported in the current version.
  • I haven't tested any pre/post build tagging type operations, although they will probably work.
  • I also haven't tested master/slave configurations. This might work if the underlying SCM plugins support this mode of operation.

Implementation Notes

The implementation was easier than I originally expected, and learned a lot along the way. It basically serves as a proxy between Jenkins and existing SCM plugins. The job configuration panel uses a hetero-list similar to the build steps section. That way, all the configuration options are handled by the real SCM plugins. For the actual SCM functions, it pretty much just iterates over each configured SCM plugin and forwards the request. There is some messiness in dealing with changelogs, but it's not too bad.

Changelog

Version 0.6 (Apr 3, 2016)

  • Fix SCM polling (JENKINS-25253)
  • Export environment variables from each SCM, instead of using only the variables from the lasst SCM (JENKINS-30709)
  • Fix repository browser

Version 0.5 (Jul 13, 2015)

Version 0.4 (Mar 9, 2015)

Version 0.4-beta-1 (Jun 16, 2014)

  • Support for updated SCM API in Jenkins 1.568+.

Version 0.3

Changes not recorded.

Version 0.2 (Jan 19, 2012)

  • Fix changelog parsing for subversion (and possibly others). Extracted log component had an extra newline at the top of the file which made parsing fail if the document contained an <?xml...> declaration.
  • JENKINS-7155, JENKINS-12298 Allow polling to work with multiple instances of a single SCM type (Thanks to Jesse Glick).
  • Implement getKind() to possibly allow other clients (such as NetBeans IDE) to parse the change logs (Thanks to Jesse Glick).
  • Add override of getModuleRoots() to return the union of all contained SCMs module roots.
  • For SCM implementations that add SCMRevisionState actions to the build, these are now correctly recorded in the build, so subsequent polling works correctly.
  • Forward requests for build environment variables to the contained SCMs. Fixes missing MERCURIAL_REVISION and possibly others depending on SCM plugins used.

Version 0.1 (Mar 08, 2011)

  • Initial release

47 Comments

  1. Please provide more informations. Sounds awesome!

  2. Very useful! I need to build a job that checkouts from svn and git - will try to use it.

  3. Has anyone tried using it with a Git plugin? It looks like when Git is used with multiplescm it does not hold a url...

  4. Unknown User (nissancedric@netscape.net)

    This looks really good.

    It's something I've been wanting rather that using the command line to run the checkouts from different source repositories.

    I've tried it with Subversion and Harvest SCM, the problem is that the settings are saved, but when editing the job settings the saves settings do not load, the Harvest parameters are blank.

    This issues also appears when using the TFS plug-in. (you can give it a try, you do not need TFS or Harvest to replicate the issue, just the plug-in).

    That said the job still runs and I successfully checked out from both Subversion and Harvest.

    1. I see the same behavior in the v0.2 of the plugin w/Jenkins 1.519.

  5. I can confirm that this will not work with the Git plugin at this time: Multiple "Git" SCM entries can be created, but none of them will save any of their settings (inc. under the "Advanced..." button), as far as I can tell. Even having just a single SCM, set to type "Git" doesn't work.

    I am using Jenkins 1.400, "Jenkins GIT Plugin 1.1.6", and "multiple-scms 0.1"

    Not sure if this is a problem caused by the implementation of the Git plugin, maybe?

    I noticed that each block of HTML for the SVN UI includes a hidden INPUT tag that looks like this:

    <input name="kind" value="hudson.scm.SubversionSCM$DescriptorImpl" type="hidden">

    ...whereas this is the hidden INPUT tag that appears after each block of Git UI:

    <input name="kind" value="hudson.plugins.git.GitSCM$DescriptorImpl" type="hidden">

    ...and perhaps this indicates that the saved settings for Subversion are somehow kept in a format that allows for multiple entries, while the Git plugin's config is stored in a proprietary way?

    Likewise, compare the "remote repository" field from the Subversion UI...

    <input type="text" class="setting-input validated  " value="/bar" name="_.remote" checkurl="'/jenkins/job/Auto%20update%20Test%20VM/'+'descriptorByName/hudson.scm.SubversionSCM/checkRemote'+qs(this).addThis().toString()" id="svn.remote.loc">

    ...with the same from the Git UI:

    <input type="text" class="setting-input   " value="" name="git.repo.url">

    The name attribute in particular follows a completely different pattern.

  6. I've hacked the config for my Job to get it to work with GitHub SCM entries, allowing it to do builds with multiple git repos, though this doesn't solve the overall problem with Multiple SCM's git config.

    First I started with one GitHub SCM entry, created with Multiple SCM's UI. I then edited ~/.jenkins/jobs/MyJob/config.xml as follows (i.e. specifying a dummy repo here called git@github.com:foo/bar with a working directory of test_1), and restarted Jenkins:

      <scm class="org.jenkinsci.plugins.multiplescms.MultiSCM">
        <scms>
          <hudson.plugins.git.GitSCM>
            <configVersion>1</configVersion>
    <!-- *** Added stuff begins here *** -->
            <remoteRepositories>
              <org.spearce.jgit.transport.RemoteConfig>
                <string>origin</string>
                <int>5</int>
                <string>fetch</string>
                <string>+refs/heads/*:refs/remotes/origin/*</string>
                <string>receivepack</string>
                <string>git-upload-pack</string>
                <string>uploadpack</string>
                <string>git-upload-pack</string>
                <string>url</string>
                <string>git@github.com:foo/bar</string>
                <string>tagopt</string>
                <string></string>
              </org.spearce.jgit.transport.RemoteConfig>
            </remoteRepositories>
            <branches>
              <hudson.plugins.git.BranchSpec>
                <name>**</name>
              </hudson.plugins.git.BranchSpec>
            </branches>
            <mergeOptions/>
    <!-- *** Added stuff ends here *** -->
            <recursiveSubmodules>false</recursiveSubmodules>
            <doGenerateSubmoduleConfigurations>false</doGenerateSubmoduleConfigurations>
            <authorOrCommitter>false</authorOrCommitter>
            <clean>false</clean>
            <wipeOutWorkspace>false</wipeOutWorkspace>
            <pruneBranches>false</pruneBranches>
            <buildChooser class="hudson.plugins.git.util.DefaultBuildChooser"/>
            <gitTool>Default</gitTool>
    <!-- *** Changed relativeTargetDir: *** -->
            <relativeTargetDir>test_1</relativeTargetDir>
            <excludedRegions></excludedRegions>
            <excludedUsers></excludedUsers>
            <gitConfigName></gitConfigName>
            <gitConfigEmail></gitConfigEmail>
            <skipTag>false</skipTag>
          </hudson.plugins.git.GitSCM>
        </scms>
      </scm>
    

    Note that originally everything between the <configVersion> and <recursiveSubmodules> nodes was missing, and I copied the <remoteRepositories>, <branches>, and <mergeOptions> nodes from another Job that had a single Git SCM. Also, evidently the <relativeTargetDir> node wasn't being saved either, so I put this in.

    After I verified that the build/clone worked, I then copied the whole <hudson.plugins.git.GitSCM> node as required for additional Git entries. Again, after a restart of Jenkins, I was able to run the build with multiple git clones.

    While this still doesn't seem to show the config when you're editing the Job, it does appear to do the git clone when you build the Job.

    Maybe someone else who knows Jenkins/Hudson plugins better, as well as Java, can figure out why it doesn't save/load that additional config (smile)

  7. Note: CVS & Subversion

    I can confirm that this does not work with CVS and Subversion yet.

    It does show both plugins when you choose "Multiple SCMs".
    However, when the job is actually run, only the last SCM's data will be left on the workspace.

    For example, if I have two checkouts, projectA from CVS and projectB from Subversion, if I selected the CVS checkout as the first SCM,
    only projectB will be in the workspace at the end of all checkouts (since it checked out from Subversion last).

    However, I can see the plugin moving in right direction, and it will be very useful! Thank you so much for working on it!

    1. Be aware that the "workspace" directory (i.e. the working directory), at run-time, is not currently managed by "Multiple SCMs". Hence you need to make sure each respective SCM entry has its own workspace directory override... otherwise you'll see what you just described: They both check out to the same location, which will almost certainly NOT be what you want. This is explained (briefly) in the Usage section

      Specify an explicit workspace directory for each of your SCMs, and you'll hopefully be lucky enough to find that Multiple SCMs already works for you, as-is :)

  8. First of all it is exactly what i was looking for some time. Thanks for the initiation. i am experiencing small problem when i use subversion and TFS ( i needed to aggregate different scripts from different sources for deploy). it works as expected but when i open the configuration second time for modifications, it does not show the values in the TFS (Team foundation Server SCM). I see those values in the config.xml under scms tag. but it just not  able to load them i guess. Any thought on this.? if it is a simple fix could you please guide me through.?

    Thanks 

  9. FWIW, note that SCM polling does not work well when two of the same SCM type are configured for one job. For example:

    (x) Multiple SCMs
      - Git
        URL: git://host/repoA.git
        Local dir: repo-A/
      - Git
        URL: git://host/repoB.git
        Local dir: repo-B/

    When this job polls SCM for changes, it will fetch both A and B, then execute if needed. But it will only store the most recent revision from "repoB". So when it polls again, it will get 2 different sha1 hashes for repoA and repoB, and notice that repoA will be different from the previously saved hash for repoB, and therefore it will trigger a build. Even though nothing is changing in either repo, the fact that A is different from B is causing the plugin to trigger a build. For this to be effective, each repository will need to be named by the Multiple SCMs plugin, and each build's revision will have to be saved on each execution, and only trigger a build if one of the repositories differs from the previous build, matching only repository-to-repository.

    The only workaround is to ditch the plugin and use the "Execute Shell" build step to perform the 2nd+ checkouts.

    1. I may have been wrong here: there is a setting hidden away in the Advanced section of the Git SCM for a "unique name" that you must enter in order to use multiple git SCMs with this plugin. I didn't see that initially, and it isn't explained in this wiki page. Hopefully this saves someone else the confusion.

    2. I think 0.2 fixes that (JENKINS-12298). But did not test with Git specifically.

      1. See my reply above: it did work with two repositories, but it requires entering the "Unique name" field for the Git SCM configuration. Shown here:

        1. Where in Jenkins did you find this setting?  I see only the Advanced options Name and Refspec.  I am using Multiple SCM Plugin 0.4.

  10. We Are using MKS Integrity as our SCM .WIth new version 1.2 also we have SCM polling log, From my understanding It would be better To rename "changelog.xml.temp" for each repository pollings. Also I see below error from jenkins err.log.

    org.xml.sax.SAXParseException: The element type "sub-log" must be terminated by the matching end-tag "</sub-log>".

    at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.createSAXParseException(Unknown Source)

    at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.fatalError(Unknown Source)

    at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(Unknown Source)

    at com.sun.org.apache.xerces.internal.impl.XMLScanner.reportFatalError(Unknown Source)

    at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanEndElement(Unknown Source)

    at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(Unknown Source)

    at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(Unknown Source)

    at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source)

    at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(Unknown Source)

    at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(Unknown Source)

    at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(Unknown Source)

    at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(Unknown Source)

    at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(Unknown Source)

    at javax.xml.parsers.SAXParser.parse(Unknown Source)

    at javax.xml.parsers.SAXParser.parse(Unknown Source)

    at org.jenkinsci.plugins.multiplescms.MultiSCMChangeLogParser.parse(MultiSCMChangeLogParser.java:140)

    at hudson.model.AbstractBuild.calcChangeSet(AbstractBuild.java:808)

    at hudson.model.AbstractBuild.getChangeSet(AbstractBuild.java:782)

    at hudson.model.AbstractBuild.getCulprits(AbstractBuild.java:315)

    at hudson.model.AbstractBuild$AbstractRunner.post(AbstractBuild.java:632)

    at hudson.model.Run.run(Run.java:1429)

    at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)

    at hudson.model.ResourceController.execute(ResourceController.java:88)

    at hudson.model.Executor.run(Executor.java:238)

    org.xml.sax.SAXParseException: The element type "sub-log" must be terminated by the matching end-tag "</sub-log>". at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.createSAXParseException(Unknown Source) at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.fatalError(Unknown Source) at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(Unknown Source) at com.sun.org.apache.xerces.internal.impl.XMLScanner.reportFatalError(Unknown Source) at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanEndElement(Unknown Source) at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(Unknown Source) at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(Unknown Source) at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source) at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(Unknown Source) at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(Unknown Source) at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(Unknown Source) at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(Unknown Source) at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(Unknown Source) at javax.xml.parsers.SAXParser.parse(Unknown Source) at javax.xml.parsers.SAXParser.parse(Unknown Source) at org.jenkinsci.plugins.multiplescms.MultiSCMChangeLogParser.parse(MultiSCMChangeLogParser.java:140) at hudson.model.AbstractBuild.calcChangeSet(AbstractBuild.java:808) at hudson.model.AbstractBuild.getChangeSet(AbstractBuild.java:782) at hudson.model.AbstractBuild.getCulprits(AbstractBuild.java:315) at hudson.model.AbstractBuild$AbstractRunner.post(AbstractBuild.java:632) at hudson.model.Run.run(Run.java:1429) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:238)

  11. I mean There should be Two Change log .xmls. Below log shows both Change logs were saved to location "E:\jenkins\jobs\Remittance\builds\2012-01-24_13-49-33" with same Name as changelog.xml.temp which causing SCM polling Issues.

    Started by user XXXXXX
    Building on master in workspace E:\jenkins\workspace\Remittance
    Change Log: http://ci-dashboard:8080/job/Remittance/20/changes
    Build Log: http://ci-dashboard:8080/job/Remittance/20/console
    Preparing to execute si projectinfo for #p=c:/Program Files/MKS/Software Development/FX360FileProcessor/fx360filepro.pj#RemittancePollingService
    Preparing to execute si viewproject for #p=c:/Program Files/MKS/Software Development/FX360FileProcessor/fx360filepro.pj#RemittancePollingService
    Checkout directory is E:\jenkins\workspace\Remittance\RemittancePollingService
    Successfully updated workspace with 36 updates and cleaned up 568 files!
    Saving current Integrity Project configuration...
    API Response for si viewproject successfully saved to file!
    Successfully saved current Integrity Project configuration to E:\jenkins\jobs\Remittance\builds\2012-01-24_13-49-33\viewproject.dat
    Writing build change log...
    Change log successfully generated: E:\jenkins\jobs\Remittance\builds\2012-01-24_13-49-33\changelog.xml.temp
    Change Log: http://ci-dashboard:8080/job/Remittance/20/changes
    Build Log: http://ci-dashboard:8080/job/Remittance/20/console
    Preparing to execute si projectinfo for #c:/Remittance
    Preparing to execute si viewproject for #c:/Remittance
    Checkout directory is E:\jenkins\workspace\Remittance
    Successfully updated workspace with 0 updates and cleaned up 0 files!
    Saving current Integrity Project configuration...
    API Response for si viewproject successfully saved to file!
    Successfully saved current Integrity Project configuration to E:\jenkins\jobs\Remittance\builds\2012-01-24_13-49-33\viewproject.dat
    Writing build change log...
    Change log successfully generated: E:\jenkins\jobs\Remittance\builds\2012-01-24_13-49-33\changelog.xml.temp
    No emails were triggered.
    Path To MSBuild.exe: C:\Program Files (x86)\Microsoft Visual Studio 9.0\Common7\IDE\devenv.com
    Executing command: cmd.exe /C "C:\Program Files (x86)\Microsoft Visual Studio 9.0\Common7\IDE\devenv.com" Source\RemittanceEngineSln.sln /clean Release && exit %%ERRORLEVEL%%
    [Remittance] $ cmd.exe /C "C:\Program Files (x86)\Microsoft Visual Studio 9.0\Common7\IDE\devenv.com" Source\RemittanceEngineSln.sln /clean Release && exit %%ERRORLEVEL%%
    Started by user XXXXXX

    Building on master in workspace E:\jenkins\workspace\Remittance

    Change Log: http://ci-dashboard:8080/job/Remittance/20/changes

    Build Log: http://ci-dashboard:8080/job/Remittance/20/console

    Preparing to execute si projectinfo for #p=c:/Program Files/MKS/Software Development/FX360FileProcessor/fx360filepro.pj#RemittancePollingService

    Preparing to execute si viewproject for #p=c:/Program Files/MKS/Software Development/FX360FileProcessor/fx360filepro.pj#RemittancePollingService

    Checkout directory is E:\jenkins\workspace\Remittance\RemittancePollingService

    Successfully updated workspace with 36 updates and cleaned up 568 files!

    Saving current Integrity Project configuration...

    API Response for si viewproject successfully saved to file!

    Successfully saved current Integrity Project configuration to E:\jenkins\jobs\Remittance\builds\2012-01-24_13-49-33\viewproject.dat

    Writing build change log...

    Change log successfully generated: E:\jenkins\jobs\Remittance\builds\2012-01-24_13-49-33\changelog.xml.temp

    Change Log: http://ci-dashboard:8080/job/Remittance/20/changes

    Build Log: http://ci-dashboard:8080/job/Remittance/20/console

    Preparing to execute si projectinfo for #c:/Remittance

    Preparing to execute si viewproject for #c:/Remittance

    Checkout directory is E:\jenkins\workspace\Remittance

    Successfully updated workspace with 0 updates and cleaned up 0 files!

    Saving current Integrity Project configuration...

    API Response for si viewproject successfully saved to file!

    Successfully saved current Integrity Project configuration to E:\jenkins\jobs\Remittance\builds\2012-01-24_13-49-33\viewproject.dat

    Writing build change log...

    Change log successfully generated: E:\jenkins\jobs\Remittance\builds\2012-01-24_13-49-33\changelog.xml.temp

    No emails were triggered.

    Path To MSBuild.exe: C:\Program Files (x86)\Microsoft Visual Studio 9.0\Common7\IDE\devenv.com

    Executing command: cmd.exe /C "C:\Program Files (x86)\Microsoft Visual Studio 9.0\Common7\IDE\devenv.com" Source\RemittanceEngineSln.sln /clean Release && exit %%ERRORLEVEL%%

    [Remittance] $ cmd.exe /C "C:\Program Files (x86)\Microsoft Visual Studio 9.0\Common7\IDE\devenv.com" Source\RemittanceEngineSln.sln /clean Release && exit %%ERRORLEVEL%%

  12. When creating a job with multiple TFS repositories, the config saves correctly and the job runs correctly, but when you open the project again the tfs info is gone.  

  13. It seems to work fine with 2 Git repos, as well as with Git and svn repo.

    Any timeline for supporting  repository browser configuration?

  14. Unknown User (raphau)

    Since Jenkins Version 1.474 this Plugin doesn't work with Git Repositories.

    1. Just tried it on 1.487. Works like a charm.

  15. I was surprised to discover that in order to make it work you need to configure different directories for each repository.

    If I use mercurial SCM, jenkins will clone the repository in a subdirectory, but this plugin will clone it into the root by default. I do not see any good reason for this. It would be essential to have a default behaviour that would match the single repository setup.

    Is there a special reason for implementing it like this?

    1. As far as I get it the plugin just reuses existing git plugin functionality. So the default behavior you're seeing is due to git plugin implementation.

  16. This is a lifesaver, thank you!

  17. Hello,

    Not sure if I need to file a bug or not. Here is what happened:

    I have:

    - Jenkins 1.505 on Windows 2008 Server

    - Git Plugin 1.3.0

    - Git Client Plugin 1.0.2

    -GitHub Plugin 1.5

    - Multiple SCMs plugin 0.2

    My project is connecting to 2 different repositories in Github Enterprise.

    I am using post-commit hooks to trigger builds on Jenkins whenever a change is checked into GHE.

    A change was recently checked into the 2nd repository configured in my project.

    This triggered a build in Jenkins.

    However, the build failed with this error:

    "ERROR: Couldn't find any revision to build. Verify the repository and branch configuration for this job.
    java.io.FileNotFoundException: File 'C:\Jenkins\jobs\MYPROJECT\builds\2013-05-20_17-48-25\changelog.xml.temp' does not exist
    17:52:13 at org.apache.commons.io.FileUtils.openInputStream(FileUtils.java:137)
    17:52:13 at org.apache.commons.io.FileUtils.readFileToString(FileUtils.java:1102)
    17:52:13 at org.apache.commons.io.FileUtils.readFileToString(FileUtils.java:1120)
    17:52:13 at org.jenkinsci.plugins.multiplescms.MultiSCM.checkout(MultiSCM.java:127)
    17:52:13 at hudson.model.AbstractProject.checkout(AbstractProject.java:1353)
    17:52:13 at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:683)
    17:52:13 at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:88)
    17:52:13 at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:588)
    17:52:13 at hudson.model.Run.execute(Run.java:1567)
    17:52:13 at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
    17:52:13 at hudson.model.ResourceController.execute(ResourceController.java:88)
    17:52:13 at hudson.model.Executor.run(Executor.java:237)I checked on the Jenkins server, and in fact C:\Jenkins\jobs\MYPROJECT\builds\2013-05-20_17-48-25\changelog.xml.tempdoes not exist.

    However, C:\Jenkins\jobs\MYPROJECT\builds\2013-05-20_17-48-25\changelog.xml.temp2DOES exist.

    -rwx------+ 1 Administrators Domain Users     452 May 16 12:44 changelog.xml*
    -rwx------+ 1 Administrators Domain Users     137 May 17 11:29 changelog.xml.temp2*

    So, it seems like the MSCM plugin records the changelog for the 2nd repo in changelog.xml.temp2, but only looks for changelog.xml.temp when a build is triggered via Github web hook.

  18. Hi,

    this plugin is awesome!

    However, we noticed that there might be a but when you use the login\password from the system configuration. We set the user and password in the "Global Password" configuration, so the users do not need to fill in their personal user/passwd when creating the jobs. However, when we use this feature (to get from the global password), and we use multiple SCM, it does not save the information about the SCM, i.e., after we fill the information in and we return to the "Configure" page again, the SCM configurations are gone!

    However, if we set directly the user/passwd, it saves correctly.

    Regards,

  19. a tip: if there are mutiple subversion repository,it's better setup as add new location than add a svn scm,such as the example as below:

    config.xml
    <?xml version='1.0' encoding='UTF-8'?>
    <project>
      <actions/>
      <description></description>
      <keepDependencies>false</keepDependencies>
      <properties/>
      <scm class="org.jenkinsci.plugins.multiplescms.MultiSCM" plugin="multiple-scms@0.2">
        <scms>
          <hudson.scm.SubversionSCM plugin="subversion@1.45">
            <locations>
              <hudson.scm.SubversionSCM_-ModuleLocation>
                <remote>http://yoursvn/trunk</remote>
                <local>trunk</local>
                <depthOption>infinity</depthOption>
                <ignoreExternalsOption>false</ignoreExternalsOption>
              </hudson.scm.SubversionSCM_-ModuleLocation>
              <hudson.scm.SubversionSCM_-ModuleLocation>
                <remote>http://yoursvn/services/</remote>
                <local>service</local>
                <depthOption>infinity</depthOption>
                <ignoreExternalsOption>false</ignoreExternalsOption>
              </hudson.scm.SubversionSCM_-ModuleLocation>
            </locations>
            <excludedRegions></excludedRegions>
            <includedRegions></includedRegions>
            <excludedUsers></excludedUsers>
            <excludedRevprop></excludedRevprop>
            <excludedCommitMessages></excludedCommitMessages>
            <workspaceUpdater class="hudson.scm.subversion.UpdateUpdater"/>
            <ignoreDirPropChanges>false</ignoreDirPropChanges>
            <filterChangelog>false</filterChangelog>
          </hudson.scm.SubversionSCM>
          <hudson.plugins.git.GitSCM plugin="git@1.4.0">
            <configVersion>2</configVersion>
            <userRemoteConfigs>
              <hudson.plugins.git.UserRemoteConfig>
                <name></name>
                <refspec></refspec>
                <url>http://yourgit/xxx.git</url>
              </hudson.plugins.git.UserRemoteConfig>
            </userRemoteConfigs>
            <branches>
              <hudson.plugins.git.BranchSpec>
                <name>master</name>
              </hudson.plugins.git.BranchSpec>
            </branches>
            <disableSubmodules>false</disableSubmodules>
            <recursiveSubmodules>false</recursiveSubmodules>
            <doGenerateSubmoduleConfigurations>false</doGenerateSubmoduleConfigurations>
            <authorOrCommitter>false</authorOrCommitter>
            <clean>false</clean>
            <wipeOutWorkspace>false</wipeOutWorkspace>
            <pruneBranches>false</pruneBranches>
            <remotePoll>false</remotePoll>
            <ignoreNotifyCommit>false</ignoreNotifyCommit>
            <useShallowClone>false</useShallowClone>
            <buildChooser class="hudson.plugins.git.util.DefaultBuildChooser"/>
            <gitTool>Default</gitTool>
            <submoduleCfg class="list"/>
            <relativeTargetDir>xxx</relativeTargetDir>
            <reference></reference>
            <excludedRegions></excludedRegions>
            <excludedUsers></excludedUsers>
            <gitConfigName></gitConfigName>
            <gitConfigEmail></gitConfigEmail>
            <skipTag>false</skipTag>
            <includedRegions></includedRegions>
            <scmName></scmName>
          </hudson.plugins.git.GitSCM>
        </scms>
      </scm>
      <canRoam>true</canRoam>
      <disabled>false</disabled>
      <blockBuildWhenDownstreamBuilding>false</blockBuildWhenDownstreamBuilding>
      <blockBuildWhenUpstreamBuilding>false</blockBuildWhenUpstreamBuilding>
      <triggers class="vector">
        <hudson.triggers.SCMTrigger>
          <spec>* * * * *</spec>
          <ignorePostCommitHooks>false</ignorePostCommitHooks>
        </hudson.triggers.SCMTrigger>
      </triggers>
      <concurrentBuild>false</concurrentBuild>
      <builders>
        <hudson.tasks.Shell>
          <command>pwd</command>
        </hudson.tasks.Shell>
      </builders>
      <publishers/>
      <buildWrappers/>
    </project>
  20. Hello,

    It seems to not work with bazaar plugin, while bazaar plugin works fine.

    Has someone already tried/fixed it ?

    I get this error :$ bzr revision-info -d /home/jenkins/workspace/Newterra
    info result: bzr revision-info -d /home/jenkins/workspace/Newterra returned 0. Command output: "205 zz.zz@zz.tld-20130402172310-ucho57xkqtbqm4z9
    " stderr: ""
    FATAL: org.jenkinsci.plugins.multiplescms.MultiSCMRevisionState cannot be cast to hudson.plugins.bazaar.BazaarRevisionState
    java.lang.ClassCastException: org.jenkinsci.plugins.multiplescms.MultiSCMRevisionState cannot be cast to hudson.plugins.bazaar.BazaarRevisionState
    at hudson.plugins.bazaar.BazaarSCM.buildEnvVars(BazaarSCM.java:460)
    at org.jenkinsci.plugins.multiplescms.MultiSCM.buildEnvVars(MultiSCM.java:71)
    at hudson.model.AbstractBuild.getEnvironment(AbstractBuild.java:938)
    at hudson.plugins.bazaar.BazaarSCM.cleantree(BazaarSCM.java:340)
    at hudson.plugins.bazaar.BazaarSCM.update(BazaarSCM.java:279)
    at hudson.plugins.bazaar.BazaarSCM.checkout(BazaarSCM.java:260)
    at org.jenkinsci.plugins.multiplescms.MultiSCM.checkout(MultiSCM.java:117)
    at hudson.model.AbstractProject.checkout(AbstractProject.java:1387)
    at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:676)
    at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:88)
    at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:581)
    at hudson.model.Run.execute(Run.java:1593)
    at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
    at hudson.model.ResourceController.execute(ResourceController.java:88)
    at hudson.model.Executor.run(Executor.java:247)Thanks.


  21. Help. I use SVN+GIT. SVN changes aren't shown. change in GIT doesn't understand Cyrillics

  22. Hi,

    I'm using jenkins version 1.509.2 and Multiple SCM plugin version 0.2.

    I have 4 git clones. I'm not sure what was the last change I have made but I think it was adding the Branch Specifier to each clone, poll scm started to run in an endless loop.

    Has anyone encountered such issue?

    Thanks

    Gil

    1. OK, found the issue.

      It had nothing to do with the "Branch Specifier" but with the "Local subdirectory for repo".

      When I use two folder path (e.g. a/b), the endless loop starts

      Thanks

      Gil

  23. Hi,

    Is there any plan to support Repository browser configuration in future?

    It would be really helpful if this plugin supports Repository browser configuration.

    Thanks,

    Ravi Kumar G

  24. Does it work with notifyCommit? http://jenkins/mercurial/notifyCommit?url=*** works for jobs with single repository, but if repository used in multiple scm plugin, I'm getting a reply: "No Mercurial jobs found using repository: ***"

  25. Hi,

    We have one project that polls from 2 Perforce server using this plugin.I noticed that there is always "Changes found" in the polling log although I'm pretty sure we don't have any new check-ins to both Perforce. Setting "Poll Exclude File(s)" doesn't help at all. Just wonder how do we determine the change log with multiple SCMs.

    Thanks.

    Jason

  26. works fine with 3 gits,.

    Repository browser configuration is missing and is vital for normal work.

  27. Jenkins 1.562

    PTC Integrity Plugin 1.24

    Multiple SCMs plugin 0.3

    Not sure bug from where, I configured a username/password under Manage Jenkins -> Configure System -> Integrity CM

    I created a Multiple SCMs job with its own username/password different than the one above.

    The username/password in my Multiple SCMs job does not get saved. Instead the ones from Manage Jenkins -> Configure System -> Integrity CM are always used.

    When I checked my job's config.xml under C:\Program Files\Jenkins\jobs\MultipleSCMJOB I see the right username there, but not when I view it from my browser neither when I execute my job !

  28. I have multiple repos each with multiple branches using Multiple SCMs plugin 0.3. Can someone explain the default branch checkout strategy? Would it just checkout the latest commit on each repo for the branches specified?

  29. We have 74 git repositories that we clone every build.  It takes approximately seven minutes to clone them all serially.  When we do it in parallel through scripts, it takes about one minute.  If you're bored and want to maybe add a "pull in parallel" option, it would not go unnoticed...  :)

  30. Does anyone know if there's a way to have the absolute path or complte git url instead of the relative path for changed files/dirs?

    I'm using the multiple-scm and email-ext plugin to generate some detailed build information. When I use the CHANGES token to show all the commits I cannot distinguish between changes to any of the repos. The paths are all relative to the root of the repo but since there are mutple repos I cannot tell which repo the change belongs to.

    The email-ext plugin has access to author, date, message, path, and revision which I assuming is provided by the scm plugin. Is it possible to alter the path to prefix it with the repo name or perhaps provide additional data to email-ext? 

  31. I am using jenkins 1.598 with multiple SCM plugin 0.4 beta, also tried 0.3 and I am having an issue where I enter a guerrit Repo SCM and everything is saved correctly but when I go back into my project configuration all the saved fields do not load.  

    Thanks for your help.

  32. Hi,

    It seems that the TFS settings are not loaded/displayed in the MultipleSCM configuration view? It's possible to add a new TFS configuration and it appears to be written to the job config. The job even runs OK. But when you reload the job configuration, the previously saved TFS settings are not displayed...

  33. Great plugin,

    Works with Git finally. 

    Thanks

  34. Thanks for the useful plugin. Wanted to report my findings on something you said you had not tested - using this plugin with a master/slave configuration. I find that if I try to use this plugin to use multiple SCMs on my slave, the repos are created but the file permissions are incorrect for the entire workspace directory. Specifically, I'm running a win7 slave, and the workspace dir is created as follows (as seen by cygwin):

    d---------+ 1 jenkinsuser    Domain Users 0 Jul 23 09:26 Workspace_for_MultipleSCM_Job
    drwxrwxrwx+ 1 jenkinsuser    Domain Users 0 Jun  2 13:34 Workspace_for_SingleSCM_Job
    

    Not asking for a fix since you didn't promise it would work, but if you are interested in trying to tackle this, I would be happy to give you some help/additional info. Thanks.

  35. Hi,

        We have a requirement related to Multiple SCM plugin. Actually we used GIT plugin for Mutijob. But When we use Multijob we got deadlock issue. So we changed our configuration.

        We use Mutiple SCM. But how to implement custom tagging?

        Please help me if you have any suggestion.

    Thanks,

    Jivesh

  36. Hi,

    I have two different workspace for 2 different repository and i want to tag them. How can i do it...? Please help met as am new to this.

    Thanks,

    Titto