Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 91 Next »

Plugin Information

View Pretested Integration on the plugin site for more information.

Developed by

Sponsored by

Introduction

The Pretested Integration Plugin offers a branchy approach to pretested integration (also known as pre-tested commits), which upholds the invariant; that for a specific branch, known as the integration branch, all commits have been verified.
This plugin is developed by Praqma and sponsored by Atmel - It's maintained in the scope of Joint Open Source Roadmap Alliance (JOSRA).

The plugin rely on the SCM plugin to establish the workspace and then takes over to do the integration. Last the job make the decision whether to like and commit the result or hate and discard it. To get a comprehensive introduction to how the plugin relies on the SCM plugin please read the blog post at the JOSRA: Pretested Integration Plugin.

References

Limitations, requirements and dependencies

  • The plugin only supports the Jenkins Freestyle project (not Maven or Multi-configuration aka. Matrix). In other project types than the Freestyle, the UI of Pretested Integration will not show up.
  • Git 1.9.x must be installed and available on the Jenkins build executor in command line. On Windows we depend on git commands to be available in the Windows command shell (not Git Bash, nor Cygwin).
  • Git configuration dependencies: the user account of the Jenkins build executor must be available to push to your integration repository. This typically means that you have to use either http- or ssh-credentials configured for the user account running the Jenkins slave.

Support and contact

Please send us an email on support@praqma.net if you have a request or question regarding the plugin.

If you find an error or have feature requests, use the Jenkins issue tracker.

Comments and discussions on this page might not always be noticed.

The recommend setup and git workflow

Here is a simple git workflow where you can work on a features branch, and when ready push to a ready-branch. The Pretested Integration plugin, if configured as described will then pick up your changes, merge them and verify them. If verified they are integrated on the integration branch. The ready branch are automatically deleted if integration was successful.

Recommendation:

  • Use one repository in your job configuration - the integration repository. Avoid using several repositories - model your dependencies in other (better) ways.
  • Use master as integration branch (destination).
  • Use ready/** as specifier for ready branches - only branches matching this expression will trigger the build

The simple plugin configuration

To configure the plugin to pick up ready branches, verify them and deliver them to the integration branch use the following configuration:


Configure the SCM as above.

  • Recommend ready branch specifier is the 'Branch Speciifier' : ready/**
  • Remember to add the two additional behaviours to ensure a clean work space and that old deleted branches are cleaned
  • It is recommended to name your repository in the 'Name' field.


Configure the Pretested Integration plugin with integration branch 'master' and the named repository 'origin'.
There is a Pretested Integration plugin post-build step, which ensure verified changes are published. You don't have to configure it - it is automatically added.

Now you only need to configure your definition of done - the verification. How should the job verify the commits? It can be compile and measure compiler warning, build and run unit tests and check that coverage do not drop.

The simple Git workflow

Get your repository up to date:

git fetch --prune

git checkout master

git pull origin master

Create a feature- or development- or... branch

git checkout -b feat_1337-improved-login-screen

...work, stage and commit changes.

Then push changes to a ready branch, which is basically just a branch following a naming conventions for branches matching ready/**.

git push origin feat_1337-improved-login-screen:ready/feat_1337-improved-login-screen

The change will be picked up by the plugin if configured as shown in next section.

  • You can then delete your local branch and continue with a new feature or development branch.
  • You are free to push to any branch name not matching the ready branch naming convention without triggering an integration build.

Help and error messages

*NOT YET DONE - work in progress* (We have gathered some common errors and problems seen, together with some suggested solutions - Help and error messages)

Demo jobs and example on commit messages and output

We have created some Jenkins job on our public Jenkins master that shows the plugin in action, in the version we have installed (typically latest one or even latest snapshop).

There is actually a demo job you can run, that triggers a demonstration:

The commits end up in our Github test repository called blackhole

Accumulated strategy - commit message

Version 2.3.0 vs. version 2.2.3

Note the difference with committer and author between the two version.
From version 2.3.0 the integration commit is authored by author of the last commit on the development branch.

In the example it is the "demo-job-user" that is the author, while the committer the git user configured for the Jenkins build slave environment (ReleasePraqma in this example). |

Squashed strategy - commit message

Version 2.3.0 vs. version 2.2.3

Note the difference with committer and author between the two version.
From version 2.3.0 the integration commit is authored by author of the last commit on the development branch.

In the example it is the "demo-job-user" that is the author, while the committer the git user configured for the Jenkins build slave environment (ReleasePraqma in this example). |

Accumulated strategy - build console output

Version 2.3.0 build console output example
[PREINT] Preparing environment using Pretested Integration Plugin 2.3.0-SNAPSHOT (cbd24) 
[PREINT] Checking out integration branch master:
[PREINT]  git checkout -B master origin/master
[pretested-integration-plugin__demo-job-accumulated-strategy] $ git checkout -B master origin/master
[PREINT]  git pull origin master
[pretested-integration-plugin__demo-job-accumulated-strategy] $ git pull origin master
[PREINT] Preparing to merge changes in commit b1e190f90a8c1a22402af4492c70ae45eb32c981 on development branch origin/ready/feature-accumulated-32 to integration branch master
[PREINT] Collecting commit messages on development branch origin/ready/feature-accumulated-32
[PREINT] Done collecting commit messages
[PREINT] Collecting author of last commit on development branch
[PREINT] Done colecting last commit author: demo-job-user <support@praqma.net> 1435686815 +0200
[PREINT] Starting accumulated merge (no-ff) - without commit:
[PREINT]  git merge -m "Accumulated commit of the following from branch 'origin/ready/feature-accumulated-32':

commit b1e190f90a8c1a22402af4492c70ae45eb32c981
Author: demo-job-user <support@praqma.net>
Date:   Tue Jun 30 19:53:35 2015 +0200

    Demo commit message for Pretested Integration:
    
    This is a longer commit message about what this commit is about.
    
    We use this commit to illustrate the Prested Integration Plugin commit messages, based on their strategies. There are two demo jobs, one for each strategy and a job that does two commits, one on two different branches to be picked up as ready branches.
    

" b1e190f90a8c1a22402af4492c70ae45eb32c981 --no-ff --no-commit
[pretested-integration-plugin__demo-job-accumulated-strategy] $ git merge -m "Accumulated commit of the following from branch 'origin/ready/feature-accumulated-32':

commit b1e190f90a8c1a22402af4492c70ae45eb32c981
Author: demo-job-user <support@praqma.net>
Date:   Tue Jun 30 19:53:35 2015 +0200

    Demo commit message for Pretested Integration:
    
    This is a longer commit message about what this commit is about.
    
    We use this commit to illustrate the Prested Integration Plugin commit messages, based on their strategies. There are two demo jobs, one for each strategy and a job that does two commits, one on two different branches to be picked up as ready branches.
    

" b1e190f90a8c1a22402af4492c70ae45eb32c981 --no-ff --no-commit
[PREINT] Accumulated merge done
[PREINT] Merge was successful
[PREINT] Starting to commit accumulated merge changes:
[PREINT]  git commit --no-edit "--author=demo-job-user <support@praqma.net> 1435686815 +0200"
[pretested-integration-plugin__demo-job-accumulated-strategy] $ git commit --no-edit "--author=demo-job-user <support@praqma.net> 1435686815 +0200"
[PREINT] Commit of accumulated merge done
[PREINT] Commit was successful
[pretested-integration-plugin__demo-job-accumulated-strategy] $ /bin/sh -xe /tmp/hudson2956180739635715796.sh
+ echo Build and verification was successful
Build and verification was successful
[PREINT] Performing pre-verified post build steps
[PREINT] Pushing changes to integration branch:
[PREINT]  git push origin master
[pretested-integration-plugin__demo-job-accumulated-strategy] $ git push origin master
[PREINT] Done pushing changes
[PREINT] Deleting development branch:
[PREINT]  git push origin :ready/feature-accumulated-32
[pretested-integration-plugin__demo-job-accumulated-strategy] $ git push origin :ready/feature-accumulated-32
[PREINT] Done deleting development branch

Squashed strategy - build console output

Version 2.3.0 build console output example
[PREINT] Preparing environment using Pretested Integration Plugin 2.3.0-SNAPSHOT (cbd24) 
[PREINT] Checking out integration branch master:
[PREINT]  git checkout -B master origin/master
[pretested-integration-plugin__demo-job-squashed-strategy] $ git checkout -B master origin/master
[PREINT]  git pull origin master
[pretested-integration-plugin__demo-job-squashed-strategy] $ git pull origin master
[PREINT] Preparing to merge changes in commit 26576c087d13a50179a13054cc6edac603271abd on development branch origin/ready/feature-squashed-32 to integration branch master
[PREINT] Collecting commit messages on development branch (for debug printing): origin/ready/feature-squashed-32
[PREINT] Done collecting commit messages
[PREINT] Collecting author of last commit on development branch
[PREINT] Done colecting last commit author: demo-job-user <support@praqma.net> 1435686849 +0200
[PREINT] Starting squash merge - without commit:
[PREINT]  git merge --squash origin/ready/feature-squashed-32
[pretested-integration-plugin__demo-job-squashed-strategy] $ git merge --squash origin/ready/feature-squashed-32
[PREINT] Squash merge done
[PREINT] Merge was successful
[PREINT] Starting to commit squash merge changes:
[PREINT]  git commit --no-edit "--author=demo-job-user <support@praqma.net> 1435686849 +0200"
[pretested-integration-plugin__demo-job-squashed-strategy] $ git commit --no-edit "--author=demo-job-user <support@praqma.net> 1435686849 +0200"
[PREINT] Commit of squashed merge done
[PREINT] Debug print - commit message for squash merge:
Demo commit message for Pretested Integration:

This is a longer commit message about what this commit is about.

We use this commit to illustrate the Prested Integration Plugin commit messages, based on their strategies. There are two demo jobs, one for each strategy and a job that does two commits, one on two different branches to be picked up as ready branches.

[PREINT] Commit was successful
[pretested-integration-plugin__demo-job-squashed-strategy] $ /bin/sh -xe /tmp/hudson3927922213950794742.sh
+ echo Build and verification was successful
Build and verification was successful
[PREINT] Performing pre-verified post build steps
[PREINT] Pushing changes to integration branch:
[PREINT]  git push origin master
[pretested-integration-plugin__demo-job-squashed-strategy] $ git push origin master
[PREINT] Done pushing changes
[PREINT] Deleting development branch:
[PREINT]  git push origin :ready/feature-squashed-32
[pretested-integration-plugin__demo-job-squashed-strategy] $ git push origin :ready/feature-squashed-32
[PREINT] Done deleting development branch

Manual building

In general you are not able to use 'Build now' with a pretested integration job configuration. The Pretested Integration Plugin is first in action when a "workspace" is handed over from the Git Plugin (or Multiple SCMs Plugin), and typically those will serve the last build revision again. If that succeeded last time, the revision is deleted after the integration and retrying the integration fails.

Some successful and failing cases using manual builds are:

  • If last build failed, thus the integration failed, for a non-persistent error (slave problem, licensing problem ...) rebuilding the job can succeed if no other build have been executed since last time.
  • Rebuilding after a successful build will typically fail, if the build is rebuilding the same revision again. That revision doesn't not exist as it is deleted after a successful integration in last build.
  • If you have done a job configuration change, and need to trigger the job to test the configuration you typically need to make a commit that triggers the job. Push a commit to a ready branch, or wait for one.
  • There is a work around, that often enables you to build manually: Make the job parametrized with BRANCH_TO_BUILD and use that variable in the 'branch specifier'. Make BRANCH_TO_BUILD have the default ready-branch specifier, so if not given the job works as if there were no parameters. If you now build the job manually, you can type in a branch to build.

You are welcome to contribute with ideas and use-cases for manual build and support for it on the roadmap in the Trello board.

Using multiple repository configurations

The plugin is designed to work on only one repository - as in you can only integrate on one repository pr. Jenkins job, but your Jenkins job can be used with multiple repositories for eg. checking out several repositories into a given folder structure you need for building the software. Though we consider this a work-around for not having the right dependency management, it can be done. You have the choice of using the Git Plugin or the Multiple SCMs Plugin, but need to follow the configuration examples below.

Requirements if using multiple repository jobs

Every repository should be explicitly and uniquely named* in the git repository configuration under 'Advanced'. Then use this as integration repository in the prested integration configuration

Example with multiple git repositories

  • The two repositories 'capital-letters' and 'numbers' are checked out to the same workspace.
  • 'numbers' are the repository that is being integrated, so the branch specifier will typically match that patter ('capital-letters' might just be needed for building)
  • The pretested integration configuration now explicitly states that the 'numbers' repository is the integration repository.
  • Also remember to add the two additional behaviours to ensure a clean work space and that old deleted branches are cleaned

Basically the same setup needs to be applied if using the Multiple SCMs plugin, but you can benefit from additional behaviours like checking each git repository out to a subdirectory if needed. Still every git repository must be explicitly and uniquely named.

Acknowledgement

Code contributions were initially made by Computer Science students at University of Copenhagen as part of a study project

Ronni Elken Lindsgaard
Alexander Winther Uldall
Esben Skaarup
Andreas Frisch

Issues

type key summary

Data cannot be retrieved due to an unexpected error.

View these issues in Jira

Changes

(unreleased) 2.3.0

Available as snapshot artifact in our Jenkins pipeline

  • More logging and information in general for better debugging and understanding why builds fail. Part of feature implementation and re-factoring for JENKINS-27690, JENKINS-28590
  • Output to build console is extended and streamlines between the two different integration strategies (JENKINS-28590)
  • JENKINS-28590: Author of the integration commit is the author of the last commit on the development branch (instead of always the build user).
  • JENKINS-28596: It is possible for the accumulated to use a custom integration branch, a bug was fixed in this area.
  • Test were written to check commit message with quotes will not fail integration ( JENKINS-27662). There are still issue though, reported in unfixed issue JENKINS-28640
Version 2.2.3
  • JENKINS-27516 Plugin hangs - reproduced with failed earlier build. A regression problem, related to never Jenkins cores and that the plugin used semaphores. Problem was seen, by the plugin prints the version number in the job console and then the job hangs.
Version 2.2.1
  • JENKINS-26568 New accumulated commit message. Almost identical to what you get from squash commit, thus it have much more traceability by including the commit messages from all included commits.
Version 2.2.0
Version 2.1.2
  • Implemented logging tracing
Version 2.1.1
Version 2.1.0
  • Protected master branch (Plugin tries to delete origin/master JENKINS-24286)
  • Re-using last commit message in accumulated strategy (Improve commit message JENKINS-24285)
  • Removed the "origin" from the description (JENKINS-24284)
  • When squashing commits, now using author from tip of branch (JENKINS-24443)
  • Additional tests added as well
Version 2.0
  • Git integration is now supported
Version 1.1
  • Dependency of Mercurial plugin set to 1.39 due to previous failure to trigger on merge commits
  • Removed UI elements that should not have been there
Version 1.0
  • Release of the first stable versionI
  • No labels