Jenkins : Build Failure Analyzer

This plugin analyzes the causes of failed builds and presents the causes on the build page. It does this by using a knowledge base of build failure causes that is built up from scratch.
Saving statistics about failure causes is also possible.

Plugin Information

View Build Failure Analyzer on the plugin site for more information.

Older versions of this plugin may not be safe to use. Please review the following warnings before using an older version:

Knowledge base

The plugin comes with an empty knowledge base of failure causes. Populating this knowledge base is done by using the link "Failure Cause Management".
The link is shown only if the permission UpdateCauses is set for the current user. Press "Create new" and add a name and a description for the Failure Cause.
The description should contain the reason why this build failed as well as possible solutions for the build failure.

One or more optional categories can be added as a whitespace-separated list. These categories are only used for the statistics part of the plugin.
Add one or more Indications by pressing "Add Indication". By default, only one type of Indication exists, a "Build Log Indication".
Plugin developers can add new Indication types and more are planned to be added to the plugin itself.
The Build Log Indication searches through the build log, one line at a time, for a regular expression.

It uses Pattern.match, so the regular expression needs to match should be as follows

Match type


Single line

.*some words in the middle of a line.*

Multi line

some words here(.*\s)and some words here


Adding new failure causes and indications to the knowledge base.

From version 1.3.1 of the plugin, regular expressions can be tested on the Failure Cause Management page, in two different ways:

  • Writing a text in the text field shown above and testing against that.
  • Writing a URL to a build log in the text field. The plugin then runs through the log trying to match the regexp.

When accessing the Failure Cause Management page from a build, the URL will be added to the text field automatically.

The plugin comes with two ways of saving the knowledge base:

  • Local knowledge base. Saves the knowledge base in memory and serializes it as an xml file on the local Jenkins server (i.e. the "standard" Jenkins way of saving information).
  • MongoDB knowledge base. Saves the knowledge base in a Mongo database. This can be used to share the
    same knowledge base between servers. The knowledge base is still cached locally in-memory to avoid unnecessary database accesses.
    Plugin developers can add new knowledge base types.

Build log scanning

All builds on the server that are non-successful (aborted, failed, unstable) will be scanned for all failure causes. 
If an indication is found, the description will be put directly on the build page, with a link to the matching line in the build log.
If no cause is found, a text stating this will be shown. The text is configurable on the main configuration page of the server.

The build page when the build failure analyzer has found a failure cause.


The build log with the matching line marked in red.

Statistics

If MongoDB or some knowledge base type that supports statistics is used, statistics will be saved to that database. The same information
that is shown on the build page is saved to the database.

Administrative settings

On the configure system page in Jenkins, under Build failure analyzer, some new settings are available:

  • Enabled - since the plugin scans all failed builds, we felt the need to be able to disable the scanning. Uncheck to disable.
  • Text when no failure causes are found - text to show for failed builds whose failure cause is not found in the knowledge base.
  • Storage type - Jenkins Local or Mongo DB, as described above. For Mongo DB, configuration details to fill in are as follows: Host, Port, Database name, Username and Password for the Database, Enable Statistics logging (described above).
  • Convert knowledge base - If this check box is checked when the configuration is saved and a change has been made to the knowledge base settings, the data from the old knowledge base will be added to the new one. Note that duplicates could appear this way, so make sure that you untick this check box if you for example just have changed the username or password, or if you want to start with a clean knowledgebase.
  • Send notifications to Gerrit-Trigger-plugin - if enabled, will send the text for the found failure cause via the gerrit-trigger-plugin to Gerrit.
  • Concurrent scans - To speed up the scanning, each build will get a threadpool of this number of threads, with each thread handling one indication. For a small system, 3 is usually enough.

Token Macro integration

Usage:

${BUILD_FAILURE_ANALYZER, includeTitle=true, includeIndications=true, useHtmlFormat=true, noFailureText="Sometext"}


Parameters (=default):

  • includeTitle (=true) -- When true, the "Identified problems:" title will appear over the causes.
  • includeIndications (=true) -- When true, the indication numbers and links into the console log are included in the token replacement text.
  • useHtmlFormat (=false) -- When true, the replacement will be an HTML snippet.
  • wrapWidth (=0) -- Wrap long lines at this width. If wrapWidth is 0, the text isn't wrapped. Only applies if useHtmlFormat == false.
  • noFailureText (="") -- Text to provide if there are no found failures

Token macro in Jenkins Pipeline DSL:


node {
   def buildFailure = tm('${BUILD_FAILURE_ANALYZER}')
}

Placeholders in description

Replace description placeholders from captured expressions in found indications

Substitutions may be made within the description with placeholders of the form ${I,G}, where I is the indication number and G is the captured group within the indication expression. e.g., ${1,1} would be replaced with the first indication's first captured group and ${1,2} would be replaced with the first indication's second captured group.

Tips & Tricks

Aggregate statistics to Graphite

If you are using the MongoDB KnowledgeBase, you can use these scripts in a cron job to aggregate the statistics into Graphite. Note that they only work on mongo 2.x node js driver

Known bugs

A full list of open bugs raised against build-failure-analyzer component..

Key Summary T Created Updated Assignee Reporter P Status
JENKINS-65941 bfa cannot connect to DocumentDB(mongodb) if tls is enabled Bug Jun 21, 2021 Sep 01, 2021 Tomas Westling Rajeev Blocker Open
JENKINS-66098 Newlines are not preserved in BFA failure cause descriptions on build summary page Bug Jul 08, 2021 Jul 28, 2021 Tomas Westling Eric Herrmann Major Open
JENKINS-66078 Builds failure page says add a cause, but can't as there is no link. New Feature Jul 07, 2021 Jul 07, 2021 Tomas Westling Rick Patterson Minor Open
JENKINS-64487 BFA Prometheus Metrics New Feature Dec 19, 2020 Jul 02, 2021 Philipp Philipp Minor Open
JENKINS-66014 Add support to disable retryWrites for DocumentDB Improvement Jun 30, 2021 Jun 30, 2021 Tomas Westling Rajeev Blocker Open
JENKINS-65876 As a User of Build Failure Analyzer Plugin I want to have two new fields in it Improvement Jun 11, 2021 Jun 11, 2021 Tomas Westling Aude Major Open
JENKINS-60578 Support to export failure causes in excel New Feature Dec 24, 2019 Jun 01, 2021 Tomas Westling Gajanan Mahajan Minor Open
JENKINS-65384 Display list of (and link to) failed jobs per failure cause Improvement Apr 16, 2021 Apr 16, 2021 Tomas Westling Bram Mertens Minor Open
JENKINS-64094 2.27.0 version in maven is causing issues with dependabot Task Nov 01, 2020 Apr 04, 2021 Tomas Westling Arnaud TAMAILLON Minor Open
JENKINS-65236 Allow "count" threashold on the pattern. New Feature Mar 25, 2021 Mar 25, 2021 Tomas Westling Michael Carter Minor Open
JENKINS-64819 build-failure-analyzer-plugin: Migrate documentation from Wiki to GitHub Improvement Feb 08, 2021 Feb 08, 2021 Stephen Topley Stephen Topley Minor Open
JENKINS-64380 "Slack Channel" field does not recognize name of the channel Bug Dec 07, 2020 Dec 07, 2020 Tomas Westling Dariusz Lisiak Minor Open
JENKINS-64321 Multi-line build log indication stops matching after some number of characters Bug Nov 26, 2020 Nov 26, 2020 Tomas Westling Dave Rigby Major Open
JENKINS-63793 BFA plugin version suitable for MongoDb client 5 Bug Sep 28, 2020 Oct 05, 2020 Tomas Westling Thrupti NS Minor Open
JENKINS-63773 pls support Jenkins configuration as code in BUILD_FAILURE_ANALYER Improvement Sep 24, 2020 Sep 24, 2020 Tomas Westling Roman Maharshak Minor Open
JENKINS-63077 Using BUILD_FAILURE_ANALYZER token in office365Connector results in no reasons Bug Jul 15, 2020 Jul 15, 2020 Tomas Westling NoNo User Major Open
JENKINS-62848 build-failure-analyzer may break with form tables-to-divs changes Bug Jun 29, 2020 Jun 29, 2020 Tomas Westling Félix Queiruga Balado Minor Open
JENKINS-62690 NoSuchMethodError: TimeTableXYDataset.add(Lorg/jfree/data/time/TimePeriod;DLjava/lang/String;)V Bug Jun 15, 2020 Jun 16, 2020 Tomas Westling Torsten Reinhard Minor Open
JENKINS-62600 BFA not matching all the cases Bug Jun 08, 2020 Jun 09, 2020 Tomas Westling Alice Test Critical Open
JENKINS-62399 BFA Results Unavailable During Run Bug May 21, 2020 May 23, 2020 Tomas Westling Garrett Fredley Minor Open
Authenticate to retrieve your issues
Showing 20 out of 119 issues

Change Log

From 1.23.1 the change log is on GitHub releases

Version 1.23.0 (released Aug 23, 2019)

  • Update jackson from 2.9.3/2.9.7 to 2.9.9 CVE-2019-12086 (Pull #99)
  • JENKINS-47027 - "Not valid UTF8" when using umlauts in Failure cause management Open  Fix umlauts (Pull #109)
  • New feature: fallback causes (Pull #108)
  • JENKINS-22026 - Propagate failure causes from downstream builds Reopened  New DownstreamBuildFinder using build-cache-plugin (Pull #107)
  • JENKINS-54439 - Support configuration as code plugin Closed  - JCasC compatibility (Pull #97)
  • JENKINS-42755 - BFA fails to catch pipeline related exceptions Closed - Missing pipeline logs (Pull #103)
  • JENKINS-54840 - UnsupportedOperationException getLogFile Closed  - Stop calling get log file (Pull #106)
  • Upgrade parent pom and checkstyle (Pull #98)
  • JENKINS-57812 - Test connection doesn't work for mongo DB Closed  - Fix test connection (Pull #102)
  • JENKINS-55110 - BFA 1.21.0 fails to create a failure cause / scan jobs Closed  - Fix deserialisation for mongo (Pull #105)

Version 1.23.0-beta-1 (released Jun 28, 2019)

Version 1.22.0 (released Feb 15, 2019)

  • Add empty check to password (Pull #95(Mongo Client regression)
  • Add support for Token Macro tm pipeline step (Pull #94)

Version 1.21.0 (released Dec 6, 2018)


Version 1.20.0 (released Jun 14, 2018)

  • Moved to a slightly more modern minimum Jenkins core version (2.7.3) (Pull #81, #83)
  • Display HTML for cause description in matrix (Pull #84)
  • Improve support sorting "Never" on cause management page (Pull #86)

Version 1.19.1 (released Jan 8, 2018)

Version 1.19.0 (released May 5, 2017)

  • Use fixed pool size, parse all single line matchers in one thread (Pull #57)

Version 1.18.1 (released Mar 7, 2017)

  • Corrected JSON structure of RabbitMQ message (Pull #66)

Version 1.18.0 (released Feb 17, 2017)

  • Added a FailureCauseProvider for the MQ Notifier plugin (Pull #64)
  • JENKINS-41279 Fixed broken div wrapping (Pull #63)
  • Better feedback to Gerrit when no FailureCauses are found for a build (Pull #61)

Version 1.17.2 (released Oct 21, 2016)

  • JSON Serialization fix for MultiLineBuildLogIndication (Pull #59)
  • Performance fix for MultiLineBuildLogIndication (Pull #59)
  • Line break fix (Pull #58)

Version 1.17.1 (released Sep 5, 2016)

  • Fix Gerrit feedback for nested builds (Pull #56)
  • Introduced and separated out Renderer for report formatting (Pull #55)
  • Escape single quote correctly (PR #54)

Version 1.17.0 (released Aug 17, 2016)

  • Add configurable limit for log size to be scanned. (Pull #53)

Version 1.16.0 (released Jun 20, 2016)

Version 1.15.0 (released Apr 19, 2016)

Version 1.14.0 (released Apr 5, 2016)

Version 1.13.5 (released Feb 17, 2016)

  • Added build.displayName to stored statistics. (Pull #42)

Version 1.13.4 (released Feb 5, 2016)

  • Some minor formatting fixes in UI strings. (Pull #40)
  • Scanning-threads now gets the name of the indication and pattern currently being scanned for, for easier thread dump analysis. (Pull #41)

Version 1.13.3 (released Jan 5, 2016)

Version 1.13.2 (released Nov 25, 2015)

  • Fixed a issue when testing expressions on build logs located in folders. (Pull #36)

Version 1.13.1 (released Sept 25, 2015)

Version 1.13.0 (released Apr 10, 2015)

Version 1.12.1 (released Jan 16, 2015)

Version 1.12.0 (released Jan 15, 2015)

  • JENKINS-24434 fix trim() usage. (Pull #29)
  • Configurable anonymous access to the list of failure causes. (Pull #31)
  • Having the token expand into some text when no failure cause is identified. (Pull #32)

Version 1.11.0 (released Nov 27, 2014)

  • Failed Tests can be shown as failure causes, but not counted in the statistics (Pull #25)
  • [UI] Added space between indication links. (Pull #26)

Version 1.10.3 (released Oct 13, 2014)

  • One more fix for icons not correctly displayed (Pull #24)

Version 1.10.2 (released Oct 7, 2014)

Version 1.10.1 (released Sep 30, 2014)

Version 1.10.0 (released Sep 19, 2014)

Version 1.9.1 (released Jun 25, 2014)

  • "Failure Scan Options" is not hidden for users without build or configure permissions, it also hides when scanning is turned off.

Version 1.9.0 (released Jun 18, 2014)

  • Statistics upstream link - added upstream link info to statistics.
  • JENKINS-18518 Red highlights not showing when clicking on indications (pull #21)
  • BuildFlow Dependencies And Nested Failure Causes in Gerrit (pull #18)
  • Address multiline "Match Text" failure (pull #17)

Version 1.8.1 (released May 22, 2014)

  • Address NPE in getNotScannedBuilds() (pull #16)

Version 1.8.0 (released May 19, 2014)

  • Project page shows last build failure cause
  • Failure Cause Management link hidden for projects with disabled scanning

Version 1.7.0 (released Apr 1, 2014)

  • Multi line build log indications
  • Optionally store statistics about successful builds

Version 1.6.0 (released Mar 10, 2014)

  • Ability to re-scan non scanned builds (for new installations) and all builds for a project
  • Graphs on projects, slaves and master(s) if using a statistics logging enabled knowledge base (like the MongoDB Knowledge base)
  • Shows failure causes from downstream builds directly on the upstream build page.
  • ListView column showing the failure cause of the last build, if there is one.

Version 1.5.1 (released Nov 19, 2013)

  • Fixed an XSS vulnerability

Version 1.5.0 (released Apr 24, 2013)

New Features
  • The found failure cause is exposed to the REST Api (jobX/1/api).

Version 1.4.1 (released Mar 14, 2013)

Bugs fixed

Version 1.4.0 (released Feb 15, 2013)

New Features
  • Possibility to test regexps on a build log
Bugs fixed
  • Log annotation bugfixes.
  • JENKINS-15948 Build Failure Analyzer icons aren't displayed if Jenkins isn't installed at root context.(again)
  • JENKINS-15926 Build Failure Analyzer with Timestamper output ugly.(again)
  • JENKINS-16596 Repeat/double loggin issue due to Build failure Analyzer.
  • JENKINS-16104 Build Failure Analyzer: Ugly output from plugin.
  • NPE fix when a slave is taken offline during a build.
  • Fix for internal serialization of matrix aggregated indications.
  • Small UI fix in failure cause management page.

Version 1.3.0 (released Dec 06, 2012)

New Features
  • Possibility to test regexp on a line of text when editing BuildLogIndications
  • Output from Build Failure Analyzer shown in normal console.
Bugs fixed
  • JENKINS-15986 Cannot save job configuration pages on Jenkins 1.463 or newer.
  • JENKINS-15948 Build Failure Analyzer icons aren't displayed if Jenkins isn't installed at root context.
  • JENKINS-15926 Build Failure Analyzer with Timestamper output ugly.
Other
  • Updated Gerrit Trigger optional dependency: 2.7.0

Version 1.2.0 (released Nov 22, 2012)

Initial Release

Known issues

Breaks Job configuration pages on Jenkins 1.463 or newer
The problem we have seen is when loading configuration pages. We don't have a registered descriptor for our notifier,
hence, when the hetero-list for the post build actions is generated, an Exception is thrown.

Attachments: