Skip to end of metadata
Go to start of metadata

Attention

Please note that this wiki page sometimes does not show the changed content. In order to see the latest results you may need to add the ?cache attribute to the URL.

Up to now the static analysis plug-ins are optimized for freestyle builds:

  1. Configuration is done using a user interface, i.e. all possible options are visible and could be easily selected or edited using UI controls.
  2. Reporting of the static analysis results has been provided as a black box, i.e. users need not to hassle about intermediate steps, they just see the final result.

A user friendly approach for editing pipelines is not yet available, at least not for complex use cases: So pipelines with static analysis steps suffer from the same problems. This is something the Jenkins core team needs to fix, hopefully there will be some progress in the future. I don't think that a plug-in can provide a solution here.

The second point, however, can be addressed by the plug-in. Rather than providing all steps in a black box, these steps should be made accessible for pipelines. In the following sections these steps are described in detail. Moreover, all relevant issues are listed, so all requirements should be visible from here. If you have additional requirements, or if you need something different, please use this wiki page as discussion platform.

Structure of static analysis steps

In the following sections, the current structure is explained. Then, a first sketch of the new implementation for pipelines is given.

Current implementation 

All static analysis plug-ins basically are composed of several similar steps. In a freestyle job you can configure these steps using one configuration object. In a freestyle build this configuration is the input for a post build publisher. This publisher parses a set of files (or the console log) for warnings and attaches the recorded warnings as an action to the build of the job. This action is visualized afterwards from Jenkins' UI. There you can view the warnings or the trend report. Additionally, you can also set the build status and health depending on the set of recorded warnings.

Pipeline friendly implementation

For pipelines, this approach has several drawbacks, see the issues below. In order to provide a better experience for pipelines, the plugin needs to divide the big step into two sub-steps. In the next sections, a sketch of the planned functionality is given. In these sections, the word issue is used as placeholder for warnings, bugs, open tasks, duplications, etc. 

Note: if not described otherwise all steps can be called several times.

scanForIssues

Scans a set of files or the console log for issues. All parsers from the warnings plugin as well as all parsers or scanners from the other static analysis plugin can be used in this step. The result is an unfiltered set of issues. Depending on the selected parsers, the associated filenames might be still relative.

Input:

Output

  • a report that contains a set of unique issues

TODO (After Release 2):

  • JENKINS-44450: How to handle scanning the console log for only a subset of the pipeline (including just one branch of a parallel pipeline)? Either this step needs to support block-scoped usage (taking a closure that specifies the steps to include in the console log scan), or pipeline should provide a separate (possibly more generic) step to capture the console log for a group of steps that could then be passed to this step.
publishIssues

Makes the issues visible in a run: creates an associated action that will be persisted by Jenkins. Responsible for detail views and trend graphs and so on. Merges all issues of the provided parsers into a single result. Copies all conflicting files from a build agent to the master so that these files are available for rendering of the warnings details.

Input: 

  • 1, ..., n sets of issues: each set contains the issues from one scanForIssues calls.
  • strategy to pick the reference build (that is used as a baseline to compute the new issues): currently the freestyle options usePreviousBuildAsReference and useStableBuildAsReference are available. I'm not sure if it must be possible to provide the reference build as parameter or if additional strategies must be provided or implemented.
  • defaultEncoding: Encoding to use when reading input files.
  • thresholds for health reporting: healthy, unhealthy, minimumPriority (see freestyle project).
  • thresholds for run status evaluation: unstableAll, etc. (see freestyle project).
  • include or exclude filters (filename, type, package, etc.) to specify the set of issues that should be stored  
  • id: the ID of the result. If not provided, then the ID of the parser in the scanForIssues step is used. This ID is used as URL to the results and to get the correct labels for the user interface.
  • name: the name of the result. If not provided, then the name of the parser in the scanForIssues step is used. This name is used in all labels for the user interface.
  • See implementation PublishIssuesStep

Output

  • the created action object (which contains a reference to the created result)
  • side effect: current run has a new action attached.

TODO (After Release 2): 

  • Hierarchy of results: Would it make sense to have hierarchy in the warnings container object so that the necessary warnings can all be added to the same container, but different groups are distinguishable from each other? Then only a few steps would need to be aware of this hierarchy, others could just go through all items. Or could this publishing step create such a hierarchy in the web UI?
  • Blame: Compute the authors and commits for a set of issues (this works for freestyle jobs, it is not clear yet if the required information is available in pipelines).

Prototype 

The development of this feature still is in progress. The implementation is visible in the 5.0 branch of the warnings plug-in. The base plugin analysis-core has not been changed, the latest version 1.95 will be used as a dependency (in the long term, this dependency will be removed). The other static analysis plugins (checkstyle, pmd, dry, findbugs, android-lint, ccm) are obsolete now: their parsers habe been integrated into the warnings plug-in, so these plugins can be removed if all jobs have been migrated. Support for the analysis-collector plugin will be added later on. (A CI build is already available: warnings plug-in.) 

The prototype works quite well, most of the requirements from the issues below will be satisfied. Below is an example script that shows the new feature in action.

Pipeline
node {
    stage ('Checkout') {
        git branch:'master', url: 'file:///Users/hafner/Development/git/analysis-model'
    }

    stage ('Build') {
        def mvnHome = tool 'mvn-default'

        sh "${mvnHome}/bin/mvn --batch-mode -V -U -e clean test -Dsurefire.useFile=false"

        junit testResults: '**/target/surefire-reports/TEST-*.xml'

        def java = scanForIssues tool: [$class: 'Java']
        def javadoc = scanForIssues tool: [$class: 'JavaDoc']
        
        publishIssues issues:[java]
        publishIssues issues:[javadoc]
    }

    stage ('Analysis') {
        def mvnHome = tool 'mvn-default'

        sh "${mvnHome}/bin/mvn -batch-mode -V -U -e checkstyle:checkstyle pmd:pmd pmd:cpd findbugs:findbugs spotbugs:spotbugs"

        def checkstyle = scanForIssues tool: [$class: 'CheckStyle'], pattern: '**/target/checkstyle-result.xml'
        publishIssues issues:[checkstyle]
   
        def pmd = scanForIssues tool: [$class: 'Pmd'], pattern: '**/target/pmd.xml'
        publishIssues issues:[pmd]
        
        def cpd = scanForIssues tool: [$class: 'Cpd'], pattern: '**/target/cpd.xml'
        publishIssues issues:[cpd]
        
        def findbugs = scanForIssues tool: [$class: 'FindBugs'], pattern: '**/target/findbugsXml.xml'
        publishIssues issues:[findbugs]

        def spotbugs = scanForIssues tool: [$class: 'SpotBugs'], pattern: '**/target/spotbugsXml.xml'
        publishIssues issues:[spotbugs]
    }
}

Related Issues

The following issues are related. Note that not all of them are from pipeline jobs, some will also benefit from the splitting shown above.

JENKINS-44450 - Getting issue details... STATUS

JENKINS-43155 - Getting issue details... STATUS

JENKINS-40164 - Getting issue details... STATUS

JENKINS-37325 - Getting issue details... STATUS

JENKINS-31812 - Getting issue details... STATUS

JENKINS-31633 - Getting issue details... STATUS

JENKINS-30551 - Getting issue details... STATUS

JENKINS-22526 - Getting issue details... STATUS

JENKINS-22874 - Getting issue details... STATUS

JENKINS-18708 - Getting issue details... STATUS

JENKINS-17196 - Getting issue details... STATUS

JENKINS-13056 - Getting issue details... STATUS

Non Functional API Requirements

These changes will change the API of the analysis-core plug-in. Since I do not have the time to make such big changes backward compatible, a new major release of analysis-core is required:

  • All changes will be part of analysis-core 2.0 release
  • analysis-core 2.0 API will be not backward compatible
  • analysis-core 2.0 will be based on Java 8
  • analysis-core 2.0 will not contain a dependency to the maven project
  • analysis-core 2.0 will remove all @Deprecated methods
  • No labels

65 Comments

  1. looking like a good approach.

    Some questions regarding the "computeWarningsHistory":

    1. which step takes the  "warnings difference object" as input? the createBuildAction, changeBuildHealth, changeBuildStatus?
    2. a "warnings difference object" could perhaps also contain differences of two "set of warnings" from the same build? e.g. to compare outputs of different compiler versions or platform builds. in that case a more general interface could be
      1. getWarningsOfBuild(num)  / getWarningOfReferenceBuild()
      2. compareWarnings(set a, set b) → "warning difference object"
      3. publishWarningDifference[...] / createAction[...] ("warning difference object")
        1. as "history" 
        2. as "diff"

     

      1. which step takes the  "warnings difference object" as input? the createBuildAction, changeBuildHealth, changeBuildStatus?

      Thanks, I fixed this now.

      a "warnings difference object" could perhaps also contain differences of two "set of warnings" from the same build? e.g. to compare outputs of different compiler versions or platform builds. in that case a more general interface could be

      Hmm, but this would have a different semantic. Currently we have new, old and fixed warnings (for the same warning).

      What should be the output in your example? Or what properties should be compared?

       

      1. I guess in that case only names would be different...
        new > warnings in "set b" only

        old > warnings found in both sets

        fixed > warnings in "set a" only.

        So the "naming" might differ only in the view published in the last step. "as history" would use the current naming, while during "as diff" either a name of the compared sets is used or the user can define the three by just by himself...

        But don't worry too much I don't have a current use case for that, it was just an idea for an even more generic approach....

        1. With the current algorithm, old will be empty, since the warnings properties are different for two different parsers. 

  2. How would getDefaultReferenceBuild choose the reference build?

    Could we specify a reference build from another job?


    For example:

    Using the github plugin we build the master branch.

    I then create a feature and then create a pull request → which results in Jenkins building a PR branch.

    I'd like to be able to compare the pull request job warnings with the warnings from the last build of the master branch job (and fail the PR build if it adds new warnings).

    1. This would be a very useful ability, so that we can avoid false output for PR requests which don't happen to fix a warning in master, but don't add any new warnings. I would very much like to be able to do this kind of check.

    2. I think it makes sense to let getDefaultReferenceBuild return the pervious build with results for the same parser. This is the current behavior and should not change.

      However, you can pass in any build you like. The only requirement: there must be a result for the same parser, otherwise all warnings will be new. Do you need anything from the plugin to choose this build? I thought you can compute this build on your own (if the default one does not work for you).  

      1. "getDefaultReferenceBuild return the pervious build with results for the same parser. This is the current behavior and should not change." - I agree. Currently we can choose to use the last build, or the last stable build as a reference. How would that be specified?

        "Do you need anything from the plugin to choose this build?" → No, I think you are right, it shouldn't be the responsibility of the warnings parser to choose builds.

        How would I get the warnings container for the build I want to compare is my question i guess. I want to do something like this psudo code:

        def warnings = scanForWarnings("**/result.xml""MSBuild")
        def latestStableMasterBuildWarnings = getWarningsOfBuild(JOBNAME)
        computeWarningsHistory(warnings, latestStableMasterBuildWarnings)

        etc etc...

         

        I think I may just not have fully grasped how this will fit together. Are you able to post some psudo code of how you envisage the getDefaultReferenceBuild working, as well as computeWarningsHistory? That would be very helpful for my understanding.

      2. I agree that analysis/warnings ideally shouldn't really be involved with getting a build, but I suspect that you can't really get the correct build directly from pipeline in unprivileged pipelines. I'm not really sure what all you can access, but I suspect you'd have to access the Jenkins object or rawBuild object directly to get the information we need, which is not allowed in sandboxed scripts.

        1. I updated the pseudo code. 

          If you cannot determine the build, are there some other algorithms that the plugin can provide?

          1. Thanks the updated pseudo code makes it much clearer in my mind! Looks good to me. I can't think of any additional methods that would be needed off the top of my head.

            1. Pete Whitehead do you have a concrete idea how to get the reference build using git/github? We had a discussion on JENKINS-13056 about that and as far as I know it is not that simple to get this information from SCM (git). I think we need to find the build that built the "nearest parent commit".

              1. No i haven't a clue...yet. I Just assumed it would be possible, either via the git commandline, or possibly just via naming convention I could infer the Jenkins job name and load the log files of the 'master' job from archived artifacts, then re-process the warnings using these new commands.

                1. If there is something I can do in my plugin let me know. E.g., using the latest build of a different project, or a build with a specific git commit (there is already a dependency to git so you can write a small class that computes the reference build based on some heuristics.). 

  3. On another note, your psudo code has 'createBuildAction', but i don't see that defined anywhere on this page. Is that a generic Jenkins pipeline thing? I'm not familiar with it

    1. I don't think it's a generic pipeline thing at all. I think it's because warnings stores results as a buildAction object in the build. I think this should be named something more intuitive like "publishWarnings", or something similar, since this is what causes the warnings to show up in the build results.

  4. Another interesting question: when you are parsing checkstyle, findbugs, pmd, dry warnings, or open tasks, would you like to have these also integrated into the scanForWarnings step as the other warnings parsers? Or is there a different step required?

  5. I added another pseudo code part that now uses a minimal set of steps, would such an approach be more user friendly? 

  6. I'm a little biased, but I'd like this to be a bit more Declarative friendly - i.e., less composing of steps by passing variables around, more single steps that can be called and meaningful on their own. It'd be great if we're still able to do just a simple checkstyle(...) etc call.

    Also,  JENKINS-27395 - Getting issue details... STATUS  is similar to  JENKINS-37325 - Getting issue details... STATUS  and friends - I'd recommend waiting on that until we've figured out how we want to handle test/report results per step/stage more generally. And then there's  JENKINS-43995 - Getting issue details... STATUS , which I'm working on as well. =)

    1. I think both styles are valid, which is the difficult part. For example, I want the ability to obtain the warnings results in my pipeline without setting the overall build status, so that I can use that information to programatically change filenames.

      1. Completely understood and agreed! I just want to make sure we still have access to the simple step behavior, even if it's just a wrapper for calling the same underlying classes/methods as the more granular steps put together.

        1. The simpler step will still be available (the freestyle build as well). There are just several use cases where this step does not work well. 

          How do steps typically communicate in a declarative pipeline? E.g., a step to scan for findbugs, a step to scan for checkstyle, and an aggregating step to publish the result in Jenkins (as an action). Or is the communication supposed to work in the same manner as in freestyle jobs: the first step attaches a result to the build and the next step reads this result and acts accordingly?

  7. As per the following JIRA, I would love to be able to configure the defaults of the warnings plugin entirely as code:  JENKINS-46095 - Getting issue details... STATUS

  8. Hi Ulli, 

    according to JIRA  JENKINS-47200 - Getting issue details... STATUS  we get many links "OPEN TASKS" generated in the dashboard of a build.

    I use pipeline "withMaven" step that generates all junit, findbugs and taskScanner reports.

    TaskScanner should only generate one link on the left side of the build.

    cf all the generated links in the screenshot.

  9. Is there any way to access and use the functionality described here? As far as I can tell this is all locked up on the 2.0 branch in Github. All of this sounds really nice, and great to use. I would really like to test it out.

    I currently have a pipeline that I can't fully complete because I need to report on both a javascript ESlint result, and a Flow result. The Flow I can convert to Checkstyle XML, but that means that I have to use the current `warnings` function with a JSLint parser for both of them. This results in a report that isn't much use because it keeps thinking that errors are fixed because it processes twice. The resulting report also seems to not render. Nevermind that two completely different types of results would now show up in the same report. 

    It would be great if there were compiled nightly releases or something that could be downloaded from github or something.

    1. Actually, the code still is alpha (and now in branch 3.0 since the other branch destroyed the backward compatibility for freestyle builds, i.e. I started again from the beginning). 

      It still needs a lot of work until I can provide a beta. 

  10. What's the current status? I still know of several companies that need this to make the move from an old "multijob" setup to actual pipelines.

    1. I've just tried out your changes so far, based on the 3.0 branches of both analysis-core and the warnings plug-in.
      After I figured out I had to define an id when calling publishIssues with the same tool for the second time, it kind of worked.

      However, I would like for the build to fail as soon as I call publishIssues and a threshold is exceeded (perhaps by setting an additional parameter, for example failFast, to true).
      In general we'd like the build to fail as fast as possible if quality checks are not passed.

      1. I'm not sure if I understand the problem correctly: you want to skip the second publish step (or scan?) if the first publish set the build to a failure? I can add such a flag, I thought that would be simpler to be coded in the pipeline outside of the plugin (by guarding the steps with an if).

        1. Just noticed we are missing some other functionality that we have use in our freestyle jobs. We calculate the delta on the number of warnings compared to the last successful build. This way we make sure that the number of warnings may only be reduced. This is kind of necessary in this particular use case because we do have a lot of legacy code and most of the time we do not want to touch it too much (because it has been running in the field/production for a long time, hence its stability is proven). But if we have to edit it, we don't want more (and probably new) warnings to be introduced.

          Currently we have a multijob with freestyle jobs for each of our "stages", which works fine because of the isolation between the jobs. The goal is to transform this into a declarative pipeline, where the separate jobs would thus be separate stages.
          At the end of each given stage we should decide if quality checks are met, if not, there is no point in executing the rest of the pipeline, because the code will never reach production.

          1. So your stages are using a different SCM baseline? This is kind of unusual. That means you need to compare results within a single run. This should be possible by resolving and comparing the corresponding results by code.  

            1. No, in the current setup, the workspace is shared amongst the different jobs. We do not want to compare within a single pipeline. We want to compare the current number of found warnings with the number of warnings found in a previous run of the same pipeline within the same "category" of warnings. The total number may not increase.

              1. This is already supported: 

                publishIssues issues:[your-issues-of-current-run], unstableNewAll:1, ignoreAnalysisResult=true, overallResultMustBeSuccess=true

                It will compare with the results with the same ID from the reference build. The reference build can be adjusted with the two parameters (see above).

                1. Thanks. But does publishIssues also give me feedback (like a return value for instance) that I can use to determine if the build result is going to be "failed"?

                  1. Yes, the step returns a ResultAction.

      2. The beauty if pipeline is that you can code this by looking at the threshhold yourself and handling the flow outside of the plugins.


        That's sort of the point of the updated code for scanning and publishing issues, is to avoid having a ton of slightly different parameters to enable or disable behaviors. Instead, let the pipeline scripts define their behavior.

  11. Hi,

    I build an HPI from Github under branch 3.0 - However when I try to run the following pipeline

    node {
     stage('Preparation') { // for display purposes
    git 'https://github.com/jglick/simple-maven-project-with-tests.git'
    }
    stage('Build') {
    def java = scanForIssues tool: [$class: 'Java']
    }
    }

    However when I am running the job It doesn't recognize the scanForIssues step

     

    java.lang.NoSuchMethodError: No such DSL method 'scanForIssues' found among steps [ansiColor, archive, bat, build, catchError, checkout, container, containerLog, deleteDir,

    What am I missing ?

    1. Did you also install analysis-core?

      You need to install both plug-ins to get the new suite running. (A CI build is already available for the analysis-core and warnings plug-in.)

      1. Thanks Ulli for your quick answer.

        I did manage to progress, I performs a checkstyle on a nodejs code. In small code repositories I managed to get even the publish part,

        However, I keep getting the following errors in the log. (The publishIssues hits the same issue)

        [Pipeline] scanForIssues
        13:29:03 [CheckStyle] Searching for all files in '/home/jenkins/workspace/monitor_master_dev-6G74EQSJPLUUCZUYGIF5A3SRCAXWEFUPUYZ3MGXOYFEVJGFWBB4Q' that match the pattern 'coverage/eslint.xml'
        13:29:03 [CheckStyle] -> found 1 file
        13:29:03 [CheckStyle] Successfully parsed file /home/jenkins/workspace/monitor_master_dev-6G74EQSJPLUUCZUYGIF5A3SRCAXWEFUPUYZ3MGXOYFEVJGFWBB4Q/coverage/eslint.xml: found 42 issues (skipped 0 duplicates)
        13:29:03 [CheckStyle] Resolving absolute file names for all issues
        13:29:03 [CheckStyle] Resolved absolute paths for 1 files (Issues 42 resolved, 0 unresolved, 0 already absolute)
        13:29:03 [CheckStyle] Resolving module names from module definitions (build.xml, pom.xml, or Manifest.mf files)
        13:29:03 [CheckStyle] Resolved module names for 42 issues
        13:29:03 [CheckStyle] Using encoding 'UTF-8' to resolve package names (or namespaces)
        13:29:03 [CheckStyle] Resolved package names of 1 affected files
        13:29:03 [CheckStyle] Using encoding 'UTF-8' to read source files
        13:29:03 [CheckStyle] [ERROR] Can't read file '/home/jenkins/workspace/monitor_master_dev-6G74EQSJPLUUCZUYGIF5A3SRCAXWEFUPUYZ3MGXOYFEVJGFWBB4Q/nodejs/monitor.js': java.nio.file.NoSuchFileException: /home/jenkins/workspace/monitor_master_dev-6G74EQSJPLUUCZUYGIF5A3SRCAXWEFUPUYZ3MGXOYFEVJGFWBB4Q/nodejs/monitor.js

        The build was successful, but the above error appears for each warning both on scanForIssues and on the publishIssues, for each scanned file.

        I am running the following configuration

        Jenkins 2.113 (over K8S). I am using both warnings and analytics core as you suggested.

        I validated the permissions of the file which are 

        -rw-r--r--    1 jenkins    jenkins monitor.js
         

        I tried to run inside of a container and outside, but no success. I even changed the permissions to 0777, and got the same result, any idea ?

         

        1. Is the filename correct? Ist the build on a slave?

          1. Yes, the filename is correct and the build is on a slave (a pod that I spawn for the purpose of this build)

            1. Ok, then there is a bug. I haven't yet tested the plugin with a master slave configuration. Can you please file a bug report in JIRA for the analysis-core Plugin?

              1. Hi Ulli, I opened a bug JENKINS-50926 - Getting issue details... STATUS  please update me if there is any progress,

                Thanks

      2. How do you install these?  Through the advanced tab in Jenkins?  

        1. You should be able to grab the hpi files from the CI build and install them through the advanced tab, yes.

  12. Hi,

    Is the Task Scanner plugin not updated to work with this implementation? I may be missing something but I cannot find a 3.0 branch or any classes which extend StaticAnalysisTool.

    Thanks

    1. Not yet. I'm still not sure if the functionality should be merged into the warnings plugin of if it should be part of a standalone tasks plug-in. Scanning for open tasks is somewhat different, does it make sense to be available as an option in the select box of the static analysis tools? What do you think? 

      1. Ah, I see. I guess I don't have a strong opinion on that but it seems like it would be convenient to have open tasks functionality in warnings.

        So I am trying to do two things: the first is to graph the results checkstyle, warnings (for Java compiler) and open tasks plugins in one graph, similar to the first picture attached. Previously it was possible to do something like this with the interface in the second picture. It seems as if publishIssues does not distinguish issues by type, so is there any way to do this? I understand if the open tasks cannot be included because it's not updated.

        The second thing I am trying to do is reference a different (not just the previous) build to get the difference in warnings created, and to use that difference to make decisions in the code in the Jenkinsfile. Is this possible?

        Thank you!

         

         

        1. This kind of graph is not yet implemented. (The plan is to create one after the 2.0 release. The data is available, but the graph visualization still needs to be done. I don't want to build the graph with the old server side PNG image API, I rather prefer a JS client side chart. This is currently blocked by JENKINS-49319.) 

          The reference build can be selected using three options in the reference build section. Which kind of build would you like to select? A fixed build number? 

          1. Thanks for the response - I would like to reference the most recent successful build on a specific branch (master).

            1. This should work in the new release. I think I will add a new test case for the multi-branch plugin to verify this behavior. 

  13. I am getting an error while trying to set a pipeline step in blueocean. What would be the correct input for issues field ?


    1. The issues variable is the return value (actually an Report instance) of a previous scanForIssues step. You can't use that step in declarative pipelines, here you need the recordIssues step that combines scanning and publishing in one step (see examples). 

      Note that using the plugin in blue ocean will not help much since there is no visualization available yet.

  14. Hello Dear Ulli Hafner,

    Save functionality is not working for setting reference builds on free-style projects.

    I downloaded and installed latest builds from https://ci.jenkins.io/job/Plugins/job/analysis-core-plugin/job/3.0/ and https://ci.jenkins.io/job/Plugins/job/warnings-plugin/job/3.0/

    1. Well, everything is not yet ready to be released (wink) Can you please create a bug report (with property EPIC set to JENKINS-49911)

      1. Thank you for the quick fix but I'm afraid now looks like it's not possible to set quality gate thresholds. After apply & save threshold settings are disappearing again and all became undefined. I think it was working fine before the fix.





        1. I wonder if this is related. Seems that this worked never. (The fields are correctly saved though, just showing the values again does not work).

          1. Ah, now I get it. Thank you for pointing this out. I was setting value to "0" but setting it to "1" works but that brings another issue:

             

            The reference build has 919 warnings and previously publishing pmd report has a feature called delta warnings. Is it possible to set new warnings to not exceed total warning count from reference build ? For example currently setting new warning threshold to "2" will fail for the screenshot above. What I'm looking for checking new warnings based on total warning counts.

            1. I did not implement that part in the new model. (In the next couple of months I will try to improve the detection of new warnings. This should make the old workaround 'delta warnings' hopefully obsolete.)

              1. I guess either very few people are working on legacy code with feature branching or they have custom workarounds for not increasing warning counts.

                Is there a way to get total count of warnings from a reference build ? May be it's not a proper way to do it but just a suggestion. Would it be too hard to fail a build if total warning count increased comparing to reference build instead of using a complicated algorithm ?

                Considering that this is an open source project it's perfectly understandable that it could take months to implement such complicated detection but until that point a simple workaround could help many others (smile)

                1. I guess either very few people are working on legacy code with feature branching or they have custom workarounds for not increasing warning counts.

                  I'm not sure if I get the point. If the new warning detection would work correctly (without false positives), then in your build above you would have just 2 new warnings. 10 warnings should have been filtered as not new. This is unrelated to legacy code. One thing that would help me is to investigate why there are 10 warnings that are new. You can check on your own and open the pmd-new-issues.xml file in your build folder in both builds and check what the value of contextHashCode is for these 10 warnings.

                  I think it would make sense to add an option to the configuration, that a build should be failed or set to unstable if the total number of warnings increases. (I can remove that option if the new warnings detection works better). Can you please create an issue in Jira? I think that I can add this small feature if testing of the new release has been finished.



                  1. JENKINS-52098 - Getting issue details... STATUS  created a new issue and sent you an e-mail with detailed reports from our codebase checks. Please let me know if anything else I could do.

                    Regarding to where this threshold feature can make sense even with better new warnings algorithm without false positives. In (may be very) rare cases, adding few new warnings can be considered OK if developer fixed 12 warnings and added 2 new. As long as it's an improvement towards better and to less warning counts.


          1. Vielen dank! Herr Hafner. 

            Thanks a lot! Mr. Hafner.