Sunday, April 9, 2017

Maven Lifecycle Phases - Fitting in Code Analysis and Other Tools

The build management portion of Maven operates on a type of Template Pattern. Maven moves from lifecycle-phase to lifecycle-phase until there a step failure or until all steps are complete. The following diagram lists the build lifecycle phases. The orange squares represent the main targets that people run. Every phase is executed starting with Validate until the requested end phase is reached.

For example
  • "mvn validate" runs just the Validate phase.
  • "mvn compile" runs Validate, Initialize, Generate Sources, Process Sources, Generate Resources, Process Resources and Compile.
Each Maven Plugin executes with in a phase. The Surefire unit test plugin, as an example, typically runs the tests in the Test phase.  This means that unit tests don't run if Validation, Compilation, class processing or any of the other preceding phases run with errors.

Maven plugins can execute in their default phase or in any phase of your choosing.  Lifecycle phase selection affects if and when the plugin runs.  You have to decide which mvn command you wish to run the plugin in and which types of failures obviate the need for that plugin.



Code Quality

Teams can use Maven to generate code quality metrics with a variety of plugins. These plugins can be used to fail the build based on minimum quality values.  Failed metrics execution aborts the process at whatever lifecycle phase they run in.  Most code quality Maven plugins have default phases but teams may wish to move them based on how they value different phases. Code quality tools can be run early prior to completing major operations or they can be run late so that some of the standard operations complete/fail prior to getting into code quality analysis.

Static Analysis Phase Selection

Static analysis tools include Checkstyle and PMD.  They can run prior to or after compilation. Some static analysis tools, like Findbugs, run against the compiled binaries. Those should be run after the compilation phase.

When should source code static analysis be run? It is really up to the team based on the priority order for failing a build.
  • It can be run prior to compilation, possibly in the Validate phase.  This lets a build skip a long compilation cycle when the code doesn't meet a minimum bar.  
  • It can be run after compilation, possibly in the Process Classes phase. This lets the analyzer only run against code that is known syntactically correct because the code is known to compile

Code Coverage Phase Selection

Code coverage tools work by instrumenting the source code or binaries prior to test execution. This means you wish to put the code coverage plugin last after other quality plugins, especially if you have generated code, JAXB, GWT or some other. 

Selecting a code coverage maven phase can take some thought.
  • Some teams just bind it to the Test phase. This has the advantage of only running the tests once.  It has the disadvantage of subtly changing the code under tests as the code coverage instrumentation makes minor changes to the test binaries.
  • Some teams bind it to the Verify phase.  This has the advantages of failing the build prior to code coverage generation.  It also has the advantage of executing the unit tests without any binary changes. Code coverage in the Verify phase has the downside of requiring the tests to be run twice, once for the test phase and once for the post-test phase.

Generating Reports in the Site Targets

Metrics are usually generated in the Build lifecycle.  They are usually aggregated into web page reports as part of  mvn site web sitegeneration cycle.  It is almost mandatory that maven metrics reports require two different executions. This means metrics are generated in one pass and then formatted for viewing in another pass:
  • mvn clean install
  • mvn site  

Profiles

Maven profiles let you change the plugins executed based on a profile name.  You can use the default profile for normal developer builds and use some custom profile to execute different plugins.

  • Teams that execute quality metrics as part of all builds don't have a lot of work to do. 
  • Teams that execute policy metrics in only some special builds may wish to create a custom CI policy that only executes as part of the the CI build.

Maven and Gradle 

Gradle has a completely different approach. It eschews build pipelines based on configuration with those based on scripts.  You can find more information about the differences between Gradle and Maven at https://gradle.org/maven-vs-gradle

Sunday, March 26, 2017

Static Analysis from IDE to CI with IntelliJ

Static program analysis is the analysis of computer software that is performed without actually executing programs (analysis performed on executing programs is known as dynamic analysis).[1] In most cases the analysis is performed on some version of the source code, and in the other cases, some form of the object code.[2] 

Static analysis provides a low cost way of automating code review of certain types of source code errors and standards.  Static code analysis, automated tests and code coverage are staples of the Continuous Integration process replacing manual effort with automation.

Full featured IDEs implement their own integrated static analysis and test measurement tools. IntelliJ comes with a comprehensive set of integrated static analysis tools and rules.  It can run the rules in an incremental fashion updating results as code is edited.  Rule violations are immediately reflected in the user interface.

CI servers and IDEs each have their own system from running code analysis, tests and test analysis. There are some times differences in the results generated by IDEs and build servers.  This makes it hard for developers to reliably implement coding standards, pass static analysis and generate expected test results and coverage. IDE code quality extensions, or plugins, can run the CI tools inside the IDE so that developers can troubleshoot inconsistencies and predict results in the CI builds.


The above diagram shows a possible workflow where developers start with the integrated IDE tools. They then use IDE extensions to run the batch/CI automated tools. Finally the project code is then analyzed again by the CI build tools.

Example Static Analysis Plugins within IntelliJ

The next three videos walk through the installation and usage of several 3rd party IntelliJ plugins that let developers run standard Java static analysis and code coverage tools.  We are really interested in
  1. Checkstyle:  A code format and style tool.
  2. PMD:  A static analysis tool
  3. Findbugs : A static analysis tool out of University of Maryland
  4. Clover: A test code coverage tool.
  5. Cobertura:  A test code coverage tool.
  6. Emma:  A test code coverage tool.
The following IntelliJ demos highlight Checkstyle, Findbugs and PMD.

QAPlug

This single unified plugin supports the running Checkstyle, Findbugs and PMD with custom configuration files.  This means can use the same custom rulesets in IntelliJ that they use in a Hudson, Bamboo, TFS or other CI build.  

QAPlug is run via the Analyze context menu available on a right mouse click.  It is configured in the IntelliJ preferences screen and can be connected to one or more IntelliJ profiles

The video on the right walks through installation and usage of the Checkstyle and PMD module for this.


FindBugs - IDEA

This single purpose plugin supports running Findbugs with custom rule files.  It can share the same rule files used in a CI build.  Findbugs-IDEA can be invoked via the "Findbugs" right mouse context menu. 

The video at the right walks through installation and usage of the Findbugs plugin





Checkstyle - IDEA

This single purpose plugin supports running Checkstyle from the right mouse Analyze context menu.  It can share Checkstyles with those used on the CI server.

The video at the right walks through installation and usage of the Checkstyle-IDEA IntelliJ extension. 







Citations

  1. Jump up^ Wichmann, B. A.; Canning, A. A.; Clutterbuck, D. L.; Winsbarrow, L. A.; Ward, N. J.; Marsh, D. W. R. (Mar 1995). "Industrial Perspective on Static Analysis." (PDF)Software Engineering Journal: 69–75. Archived from the original (PDF) on 2011-09-27.
  2. Wikipedia "Static Program Analysis"