December 10, 2009

Maven Plugins: Upgrade with Care!

Upgrading Maven Plugins: Tips and Issues

After having shown the list of current Maven plugin versions in my previous post, now I'm going to share my experiences with upgrading. Just like expected, some of the new plugins did not work out of the box or required some changes in configuration:

maven-checkstyle-plugin

We have used version 2.3 previously which is based on Checkstyle 4.4. In contrast, plugin version 2.4 is eventually built on top of Checkstyle 5 which better supports Java 5 language features (I wrote a post on that issue). The configuration is not fully compatible, so you would have to upgrade it.

maven-javadoc-plugin

Starting with version 2.6, JavaDoc plugin can detect the Java API link for the current build. This did not work for us (probably due to missing proxy configuration), so we had to switch it off by setting detectJavaApiLink property to false. See plugin site.

taglist-maven-plugin

The configuration of tags has changed and is now a bit more extensive. Old format (using tags element) is still supported, but deprecated. See plugin documentation.

findbugs-maven-plugin

I have used version 2.2 which was the most recent one when changing the POM. However, this yielded some strange errors during site generation. This is the strack trace:

Generating "FindBugs Report" report.
Plugin Artifacts to be added ->...
AuxClasspath is ->D:\Profiles\Default User\.m2\repository\org\apache\maven\reporting\maven-reporting-impl\2.0\maven-reporting-impl-2.0.jar;...
[java] Exception in thread "main" java.io.FileNotFoundException: D:\builds\...\User\.m2\repository\...\maven-reporting-impl-2.0.jar;D:\Profiles\Default (The filename, directory name, or volume label syntax is incorrect)
[java] at java.util.zip.ZipFile.open(Native Method)
[java] at java.util.zip.ZipFile.<init>(ZipFile.java:114)
[java] at java.util.zip.ZipFile.<init>(ZipFile.java:131)
[java] at edu.umd.cs.findbugs.classfile.impl.ZipFileCodeBase.<init>(ZipFileCodeBase.java:53)
[java] at edu.umd.cs.findbugs.classfile.impl.ZipCodeBaseFactory.countUsingZipFile(ZipCodeBaseFactory.java:92)
[java] at edu.umd.cs.findbugs.classfile.impl.ZipCodeBaseFactory.makeZipCodeBase(ZipCodeBaseFactory.java:46)
[java] at edu.umd.cs.findbugs.classfile.impl.ClassFactory.createFilesystemCodeBase(ClassFactory.java:97)
[java] at edu.umd.cs.findbugs.classfile.impl.FilesystemCodeBaseLocator.openCodeBase(FilesystemCodeBaseLocator.java:75)
[java] at edu.umd.cs.findbugs.classfile.impl.ClassPathBuilder.processWorkList(ClassPathBuilder.java:564)
[java] at edu.umd.cs.findbugs.classfile.impl.ClassPathBuilder.build(ClassPathBuilder.java:195)
[java] at edu.umd.cs.findbugs.FindBugs2.buildClassPath(FindBugs2.java:584)
[java] at edu.umd.cs.findbugs.FindBugs2.execute(FindBugs2.java:181)
[java] at edu.umd.cs.findbugs.FindBugs.runMain(FindBugs.java:348)
[java] at edu.umd.cs.findbugs.FindBugs2.main(FindBugs2.java:1057)
[java] Java Result: 1

As you can see, we are using Windows default for local Maven repository (D:\Profiles\Default User\.m2) which obviously is causing a problem with the path later on. How sick is that?!?

Then, after having tried this and that, I discovered there is a brand new version 2.3 available so I tested that one and guess what – everything works fine again! Hence, don't use version 2.2, it seems to be broken...

docbkx-maven-plugin

The latest version of that plugin is 2.0.9, but that did not work correctly. It failed the build with this error:

ValidationException: null:30:723: Error(30/723): fo:table-body is missing child elements.
Required Content Model: marker* (table-row+|table-cell+)

The given line/position information did not match to anything suspicious, and I could not see anything wrong in our XSL files. So, no idea what this issue is telling me or what could I do about it. That's why I rolled back to 2.0.8 which just works fine.

The Bottom Line

With the given plugin versions, we are up to date again and managed to get rid of some issues hitting us since quite some time. Additionally, we are well prepared for upgrading to Maven 3. I hope I will be able to do so soon, to check if this is really a "drop-in replacement"... I will let you know!

Maven Plugins: Current Versions

Upgrading Maven Plugins

In preparation for a later switch to Maven 3 (which is already knocking on the door) as well as to get rid of some plugin related issues we are suffering from, I decided to update the Maven plugins we use for build and site generation.

Of course, we are following best practice and are locking down the version of all plugins in project.build.pluginManagment.plugins section. This is done in the company's topmost POM, so that all company projects would use the same versions once they reference the latest parent POM.

As you might know, upgrading to new plugin versions is always an adventure and you have to test your builds seriously, which is of course hard when you are going to change the company settings...

Build Plugins

Well, here is the list of build plugins we now use with their current version, as well as (in brackets) the version that Maven 3.0-alpha5 defines in its internal POM. I have highlighted where both versions differ:

  • maven-archetype-plugin: 2.0-alpha-5 (see comment below)
  • maven-assembly-plugin: 2.2-beta-4 (2.2-beta-4)
  • maven-clean-plugin: 2.3 (2.3)
  • maven-compiler-plugin: 2.0.2 (2.0.2)
  • maven-dependency-plugin: 2.1 (2.0)
  • maven-deploy-plugin: 2.4 (2.4)
  • maven-ear-plugin: 2.4 (2.3.1)
  • maven-ejb-plugin: 2.2 (2.1)
  • maven-enforcer-plugin: 1.0-beta-1
  • maven-help-plugin: 2.1 (2.1)
  • maven-install-plugin: 2.3 (2.3)
  • maven-javadoc-plugin: 2.6.1 (2.5)
  • maven-jar-plugin: 2.3 (2.2)
  • maven-release-plugin: 2.0-beta-9 (2.0-beta-9)
  • maven-resources-plugin: 2.4.1 (2.4.1)
  • maven-site-plugin: 2.0.1 (2.0.1)
  • maven-source-plugin: 2.1.1 (2.0.4)
  • maven-surefire-plugin: 2.4.3 (2.4.3)
  • maven-war-plugin: 2.1-beta-1 (2.1-alpha-1)
  • build-helper-maven-plugin: 1.4
  • failsafe-maven-plugin: 2.4.3-alpha-1
  • cargo-maven2-plugin: 1.0
  • docbkx-maven-plugin: 2.0.8

It's a bit strange that Maven 3.0-alpha5 (which came out end of November) does not use the latest version of all those plugins, most of them having been released before that date. I don't know if this was intentional or not... Let's hope it's not because of unsure quality of latest plugin versions ;-) Anyways, I decided to upgrade to the latest available version for all plugins.

Reporting Plugins

Here's the list for plugins related to site reports:

  • cobertura-maven-plugin: 2.3
  • findbugs-maven-plugin: 2.3
  • jdepend-maven-plugin: 2.0-beta-2
  • maven-checkstyle-plugin: 2.4
  • maven-jxr-plugin: 2.1
  • maven-pmd-plugin: 2.4
  • maven-project-info-reports-plugin: 2.1.2
  • maven-surefire-report-plugin: 2.4.3
  • taglist-maven-plugin: 2.4

I'm going to show some tips and issues when upgrading to these versions in an upcoming post...

December 2, 2009

Unit and Integration Testing with Maven, Part 2

Welome Back...

... to the second installment of this little series. After having seen the requirements and hassles when using Maven for testing in the last post, we are now going to list the possible solutions.

Remember, Maven does support different phases for unit and integration tests, but there is only one source directory (usually src/test/java), making it a bit difficult to setup and organize test environments. Hence, we have to use some way to separate both types of test.

Option 1: Separate Module for Integration Tests

In case you are using modules for your project anyways and want to have integration tests which are testing these modules in integration, this is the natural solution: just create another module that depends on the other ones and only contains the integration test sources and resources.

However, if you instead want to integration-test each module individually, this approach just doubles the number of modules, which can be difficult to manage. You could put all integration tests into one big module – but that has other disadvantages, of course.

Since we do not have any sources (only test sources) in that integration-test project, the usual build lifecycle would execute a lot of unneeded steps like copying resources, compiling, testing, creating JAR file etc. We could use the POM packaging type to suppress all of this, but then we have to configure the maven-compiler-plugin to force compilation of test sources.

Independantly of packaging type, we have to do some more configurations:

  • Execute Surefire plugin in integration-test phase.
  • Prepare integration tests (for instance, start the container and deploy the application) in pre-integration-test phase. This is quite easy with Cargo Maven plugin.
  • Shutdown integration tests (stop the container) in post-integration-test phase.

Here is the relevant part of such a POM file:

<build>
<plugins>

<!-- *** Compiler plugin: we must force test compile because we're using a
pom packaging that doesn't have this lifecycle mapping. -->
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<executions>
<execution>
<goals>
<goal>testCompile</goal>
</goals>
</execution>
</executions>
</plugin>

<!-- *** Surefire plugin: run integration tests *** -->
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<executions>
<execution>
<phase>integration-test</phase>
<goals>
<goal>test</goal>
</goals>
</execution>
</executions>
</plugin>

<!-- *** Cargo plugin: start/stop application server and deploy the ear
file before/after integration tests *** -->
<plugin>
<groupId>org.codehaus.cargo</groupId>
<artifactId>cargo-maven2-plugin</artifactId>
<version>1.0</version>
<configuration>
...
</configuration>

<executions>
<!-- before integration tests are run: start server -->
<execution>
<id>start-container</id>
<phase>pre-integration-test</phase>
<goals>
<goal>start</goal>
</goals>
</execution>
<!-- after integration tests are run: stop server -->
<execution>
<id>stop-container</id>
<phase>post-integration-test</phase>
<goals>
<goal>stop</goal>
</goals>
</execution>
</executions>
</plugin>

</plugins>
</build>

Option 2: Different Source Directories

In this scenario, unit and integration test sources are placed in separate source directories, like src/test/java and src/integrationtest/java. I think this would definitely be the best solution, and it actually should be what Maven supports out of the box. Sadly, Maven does not, and as far as I know Maven 3 won't either :-(

Well, we should be able to configure things this way. For compiling and executing integration tests, you would have to configure Compiler plugin to compile the integration test sources in pre-integration-test phase (using the includes parameter), and then configure a second Surefire execution using testClassesDirectory parameter to point it to the integration test folder.

However, this option seems a bit fragile to me. Even if it works (I haven't checked), the integration test folder would probably not show up in Eclipse when using m2eclipse plugin and other plugins may have issues with this additional test source path as well. That's why I do not recommend this option.

Option 3: Different File Name Patterns

In this scenario, the package or file name is used to distinguish between unit and integration test source files. Let's say, for instance, that all integration test classes start with IT* and hence the filename pattern is **/IT*.java. We'd have to configure the Surefire plugin to execute twice: once in test phase for executing the unit tests only, and another time in integration-test phase to execute, well, the integration tests (and only those)

A common way to do so is this:

  • In configuration of Surefire plugin, set skip parameter to true to bypass all tests.
  • Add an execution element for unit tests, where skip is set to false and exclude parameter is used to exclude the integration tests.
  • Add another execution element for integration tests, where skip is set to false and include parameter is used to just include the integration tests and nothing else.
  • Prepare and shutdown of integration tests are like before.

Here is the POM section:

<build>
<plugins>

<!-- *** Surefire plugin: run unit and integration tests in
separate lifecycle phases, using file name pattern *** -->
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<skip>true</skip>
</configuration>
<executions>
<execution>
<id>unit-test</id>
<phase>test</phase>
<goals>
<goal>test</goal>
</goals>
<configuration>
<skip>false</skip>
<excludes>
<exclude>**/IT*.java</exclude>
</excludes>
</configuration>
</execution>

<execution>
<id>integration-test</id>
<phase>integration-test</phase>
<goals>
<goal>test</goal>
</goals>
<configuration>
<skip>false</skip>
<includes>
<include>**/IT*.java</include>
</includes>
</configuration>
</execution>
</executions>
</plugin>

<!-- *** Cargo plugin: start/stop application server and deploy the ear
file before/after integration tests *** -->
...

</plugins>
</build>

There is another approach to achieve the same result:

  • In configuration of Surefire plugin, use the exclude parameter to exclude the integration tests when executing unit tests in test phase.
  • Add an execution element for integration tests, where exclude is set to any dummy value (to override the default configuration) and include parameter is used to just include the integration tests and nothing else.

It's a bit shorter than the former configuration, but in the end it's a matter of taste. Again, here is the POM snippet:

<build>
<plugins>

<!-- *** Surefire plugin: run unit and integration tests in
separate lifecycle phases, using file name pattern *** -->
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<excludes>
<exclude>**/IT*.java</exclude>
</excludes>
</configuration>
<executions>
<execution>
<id>integration-test</id>
<phase>integration-test</phase>
<goals>
<goal>test</goal>
</goals>
<configuration>
<excludes>
<exclude>none</exclude>
</excludes>
<includes>
<include>**/IT*.java</include>
</includes>
</configuration>
</execution>
</executions>
</plugin>

<!-- *** Cargo plugin: start/stop application server and deploy the ear
file before/after integration tests *** -->
...

</plugins>
</build>

Failsafe Plugin

Instead of using the Surefire plugin with custom filename pattern to distinguish between unit and integration test classes, you should rather use the Failsafe plugin for executing integration tests.

This plugin is a fork of the Surefire plugin designed to run integration tests.
It is used during the integration-test and verify phases of the build lifecycle to execute the integration tests of an application. Other than Surefire plugin, the Failsafe plugin will not fail the build when executing tests thus enabling the post-integration-test phase to execute.

Failsafe plugin has its own naming convention. By default, the Surefire plugin executes **/Test*.java, **/*Test.java, and **/*TestCase.java test classes. In contrast, the Failsafe plugin will look for **/IT*.java, **/*IT.java, and **/*ITCase.java. Did you note that this matches what we used before for our integration tests? ;-)

When using Failsafe, the last POM (of option 3) looks like this:

<build>
<plugins>

<!-- *** Surefire plugin: run unit and exclude integration tests *** -->
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<excludes>
<exclude>**/IT*.java</exclude>
</excludes>
</configuration>
</plugin>

<!-- *** Failsafe plugin: run integration tests *** -->
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>failsafe-maven-plugin</artifactId>
<version>2.4.3-alpha-1</version>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>

<!-- *** Cargo plugin: start/stop application server and deploy the ear
file before/after integration tests *** -->
...

</plugins>
</build>

Of course, Failsafe would also work with first option (separate integration test module).

Conclusion

Well, after having shown all the ways that came into my mind, here are my personal "best practices":

  • Use the Failsafe plugin to execute integration tests.
  • If keeping integration tests in a separate module feels alright, do so (see option 1).
  • If you want to have unit and integration tests in the same module, choose a file or package name pattern to distinguish between both, and configure Surefire and Failsafe plugins accordingly.

Update (2010/02/18)

Note that there is a new version 2.5 of failsafe plugin available with changed group id: org.apache.maven.plugins:maven-failsafe-plugin:2.5. See the plugin site for details. Thanks to stug23 for pointing that out!