Brennan’s Blog

At work we run Cruise Control to continually build and test an application we are building for a client. It takes a long time because it is a complex client/server application which involves multiple databases (remote and local) and lots of funny business logic managed by the Drools rules engine and our own home-grown Concepts System. The project is broken up into various components and has many, many unit tests in place to validate lots of things. But we are learning a few things about the build processes...

The main lesson is something I have known for a while now, the Ant build process must be carefully constructed the way Ant wants it to be. The Ant build scripts used on this project do many things, perhaps more than they should be expected to do. But what I have seen that I have been concerned about is the heavy use of the Ant and Antcall tasks which I have feel is a messy way to run a build process.

A build manager wants to have a repeatable and predictable build process so Ant was build to discourage loops and control statements. I also believe there should be only one way to do each specific build task, such as compile, clean and package. Having more than one target capable of doing the same work in slightly different ways just adds more factors to the equation which hurts the organization of the build process. So it turns out that the Ant and Antcall tasks are a little memory hungry due to the fact Ant uses a new classloader each time Ant or Antcall tasks are used, meaning the same class may be held in memory many times over. If you use those tasks enough it can become a problem, especially if the build process is run inside Cruise Control. The inevitable problem is an OutOfMemoryException which may as well be the UsingAntCallTooMuchException! Fortunately there is a little Wiki which provides ways around the problem.

As a part of the Cruise Control process there are unit tests which are run. Unfortunately a set of unit tests causes later tests to be invalid so the database has been be reloaded before the first set of tests and again before the last set of unit tests. Typically in an Ant build process you cannot run a target more than once so you have to resort to either a copy and paste or use Antcall.

Just as you would program a batch file differently than a Java class you must use the features of the language to your advantage instead of forcing it to work as you want. A way to force it to work like a traditional script would be to use lots of Antcall statements.

<target name="integration" depends="init">
	<antcall target="compile"/>
	<antcall target="init.db"/>
	<antcall target="junit.phase1"/>
	<antcall target="init.db"/>
	<antcall target="junit.phase2"/>
	<antcall target="clean"/>
</target>

This mindset assumes that things must be called in order and that the init.db task must be called twice in one Ant process. An alternative would be to simply not use Antcall and use the depends clause instead to avoid the overuse of Antcall which chews up memory.

<target name="integration"
    depends="init,compile,init.db,junit.phase1,init.db,junit.phase2,clean">
</target>

Unfortunately Ant will not allow you to do it this way because init.db cannot be referenced twice in the depends attribute and Ant also ensures that a depends target is only called once. The depends attribute is a powerful Ant feature once you can master it, but many times it is used improperly. The way around the "runs only once" feature would be to create gateway targets with unique names so Ant does not really know it is doing the same thing twice.

<target name="integration"
   depends="init,compile,init.db.1st,junit.phase1,init.db.2nd,junit.phase2,clean">
</target>

<target name="init.db.1st"
    depends="init">
	<antcall target="init.db"/>
</target>

<target name="init.db.2nd"
    depends="init">
	<antcall target="init.db"/>
</target>

Here the depends attribute defines the sequence of targets to run and since the init.db.1st and init.db.2nd have unique names, both will be called. It does use the Antcall task, but at least it is not using the first example above which requires more Antcalls. So far this is what I am trying in order to reduce memory usage in the Cruise Control system and only time will tell if it prevents some of the memory issues we have experienced.

Alternatively, the Ant FAQ has a useful tip to leverage XML includes to let the XML parser automatically include external XML content within your document by using an entity reference. It is a clever use of the XML application but it all depends on having the right XML parser in place with Ant. Your mileage may vary.

With Ant 1.6 we can now use the import element to include external files as well. This is explained toward the end of the same FAQ answer linked above. It is a great way to segregate build files which grow and change continually. The main build.xml for this project is CVS revision 1.401 meaning it has been modified 401 times, not counting any changes to branches. I could easily see a separate file defining classpath and fileset references which may change during a project, but the main build script should remain largely unchanged during a project. I find the less often the main build script changes the better it is for those poor users who must synchronize everything each time someone changes the build script. At least with external Ant snippets being loaded as imports it may be less likely to cause trouble for the development team.

Ant keeps on evolving and will provide lots more features, but for now I think it does enough to compile, test, package and clean my working area and that is all I want it to do.