iDempiere and Buckminster: building a web-based OSGi application

iDempiere and Buckminster

iDempiere is an open source, web-based Enterprise Resource Planning (ERP) system that has been built with tools from the Java world, but instead of using a familiar build automation tool like Maven, or a newer tool like Gradle, the iDempiere team used Buckminster to manage the build.

We tackled OSGi in an earlier post, now Buckminster as well? OK, roll up your sleeves...

This post looks at the use of Buckminster in the iDempiere project so that we can get a better understanding of the build process, and why Buckminster was used.

What is Buckminster?

Buckminster is a set of frameworks and tools for automating build, assemble and deploy processes. The project is named after the architect R. Buckminster Fuller who said about problem-solving "when I have finished, if the solution is not beautiful, I know it is wrong."

R. Buckminster Fuller

The problem that Buckminster is trying to solve is how to build and assemble multiple software components from multiple sources in combinations that are specific to a given target platform.

Buckminster uses a Component Query (or CQUERY) to identify the components and their associated component information and create a bill of materials (BOM). It then locates all the components in the BOM using a Resource Map (or RMAP) and materializes the project by copying them into the target platform, resolving the dependencies according to the work context, system setup or operating system.

Components

In the context of iDempiere, a component is a collection of files that make up an OSGi bundle, or plug-in, and in the context of the Eclipse IDE, a component may be an Eclipse project. Components can also exist outside the workspace, and they may be dependent on other components. 

RMAP component location mapping

Buckminster can store component information as a Component Specification (CSPEC) in an XML file inside the component, but it is also able to read meta-data from pre-existing sources such as plugin.xml, feature.xml, POM files, etc, and translate such information on-the-fly into CSPEC, which it does in the case of iDempiere.

Buckminster allows you to retrieve components from existing repositories and component formats, such as CVS, SVN, git, Maven, or p2 repositories. This enables the iDempiere team to integrate existing Adempiere code with bespoke iDempiere plug-ins and other third-party modules, without reconfiguring them.

Note that Buckminster can get the same component from different repositories, depending on the circumstances (different geographic locations, for example, or different software development environments like DEV, TEST, QA, PROD, and so on), making it easy to build your own on-site repositories for iDempiere.

Materializing iDempiere in Eclipse

Materialization Specification (MSPEC)

When you install iDempiere in an Eclipse development environment, you must first materialize the project by using a wizard to populate the workspace from the Buckminster Materialization Specification (MSPEC):

File > Import > Buckminster > Materialize from Buckminster MSPEC, CQUERY, or BOM > 

URL: $IDEMPIERE_HOME/org.adempiere.sdk-feature/adempiere.mspec

where $IDEMPIERE_HOME is the root directory of your iDempiere download.

The iDempiere MSPEC instructs Buckminster to install the materialized components in ${workspace.root}/targetPlatform/, using the Component Query in adempiere.cquery, and gives the MSPEC a human-readable name:
org.adempiere.sdk:eclipse.feature:1.0.0.qualifier

Component Query (CQUERY)

Buckminster then reads the Component Query from the CQUERY file named in the MSPEC, triggering its resolution and provisioning process. The CQUERY tells Buckminster where to find the Resource Map, adempiere.rmap, and also defines a number of advisorNodes which specify how groups of components should be handled.

The CQUERY resolution runs for a few minutes to build the Bill of Materials. The wizard gives you the option of saving the BOM to a file if you are interested in reading 20,097 lines of XML.

Resource map (RMAP)

Finally, Buckminster materializes the BOM by looking in the Resource Map (RMAP) for the location of the components and copying the JARs to the targetPlatform.

Note: materialization takes a very looooooong time, so put the kettle on and read the BuckyBook, the definitive guide to Buckminster (perhaps not: see below).

Update

After this post was written, Low Heng Sin, one of the core architects of iDempiere, explained to me that they used Buckminster because Maven was not capable of building Eclipse plug-ins when the project started.

Since then the Tycho set of plug-ins and extensions has been developed to build OSGi bundles using Maven.  Tycho uses OSGi metadata and OSGi rules to deduce project dependencies dynamically and inject them into the Maven project model at build time.

The development of Buckminster appears to have come to a halt so Tycho may be a better bet for new OSGi projects.