iDempiere and OSGi: a modular Java architecture

iDempiere and Modularity

iDempiere is an open source Enterprise Resource Planning (ERP) system that describes itself as "OSGi + ADempiere", but what does that mean in practice? 

This post describes OSGi and the technology stack that was used to build iDempiere so that developers and solution architects can see how the system hangs together.

History of iDempiere

In the beginning, there was Compiere ("to accomplish, complete, fulfill" in Italian), an open source ERP system that was started in 1999. Initially it was very successful but after a long-standing disagreement between the developer community and the management company behind Compiere about its monetization strategy, the project was forked by a number of developers to create ADempiere (a word like 'compiere', but with the additional sense of "to fulfil obligations, or discharge duties", and also "to honor and respect").

ADempiere was also very successful, but its monolithic design had a number of limitations so some of the core architects forked the codebase in 2011 to create a more modular system called iDempiere, based on the OSGi specification.

OSGi and modularity

The OSGi specification defines a dynamic component system for Java. It is a specification that describes how individual bundles of code, also known as plug-ins, can be connected together in a dynamic way by a services layer that offers a publish-find-bind model for plain old Java objects (POJOs).

In standard Java everything in a JAR may be visible to all other JARs, but with OSGi, everything in that JAR is hidden unless it is explicitly exported in a predetermined manner. There is no sharing by default, so a bundle that wants to use another JAR must explicitly import the parts it needs. Conversely, a bundle must explicitly export functionality that is to be used by other bundles.

OSGi layered architecture

OSGi does this by specifying a layered model to manage plug-ins:

• Bundles: these are the OSGi plug-in components that deliver the application functionality, consisting of POJOs with additional meta-data defined in the MANIFEST.MF file. Bundles hide their internals from other bundles and communicate through well defined services. Bundles are self-contained, reusable units of deployable code, usually in the form of a JAR;
• Services: this layer connects bundles in a dynamic way by offering a publish-find-bind model for the POJOs registered in the service registry. Without OSGi, the standard Java solution is to use static factory methods and Java Reflection to load classes dynamically, resulting is a variety of passive, non-standard APIs and configuration mechanisms;
• Life-Cycle: the API that allows you to list, install, start, stop, update, and uninstall bundles on the fly without bringing down the whole system. Bundles can be managed directly from the OSGi console command line interface and they can be managed remotely;
• Modules: the layer that defines how a bundle's dependencies are managed by importing and exporting code according to the bundle's manifest. Modules are discrete, coarse-grained deployable units;
• Security: the layer that handles  security by separating bundles from each other and enforcing permissions to allow or deny bundles access to each other;
• Execution Environment: the underpinning OSGi platform that manages the containers. Like the Eclipse IDE, iDempiere is built on Equinox, which is the reference implementation of the OSGi specification (two other notable implementations of OSGi are Knopflerfish OSGi and Apache Felix).

Modular structure of iDempiere

The team that forked the code partitioned the monolithic Adempiere application into a large number of plug-ins, each with its own classpath and set of dependencies. If you download the iDempiere codebase and open it with Eclipse, you will see that it consists of dozens of projects: one for each OSGi bundle, or plug-in. 

This allows developers to create custom-built plug-ins without touching the upstream codebase. The Equinox OSGi container manages these plug-ins as free-standing, deployable Java objects that interact with the rest of the core iDempiere bundles by means of APIs.

Bundles are managed in a service registry

The advantages are:

• explicit dependency management: iDempiere plug-ins declare what they need and what functionality they provide, whilst otherwise remaining isolated from each other;
• versioned dependency: you can have different versions of the same Java class in different bundles if necessary because your plug-in's dependencies are managed by the OSGi container, not the JVM;
• small footprint: plug-ins are not packaged with all their dependencies, and they follow a lazy activation policy so they are only loaded as and when required;
• easy release: plug-ins can be developed and released independently of upstream developments;
• hot redeploy: individual plug-ins can be redeployed on the fly without affecting the others.

Core iDempiere plug-ins

org.adempiere.server

Now that we have looked at OSGi, let's dig deeper into the structure of iDempiere and look at the core plug-ins that make up the application.

When iDempiere starts, a Bash shell script idempiere-server.sh executes the Equinox OSGi launcher org.eclipse.equinox.launcher, passing it the name of the initial executable entry point as a runtime option, namely org.adempiere.server.application. This Java class implements the Equinox OSGi IApplication interface and starts the application.

The manifest of org.adempiere.server directs Equinox to load these bundles as dependencies:

• org.eclipse.equinox.app: implementation of the Equinox OSGi Application Container service;
• org.eclipse.jetty.apache-jsp: support for JSP in the embedded Jetty HTTP container;
• org.adempiere.base: the core functionality of iDempiere.

Equinox OSGi Application Container

The Equinox OSGi framework implementation is probably better known as the foundation of the Eclipse IDE platform, but it is actually a standalone OSGi implementation. It is launched by the org.adempiere.server plug-in when iDempiere is started and is responsible for loading all the other iDempiere plug-ins and managing the relationships and dependencies between them.

Read this tutorial on OSGi modularity with Eclipse by the ever-dependable Lars Vogel, and watch this video by Kirk Knoernschild, the author of "Java Application Architecture: Modularity Patterns with Examples Using OSGi", for a very useful overview of the current state of modularity on the Java platform.

Jetty HTTP Server

iDempiere uses the Jetty OSGi infrastructure to embed an HTTP server inside an OSGi container. This hosts the static content, servlets, OSGi web bundles (the OSGi equivalent of WAR files), and provides support for the Java ServerPages (JSPs) used by the original Adempiere code.

In production, it would be wise to put a reverse proxy in front of Jetty, running web application security software such as mod_security. Jetty can expose JMX services which should be monitored in a production environment. The Jetty configuration files are in $IDEMPIERE_HOME/jettyhome/etc/.

org.adempiere.base

This is the base plug-in that provides all the core functionality needed to build iDempiere plug-ins and includes interfaces such as:

• IResourceFinder: finds resources in each plugin;
• IColumnCallout: called whenever a column on a table changes its value;
• IModelValidator: called whenever a record is saved;
• IMenuAction: called whenever a window menu is created;
• IModelFactory: called to create new class models;
• and many others.

Wait, there's more...

These core plug-ins are at the heart of iDempiere, but there are many others that provide the application functionality, including the persistence layer, the report writer and the user interface, so follow the tutorial by Carlos Ruiz from GlobalQSS and download the code to explore further.

Here is a follow-up post that describes how Buckminster is used to build iDempiere on OSGi.