Polyglot Persistence: EclipseLink with MongoDB and Derby

Polyglot persistence has been in the news since some time now. Kicked off by the famous Fowler post from end 2011 I see more an more nice ideas coming up. Latest one was a company internal student project in which we used Scala as a backend persisting data into MongoDB, PostgreSQL and Apache Solr. I'm not a big fan of Scala and remembered EclipseLink's growing support for NoSQL databases. Given that I simply had to try something like this.

Where to start? 

The biggest issue are the missing examples. You find quite a bit stuff about how to change the data-containers (either NoSQL or RDBMS) with EclipseLink but you will not find a single one which exactly uses both technologies seamlessly. Thanks to Shaun Smith and Gunnar Wagenkrnecht we have this great JavaOne talk about Polyglot Persistence: EclipseLink JPA for NoSQL, Relational, and Beyond which talks exactly about this. Unfortunately the sources still haven't been pushed anywhere and I had to rebuild this from the talk.So, credits go to Shaun and Gunnar for this.
The magic solution is called Persistence Unit Composition. You need one persistence unit for every data container. That looks like the following basic example. You have a couple of entities in each PU and a composite PU is the umbrella.

Basic Composition Example

Let's go
You should have MongoDB in place before you're going to start this little tutorial example. Fire up NetBeans and create two java projects. Lets call them polyglot-persistence-nosql-pu and polyglot-persistence-rational-pu. Put the following entities into the nosql-pu: Customer, Address, Order and OrderLine. (Mostly taken from the EclipseLink nosql examples) and put a Product entity into the rational-pu.
The single products go into Derby while all the other entities persist into MongoDB. The interesting part is, where OrderLine has a One-to-One relation to a Product:

  @OneToOne(cascade = {CascadeType.REMOVE, CascadeType.PERSIST})
    private Product product;

This is the point where both worlds come together. More on that later.
Both PUs need to be transaction-type="RESOURCE_LOCAL" and need to contain the following line in the persistence.xml:

 <property name="eclipselink.composite-unit.member" value="true"/>

Don't forget to add the db specific configuration. For MongoDB this is

<property name="eclipselink.nosql.property.mongo.port" value="27017"/>
<property name="eclipselink.nosql.property.mongo.host" value="localhost"/>
<property name="eclipselink.nosql.property.mongo.db" value="mydb"/>

For derby this is something like this:

<property name="javax.persistence.jdbc.url" value="jdbc:derby://localhost:1527/mydb"/>
<property name="javax.persistence.jdbc.password" value="sa"/>
<property name="javax.persistence.jdbc.driver" value="org.apache.derby.jdbc.ClientDriver"/>
<property name="javax.persistence.jdbc.user" value="sa"/>

Now we need something to link those two PUs together. The combined-pu resides in a sample polyglot-persistence-web module and looks like this:

<persistence-unit name="composite-pu" transaction-type="RESOURCE_LOCAL">
    <provider>org.eclipse.persistence.jpa.PersistenceProvider</provider>
     <jar-file>\lib\polyglot-persistence-rational-pu-1.0-SNAPSHOT.jar</jar-file>
     <jar-file>\lib\polyglot-persistence-nosql-pu-1.0-SNAPSHOT.jar</jar-file>
      <properties>
            <property name="eclipselink.composite-unit" value="true"/>
        </properties>
</persistence-unit>
</persistence>

Watch out for the jar-file path. We are going to package this in a war-archive and because of this, the nosql-pu and the rational-pu will go into WEB-INF/lib folder. As you can see, my example is build with maven. Make sure to use the latest EclipseLink dependency. Even GlassFish 3.1.2.2 still ships with a lower version. MongoDB support has been added beginning with 2.4.

 <dependency>
  <groupId>org.eclipse.persistence</groupId>
            <artifactId>eclipselink</artifactId>
            <version>2.4.1</version>
        </dependency>

Beside this, you also need to turn GlassFish's classloaders around:

<class-loader delegate="false"/>

Don't worry about the details. I put up everything to github.com/myfear so, you might dig into the complete example later on your own.

Testing it
Let's make some very brief tests with it. Create a nice little Demo servlet and inject the composite-pu to it. Create an EntityManager from it and get a transaction. Now start creating prodcuts, a customer, the order and the separate order-lines. All plain JPA. No further magic here:

    @PersistenceUnit(unitName = "composite-pu")
    private EntityManagerFactory emf;

   protected void processRequest() // [...]
     {

        EntityManager em = emf.createEntityManager();
        em.getTransaction().begin();
        // Products go into RDBMS
        Product installation = new Product("installation");
        em.persist(installation);

        Product shipping = new Product("shipping");
        em.persist(shipping);

        Product maschine = new Product("maschine");
        em.persist(maschine);

        // Customer into NoSQL
        Customer customer = new Customer();
        customer.setName("myfear");
        em.persist(customer);
        // Order into NoSQL
        Order order = new Order();
        order.setCustomer(customer);
        order.setDescription("Pinball maschine");

        // Order Lines mapping NoSQL --- RDBMS
        order.addOrderLine(new OrderLine(maschine, 2999));
        order.addOrderLine(new OrderLine(shipping, 59));
        order.addOrderLine(new OrderLine(installation, 129));

        em.persist(order);
        em.getTransaction().commit();
        String orderId = order.getId();
        em.close();

If you put the right logging properties in place you can see, what is happening:
A couple of sequences are assigned to the created Product entities (GeneratedValue). The Customer entity gets persisted into Mongo with a MappedInteraction. Entities map onto collections in MongoDB.

FINE: Executing MappedInteraction()
spec => null
properties => {mongo.collection=CUSTOMER, mongo.operation=INSERT}
input => [DatabaseRecord(
CUSTOMER._id => 5098FF0C3D9F5D2CCB3CFECF
CUSTOMER.NAME => myfear)]

After that you see the products being inserted into Derby and again the MappedInteraction, that perssits the Order into MongoDB. The really cool part is down at the OrderLines:

ORDER.ORDERLINES => [DatabaseRecord(
 LINENUMBER => 1
 COST => 2999.0
 PRODUCT_ID => 3), DatabaseRecord(
 LINENUMBER => 2
 COST => 59.0
 PRODUCT_ID => 2), DatabaseRecord(
 LINENUMBER => 3
 COST => 129.0
 PRODUCT_ID => 1)]

Orderlines has an object which has the product_id which was generated for the related product entities. Further on you can also find the related Order and iterate over the products and get their descriptions:

Order order2 = em.find(Order.class, orderId);

  for (OrderLine orderLine : order2.getOrderLines()) {
            String desc = orderLine.getProduct().getDescription();
            }

The nice little demo looks like this:

Thanks Shaun, thanks Gunnar for this nice little example. Now go to github.com/myfear and get your hands dirty :)