NoSQL with Hibernate OGM - Part two: Querying for your data

After the first final version of Hibernate OGM came out end of January the team has been busy crafting a series of tutorial-style blogs which give you the chance to start over easily with Hibernate OGM. The fist part was all about getting setup and persisting your first entity. In this second part you're going to learn how to query your data. Hibernate OGM will let you get your data in several different ways:

• using the Java Persistence Query Langage (JP-QL)
• using the NoSQL native query language of the datastore of your choice (if it has one)
• using Hibernate Search queries - primarly full-text queries

All of these alternatives will allow you to run a query on the datastore and get the result as a list of managed entities.

Preparing the test class
We are going to add a new class HikeQueryTest. It will populate the datastore with some information about hikes:

    public class HikeQueryTest {

    private static EntityManagerFactory entityManagerFactory;

    @BeforeClass
    public static void setUpEntityManagerFactoryAndPopulateTheDatastore() {
        entityManagerFactory = Persistence.createEntityManagerFactory( "hikePu" );

            EntityManager entityManager = entityManagerFactory.createEntityManager();

            entityManager.getTransaction().begin();

            // create a Person
            Person bob = new Person( "Bob", "McRobb" );

            // and two hikes
            Hike cornwall = new Hike(
                "Visiting Land's End", new Date(), new BigDecimal( "5.5" ),
                new HikeSection( "Penzance", "Mousehole" ),
                new HikeSection( "Mousehole", "St. Levan" ),
                new HikeSection( "St. Levan", "Land's End" )
            );
            Hike isleOfWight = new Hike(
                "Exploring Carisbrooke Castle", new Date(), new BigDecimal( "7.5" ),
                new HikeSection( "Freshwater", "Calbourne" ),
                new HikeSection( "Calbourne", "Carisbrooke Castle" )
            );

            // let Bob organize the two hikes
            cornwall.setOrganizer( bob );
            bob.getOrganizedHikes().add( cornwall );

            isleOfWight.setOrganizer( bob );
            bob.getOrganizedHikes().add( isleOfWight );

            // persist organizer (will be cascaded to hikes)
            entityManager.persist( bob );

            entityManager.getTransaction().commit();
           entityManager.close();
    }

    @AfterClass
    public static void closeEntityManagerFactory() {
        entityManagerFactory.close();
    }
}

This methods will make sure that the entity manager factory is created before running the tests and that the datastore contains some data. The data are the same we stored in part 1.
Now that we have some data in place, we can start to write some tests to search for them.

Using the Java Persistence Query Langage (JP-QL)
The JP-QL is a query language defined as part of the Java Persistence API (JPA) specification. It's designed to work with entities and to be database independent.
Taking the entity Hike as an example:

@Entity
public class Hike {

    @Id
    @GeneratedValue(generator = "uuid")
    @GenericGenerator(name = "uuid", strategy = "uuid2")
    private String id;

    private String description;
    private Date date;
    private BigDecimal difficulty;

    @ManyToOne
    private Person organizer;

    @ElementCollection
    @OrderColumn(name = "sectionNo")
    private List<HikeSection> sections;
       
      // constructors, getters, setters, ...
}

A JP-QL query to get the list of available hikes ordered by difficulty looks like this:

SELECT h FROM Hike h ORDER BY h.difficulty ASC

Hibernate OGM will parse this query and transform it into the equivalent one in the native query language of the datastore of your choice. In Neo4j, for example, it creates and execute a Cypher query like the following:

MATCH (h:Hike) RETURN h ORDER BY h.difficulty

In MongoDB, using the MongoDB JavaScript API as a query notation, it looks like this:

db.Hike.find({}, { "difficulty": 1})

If you use JP-QL in your application you will be able to switch between datastore without the need to update the queries.
Now that you have an understanding of what's going on, we can start querying for the data we persisted. We can, for example, get the list of available hikes:

 @Test
    public void canSearchUsingJPQLQuery() {
        // Get a new entityManager
        EntityManager entityManager = entityManagerFactory.createEntityManager();

        // Start transaction
        entityManager.getTransaction().begin();

        // Find all the available hikes ordered by difficulty
        List<Hike> hikes = entityManager
            .createQuery( "SELECT h FROM Hike h ORDER BY h.difficulty ASC" , Hike.class )
            .getResultList();

        assertThat( hikes.size() ).isEqualTo( 2 );
        assertThat( hikes ).onProperty( "description" ).containsExactly( "Visiting Land's End", "Exploring Carisbrooke Castle" );

        entityManager.getTransaction().commit();
        entityManager.close();
    }

If you have used the JPA specification before you will find this code very familiar: it's the same code you would write when working on a relational database using JPA.
You can test this by switching the configuration and dependency between Neo4j and MongoDB: the test will still pass without any change in the code.
The cool thing is that you can use JP-QL queries with datastores which don't have their own query engine. Hibernate OGM's query parser will create full-text queries in this case which are executed via Hibernate Search and Lucene. We will see later how you can do this in more details.
The result of the query is a list of managed entities. This means that changes to the objects will be applied to the data in the database automatically. You also can navigate the resulting object graph, causing lazy associations to be loaded as required.
The support for the JP-QL language is not complete and it might change depending on the backend. We will leave the details to the official Hibernate OGM documentation. At the moment what's supported is:

• simple comparisons
• IS NULL and IS NOT NULL
• the boolean operators AND, OR, NOT
• LIKE, IN and BETWEEN
• ORDER BY

In case JP-QL is not a good fit for your use case, we will see how you can execute a query using the native language of the backend of your choice.

Using the native backend query language
Sometimes you might decide to sacrifice portablility in favor of the power of the underlying native query language. For example, you might want to benefit from the abilities of Neo4j's Cypher language for running hierarchical/recursive queries. Using MongoDB, let's get the hikes passing through "Penzance":

// Search for the hikes with a section that start from "Penzace" in MongoDB
List<Hike> hikes = entityManager.createNativeQuery("{ $query : { sections : { $elemMatch : { start: 'Penzance' } } } }", Hike.class ).getResultList();

The same code with Neo4j would look like this:

// Search for the hikes with a section that start from "Penzace" in Neo4j
List<Hike> hikes = entityManager.createNativeQuery( "MATCH (h:Hike) -- (:Hike_sections {start: 'Penzance'} ) RETURN h", 
Hike.class ).getResultList();

The important thing to notice is that, like JPA queries, the objects returned by the query are managed entities.
You can also define queries using the annotation javax.persistence.NamedNativeQuery:

@Entity
@NamedNativeQuery(
name = "PenzanceHikes",
query = "{ $query : { sections : { $elemMatch : { start: 'Penzance' } } } }", resultClass = Hike.class )
public class Hike { ... }

and then execute it like this:

List<Hike> hikes = entityManager.createNamedQuery( "PenzanceHikes" ).getResultList();

Using Hibernate Search queries
Hibernate Search offers a way to index Java objects into Lucene indexes and to execute full-text queries on them. The indexes do live outside your datastore. This means you can have query capabilities even if they are not supported natively. It also offers a few interesting properties in terms of feature set and scalability. In particular, using Hibernate Search, you can off-load query execution to separate nodes and scale it independently from the actual datastore nodes.
For this example we are going to use MongoDB. You first need to add Hibernate Search to your application. In a Maven project, you need to add the following dependency in the pom.xml:

<dependencies>
    ...
    <dependency>
        <groupId>org.hibernate</groupId>
        <artifactId>hibernate-search-orm</artifactId>
    </dependency>
    ...
</dependencies>

Now, you can select what you want to index:

@Entity
@Indexed
public class Hike {

    @Id
    @GeneratedValue(generator = "uuid")
    @GenericGenerator(name = "uuid", strategy = "uuid2")
    private String id;

    @Field
    private String description;

    private Date date;
    private BigDecimal difficulty;

    @ManyToOne
    private Person organizer;

    @ElementCollection
    @OrderColumn(name = "sectionNo")
    private List<HikeSection> sections;
       
    // constructors, getters, setters, ...
}

The @Indexed annotation identifies the classes that we want to index, while the @Field annotation specifies which properties of the class we want to index. Every time a new Hike entity is persisted via the entity manager using Hibernate OGM, Hibernate Search will automatically add it to the index and keep track of changes to managed entities. That way, index and datastore are up to date.
You can now look for the hikes to Carisbrooke using Lucene queries. In this example, we will use the query builder provided by Hibernate Search:

@Test
public void canSearchUsingFullTextQuery() {
    EntityManager entityManager = entityManagerFactory.createEntityManager();

    entityManager.getTransaction().begin();

    //Add full-text superpowers to any EntityManager:
    FullTextEntityManager ftem = Search.getFullTextEntityManager(entityManager);

    // Optionally use the QueryBuilder to simplify Query definition:
    QueryBuilder b = ftem.getSearchFactory().buildQueryBuilder().forEntity( Hike.class ).get();

    // A Lucene query to search for hikes to the Carisbrooke castle:
    Query lq = b.keyword().onField("description").matching("Carisbrooke castle").createQuery();

    //Transform the Lucene Query in a JPA Query:
    FullTextQuery ftQuery = ftem.createFullTextQuery(lq, Hike.class);

    //This is a requirement when using Hibernate OGM instead of ORM:
    ftQuery.initializeObjectsWith( ObjectLookupMethod.SKIP, DatabaseRetrievalMethod.FIND_BY_ID );

    // List matching hikes
    List<Hike> hikes = ftQuery.getResultList();
    assertThat( hikes ).onProperty( "description" ).containsOnly( "Exploring Carisbrooke Castle" );

    entityManager.getTransaction().commit();
    entityManager.close();
}

The result of the code will be a list of hikes mentioning "Carisbrooke castle" in the description.
Hibernate Search is a very powerful tool with many different options, it would take too long to describe all of them in this tutorial. You can check the reference documentation to learn more about it.

Wrap up
That's all for now. As you have seen, Hibernate OGM provides you with a range of options to run queries against your datastore, which should cover most of your typical query needs: JP-QL, native NoSQL queries and full-text queries via Hibernate Search / Apache Lucene. Even if you have never worked with NoSQL datastores before, you will be able to experiment with them easily.
You can find the complete example code of this blog post (and the previous one) on GitHub. Just fork it and play with it as you like.
Now that you know how to store and find your entities, we will see in the next part of the series how you can put everything inside an application container like WildFly.
We are eager to know your opinion, feel free to comment or contact us, we will answer your questions and hear your feed-back.

Thanks to Gunnar Morling (@gunnarmorling) and Davide D'Alto (@Github: DavidD) for creating this tutorial.