From Microservices to Distributed Systems - Survival guide for Java Developers

It kind of feel like the hype for Microservices is slowly coming down to earth and our industry is starting to realise, that a system according to the architectural paradigms behind Microservices can't easily be created by just exposing some HTTP interfaces on top of existing components. We do seem to have agreement on the necessity of having service optimised infrastructures, cultural and organisational changes and last but not least the outer architecture or orchestration for these architectures. The parts, that many Java developers still seem to struggle with are the concrete system architecture and the fact, that Microservices are nothing else than Distributed Systems. Unfortunately it's exactly these knowledge areas that decide about success of failure of your project. For a little bit of background, I suggest reading the wonderful InfoQ interview with Uwe and Adrian done by Daniel Bryant.

Why Microservices again? Can't I just be happy and write EJBs and Servlets?
The key idea with microservices are the properties that support independence of the rest of the application landscape and quick evolvability. Additionally, they should scale independently and require less resources than application server based applications. In a world with constantly changing business requirements and growing number of application clients, centralised infrastructures are getting way to expensive to operate and scale towards unpredictable load or load peaks. If we all would be stuck with application servers, we wouldn't have Netflix, Twitter or Amazon. So: No. You can't just stay where you are.

Microservices are Distributed Systems. What's so special about them?
The original definition of a distributed system: "A distributed system is a model in which components located on networked computers communicate and coordinate their actions by passing messages." (Wikipedia) And this is exactly what happens in Microservices based architectures. The individual services are deployed to cloud instances, physically running somewhere and they exchange messages. This is a big difference to how we used to build centralised applications. Instead of having a bunch of servers in our datacenter that handle all kinds of synchronisation, transaction and failover scenarios on our behalf, we now have individual services which evolve independently and aren't tied to each other. There are some fundamental challenges that are unique to distributed computing. Among them is fault tolerance, synchronisation, self healing, backpressure, network splits, and many more.

Aren't Distributed Systems what everybody calls Reactive Systems?
It's more complicated than that. And honestly, there is a lot going on with the word "Reactive" itself these days. To build an application or system out of individual Microservices, you need to use a set of design principles to make them Reactive, Resilient, Elastic and Message Driven. If that sounds familiar, you are probably right. The definition from the Reactive Manifesto. A distributed system which implements the four traits of the Reactive Manifesto is what should be called a Reactive System. You can read more about the design principles of Reactive Microservices Systems in Jonas' book. The Lagom framework is build on those principles, but let me be clear, that you don't necessarily need a specific framework or product to build these kind of applications. Some of them just make you a hell lot more productive and your operations more effective. Hugh McKee has another free book on design principles for Actor based systems.

What are the options to build a Microservice based System?
I personally see two different trends of solving the problems related to Microservices today. First is to push the problems down to orchestration or datacenter operating or cloud systems like DC/OS, OpenShift, Cloudfoundry, and alike.  The second solution is to natively handle them on the application or framework level (Akka, Vert.x, et al).

One container per service, or why an Anaconda shouldn't swallow a horse.
Let's look a little more detailed at the first approach. Write a microservice, package it together with the runtime in a little container and push it to the cloud. As we're all full stack, DevOps developers these days, it's easy to create the meta information needed for cloud based runtimes. Thanks to my bootiful service, all relevant monitoring information is exposed already and I can easily detect failing services and restart them. And this for sure works. You can even use a full blown application server as Microservice runtime. Plus, there are a lot of magic frameworks (NetflixOSS) which help with fighting the distributed systems challenges. The drawback for me personally is the tight coupling with the infrastructure in this case. Your system won't be able to run on anything else but the platform of choice. Further on, they suggest that you just need to use containers to solve all problems in the Microservices world. Looking back at the Reactive Manifesto, these type of systems won't help you with the requirement to use messaging between services.

Microservices without Containers? That's Peanut without Butter!
True. Containers do one thing very well. Package the complete stack in a controllable way into a deployable unit. They are isolation mechanisms on the infrastructure level. And having a container standard might actually be a good thing. So, keep your containers. But you need more.  So the key to building Resilient, self-healing systems is to allow failures to be: contained, reified as messages, sent to other components (that act as supervisors), and managed from a safe context outside the failed component. Here, being Message-driven is the enabler: moving away from strongly coupled, brittle, deeply nested synchronous call chains that everyone learned to suffer through…or ignore. The idea is to decouple the management of failures from the call chain, freeing the client from the responsibility of handling the failures of the server. No container or orchestration tooling will help you to integrate this. You are looking at Event Sourcing.  The design concepts for an event-driven architecture, using event sourcing, align well with Microservices architecture patterns.

Reactive Programming, Systems, Streams: Isn't that all the same?
Reactive has become an overloaded term and is now being associated with several different things to different people—in good company with words like “streaming”, “lightweight”, and “real-time. ”Reactive Programming offers productivity for Developers—through performance and resource efficiency—at the component level for internal logic and dataflow management. Reactive Systems offers productivity for Architects and DevOps—through resilience and elasticity—at the system level, for building Cloud Native or other large-scale distributed systems. You should really take the time and read how Jonas Bonér and Viktor Klang explain the individual differences between them.

Where can I learn more about how to design Reactive Microservices?
James Roper did a great talk at last year's Reactive Summit and took a hands on look at how the architecture of a system, including the flow of data, the types of communication used, and the way the system is broken down into components, will need to change as you decompose a monolith into a reactive microservice based system.

I did a talk at the CJUG about CQRS for Java Developers which gives you an intro. If you have particular topics that you are interested in, please let me know in the comments.

More Reading for you 

• Jonas Bonér & Viktor Klang Explain Reactive Programming vs Reactive Systems in 20 min
• Konrad did a webinar lately about Reactive Integrations in Java 8 with Akka Streams, Alpakka and Kafka
• The Basics Of Reactive System Design For Traditional Java Enterprises
• Duncan DeVore on Reactive Architecture, Design, And Programming In Less Than 12 Minutes

Read More

No, being wary doesn't hurt Java. A comment about Java licensing.

Screenshot from the Oracle Website (source)

So. Oracle want's to make money from Java. And The Register published a very polarising piece with a super catchy title about it. According to their sources, "Oracle is massively ramping up audits of Java customers it claims are in breach of its licences". While the Twitter-verse went ballistic about people criticising Oracle's behaviour, I want to take a minute to recap, why I am against this method and clarify, that the normal developer and user have nothing to fear!

People complaining about Oracle are hurting the community
You know me in and around the Oracle-sphere since many years. And tweeting at the moment is probably the most important part of how you can get the lastest news about me. I rarely post on this blog lately, because there is so much to do and I do a bunch of posts for my employer already.
The tweet I send was basically the title of The Register article and replies and reactions implied that just by spreading a FUD article, I am hurting the community. Let's look into the details.

Are Java developers affected?
NOTE: First of all, I am not a lawyer. This isn't mean to be a legal advice! If you are in doubt over your compliance to the BCL, contact a licensing lawyer, your local Oracle User Group or Oracle.

NO. We are covered. The Binary Code License (BCL) explicitly mentions:

"Oracle grants you a non-exclusive, non-transferable, limited license without fees to reproduce internally and use internally the Software complete and unmodified for the purpose of designing, developing, and testing your Programs."
(BCL, April 2013)

There are some cases you should be aware of.

"You may not use the Commercial Features for running Programs, Java applets or applications in your internal business operations or for any commercial or production purpose, or for any purpose other than as set forth in Sections B, C, D and E of these Supplemental Terms."
(BCL, April 2013)

- If you use a commercial feature in your local environment and stage it to test, you might not be covered.
- If you are accessing any of the  JMX MBeans under oracle.jrockit.management and bea.jrockit.management to enable observation of a running JVM, independently of the monitoring solutions, you are not covered.
- Flight Recorder, Mission Control and everything mentioned in the below linked PDF are commercial and you can only use them on your local machine.

A complete list of commercial features is listed in Table 1-1 on page 5 of the Java SE product edition description (PDF). As a general rule of thumb, make sure to NOT use the -XX:+UnlockCommercialFeatures option.

This mostly affects companies who are already Oracle customers and have access to the commercial features, that are most interesting (e.g. MSI Enterprise installer). Funny enough, WebLogic for example includes a Java SE license (random product link).

I'm still afraid, what are the alternatives?
You can go with various alternatives. First of all, there is the OpenJDK itself. Windows builds are a little tricky, as they are not available directly from the project. Only latest development releases are available for Windows. But you can also get binaries from Azul (Zulu) and Red Hat. If you are a Red Hat customer of any JBoss Middleware product, you also get support from them.

Why do I dislike what Oracle does?
I do like, that Oracle is sponsoring the Java development and I acknowledge that they invest significant manpower into the project. But that doesn't get them a carte blanche to get away with everything.
My main point of criticism is, that Oracle makes it easy to accidentally use commercial features. And yes, as people on Twitter pointed out, you should have just read the license and know about it. But as a matter of fact, I believe that separation of concerns is a good design decision. Instead of directing potential customers and users from the OpenJDK site to java.oracle.com with the comment "which are based largely on the same code" and not even distantly mentioning, that these do contain commercial features which you aren't allowed to use, it would be easier to have separate commercial and open source builds.

If you click around on the Oracle Java website you have a couple of hints at Licenses. But they don't get you the full picture.

Java SE Licenses overview page (source)

And there are plenty of more examples. The number one search result from (my personalized) Google for "Java License Business" leads to this page for example. Speaking of misleading information, I think, this is a good example. If license conditions aren't well known, it's also easier to change them. Which probably hasn't been the case, since the BCL PDF linked was last updated 02 April 2013.

My personal opinion is, that it can't be healthy for the community to stop improving. And improvements don't come from silence. Unfortunately, Oracle doesn't have a great history in listening to their communities which also might lead to some catchy and inappropriate headlines from time to time. Nevertheless, let's stay wary and keep communicating things that could improve. It'll help the community more than it hurts in the long run: Open communication is a sign of a healthy community and the way it deals with feedback and criticism is the gauge for their values.

Read More

Replacing legacy Java EE application servers with microservices and containers

Lightbend recently ran a survey with more than 2000 JVM developers and the results just got published. The survey was launched to discover:  correlations between development trends and IT infrastructure trends, how organizations at the forefront of digital transformation are modernizing their applications, and real production usage break-downs of today’s most buzzed about emerging developer technologies. While you can download the complete results from the official website, I would love to highlight some particular things that I found very interesting. Especially around containers and microservices.

Lightweight Containers Are Democratising Infrastructure and Challenging the Old Guard Java EE App Servers

The momentum around containers has gone much more quickly than many anticipated. People are looking at containers as that great hope for infrastructure portability that they’ve been chasing for a long time. And I was always interested in learning about how containers are actually used by developers in the wild. And bottom line is, that containers are really happening in production right now. What types of applications are people putting in containers is the million dollar question. Today it’s primarily greenfield applications, with far fewer examples of legacy applications being modernized for containers in production. This is the reason, that everybody is looking for more leightweight approaches to run their applications on the JVM without the overhead of Java EE servers. The survey has more details around which kind of containers and orchestration models.

Microservices and Fast Data Are Driving Application Modernization Efforts

Microservices-Based Architectures advocate the creation for a system build from a collection of small, isolated services, each of which owns their data, and is independently isolated, scalable and resilient to failure. Services integrate with other services in order to form a cohesive system that’s far more flexible than legacy monolithic applications. But how is this taken into production? Are people already building those systems or is this just a hype? Almost three third of the respondents run a microservice based system in production. And as I've been talking about in my talks before, the driver is mostly the need for real time data handling and streaming requirements.

The survey reveals a lot more details and I strongly suggest, that you look at the details of it. One thing is for sure, the changing requirements put on today's architectures can't be easily meet by just creating new applications on old platforms. And even Java EE is starting to adopt those new principles as JavaOne will hopefully how in a couple of days. I keep you posted.

Read More

Join me for the Reactive Microservices Roadshow

What are the benefits of the Reactive Manifesto and the Microservices approach, especially for those who want to fundamentally modernize their business? I will discuss this at the free event „Reactive Microservices Roadshow“, on 28th September in Berlin. The event is hosted by codecentric AG and Lightbend. There will be three talks which walk you through the most important parts.

Reaktive Mircroservices with Akka (Heiko Seeberger)
The Reactive Manifesto defines essential qualities which modern systems need to have in order to cope with today's requirements: Responsiveness, which is the cornerstone of usability and utility, requires resilience and elasticity; all of these are based upon asynchronous messaging. In this talk we look at the meaning of "Reactive" for a microservice architecture, for individual services and for their collaboration.

AutoScout24 goes Microservices (Christian Deger)
Fed up with stop and go in your data center? Why not shift into overdrive and pull into the fast lane? Learn how AutoScout24, the largest online car marketplace Europe-wide, are building their Autobahn in the cloud.
Reinventing themselves by making a radical transition from monoliths to microservices, from .NET on Windows to Scala on Linux, from data center to AWS and from built by devs and run by ops to a DevOps mindset.

One microservice is no microservice—they come in systems! (Markus Eisele)
Building a complete system out of individual Microservices is not as simple as we're being told. While Microservices-based Architecture continues to draw more and more attention we're also starting to learn about the trade-offs and drawbacks. Individual Microservices are fairly easy to understand and implement, but they only make sense as systems, and it is in-between the services that the most challenging (and interesting) problems arise—here we are entering the world of distributed systems.

Find a German event description on the codecentric.de website. Looking forward to interesting discussions! Make sure to REGISTER for FREE TODAY!

Read More

Lightbender at JavaOne 2016

It's only short 16 days until the Java community is starting to travel to San Francisco again. The annual family gathering at JavaOne is the place to be and it's time to get a little excited about attending once again. This year, Lightbend is a bronze sponsor and I am excited to be one of the featured speakers! A lot is going to happen during the week and you have to plan your schedule accordingly to get the most out of it.

A must attend are the sessions by Lightbend engineers.

Monitoring Reactive Microservices [CON1091] with Henrik Engström (@h3nk3)
Tuesday, Sep 20, 4:00 p.m. - 5:00 p.m. | Parc 55—Market Street
Reactive applications are the next major evolution of the internet. They allow for applications to be responsive, scalable, and resilient by building on a fully event-driven foundation. However, at the same time, this way of architecting systems introduces some new issues. One of these issues is how to monitor this type of system. This session covers the traditional monitoring approach and different ways to monitor asynchronous applications and finally looks at the way Lightbend has chosen to build a monitoring tool for reactive applications. After this presentation, developers will have a better understanding of how to monitor microservices in a reactive architecture.

End-to-End Reactive Streams, from Socket to Business [CON1852] with Konrad Malawski (@ktosopl)
Thursday, Sep 22, 11:30 a.m. - 12:30 p.m. | Hilton—Continental Ballroom 1/2/3
The Reactive Streams specification, along with its TCK and various implementations such as Akka Streams, is coming closer and closer with the inclusion of the RS types in JDK 9. Using an example Twitter-like streaming service implementation, this session shows why this is a game changer in terms of how you can design reactive streaming applications by connecting pipelines of back-pressured asynchronous processing stages. The presentation looks at the example from two perspectives: a raw implementation and an implementation addressing a high-level business need.

Stay Productive While Slicing Up the Monolith [CON6472] with myself (@myfear)
Tuesday, Sep 20, 11:00 a.m. - 12:00 p.m. | Parc 55—Mission
With microservices-based architectures, developers are left alone with provisioning and continuous delivery systems, containers and resource schedulers, frameworks and patterns to slice monoliths. How to efficiently develop them without having to provision complete production-like environments locally by hand? How to run microservices-based systems on local development machines, managing provisioning and orchestration of hundreds of services from a command-line tool without sacrificing productivity enablers. New buzzwords, frameworks, and hyped tools have made Java developers forget what it means to be productive. This session shows how much fun it can be to develop large-scale microservices-based systems. Understand the power of a fully integrated microservices development environment.

One Microservice Is No Microservice: They Come in Systems [CON6471] with myself (@myfear)
Wednesday, Sep 21, 1:00 p.m. - 2:00 p.m. | Parc 55—Embarcadero
Building a complete system out of individual microservices is hard. Microservices-based architecture is gaining attention, but there are trade-offs and drawbacks. Individual microservices are fairly easy to understand and implement, but they make sense only as systems; it’s between services that the most-challenging problems arise—in distributed systems. Slicing a system into REST services and wiring them back together with synchronous protocols and traditional enterprise tools means failure. This session distills the essence of microservices-based systems and covers a new development approach to microservices that gets you started quickly with a guided, minimalistic approach on your machine and takes you to a productive scaled-out microservices-based on the Oracle Cloud system with hundreds of services.

The Cloud-Natives Are RESTless [CON2514] Panel session with Konrad Malawski (@ktosopl)
Wednesday, Sep 21, 8:30 a.m. - 9:30 a.m. | Parc 55—Powell I/II
Representational State Transfer—the REST architecture—has served us well for the past 15 years as a style of cross-language distributed computing that is web-friendly. REST is simple and cacheable and is implemented over the original protocol for the web, good ole HTTP. For many use cases, the synchronous, request/response nature of REST fits perfectly. What are the alternatives to REST for event-based Java microservices? What reactive frameworks should Java developers learn and use in their services and overall application architecture? What synchronous cross-language alternatives should Java engineers use for high-performance, non-HTTP distributed computing in 2016 and beyond? Attend this session to find out.

I am looking forward to meet all the amazing peers from the Java Community! Find more information on the official JavaOne website and on the JavaOne blog and make also sure to follow @JavaOneConf on Twitter. And also don't forget to follow @myfear and @lightbend for more fun and games and raffles and stuff during JavaOne!

Read More

Remote JMS with WildFly Swarm

I'm blogging about WildFly swarm again? Short version is: I needed a test for remote JMS access and refused to setup something complex like a complete application server. The idea was to have a simple WildFly Swarm application which has a queue and a topic configured. Both should be accessible remotely from a standalone Java application. While the topic receives messages a Message Driven Bean (MDB) dumps the output to the console. The queue is filled with random text+timestamp messages by a singleton timer bean.
Turned out, that WildFly Swarm can do it, but for now only in the snapshot release.

The code
Find the complete code on my GitHub repository. It's not the most beautiful thing I have written but it actually shows you the complete configuration of Swarm with the relevant security settings, and the construction of the queue and the topic. In short the MessagingFraction needs the relevant security settings with remote access enabled and it also needs to define the remote topic. The NamingFraction needs to enable the remote naming service and finally the ManagamentFraction needs to define authorization handler.

How to run the example
To run the server, you can just use 'mvn wildfly-swarm:run' after the startup, you see the timer bean starting to emit messages to the queue:

2016-08-05 08:44:48,003 INFO  [sample.SampleQueueTimer] (EJB default - 5) Send: Test 1470379488003

2016-08-05 08:44:49,005 INFO  [sample.SampleQueueTimer] (EJB default - 6) Send: Test 1470379489005

If you point your browser to http://localhost:8080/ you can trigger a single message being send to the topic. This also get's logged to the console:

2016-08-05 08:44:36,220 INFO  [sample.SampleTopicMDB] (Thread-250 (ActiveMQ-client-global-threads-859113460)) received: something

The real magic happens, when you look at the standalone Java client. It performs the relevant JNDI lookups and creates the JMS connection with user and password, the session and the producer and finally produces and sends a text message.
More about the "why the hell does he needs Java EE again" in some upcoming blog posts ;)

Credits
A super big thank you goes out to Ken Finnigan who fixed the issue I ran into literally over night!

Read More

Build and deploy microservices the modern way

There's been a lot of buzz from me lately around microservices and containers. And all the efforts were directed towards today's public announcement by Lightbend and Mesosphere. If you are interested in learning more about how traditional architectures are beginning to evolve very quickly to embrace microservices architecture and various cloud and hybrid-cloud deployment models, I would love to invite you to listen to the recording of my recent webinar with Aaron Williams from Mesosphere. Find the slides on slideshare, the recording is embedded below.

The traditional model that enterprises run their businesses on has typically been delivered as monolithic applications running in a virtualized, on-premise infrastructure. We’ve seen how public and private cloud technologies have changed everything, but if the applications are not designed, or re-designed, appropriately, then it is impossible to take advantage of the advances in both distributed application services and hybrid infrastructure. Consequently, we will show how enterprise architects are looking to microservices architecture and technologies like Mesosphere DC/OS as a means to modernize their legacy applications.

This webinar introduces Lagom, a new framework specifically designed to help developers modernize legacy Java EE applications into systems of microservices and then discuss exactly what is required to run these distributed systems at enterprise scale with DC/OS.

Read More

CQRS with Java and Lagom

I've had the pleasure to talk at the Chicago Java User Group and talk about how Lagom implements CQRS, the Command Query Responsibility Segregation pattern. Thankfully, there is a recording and I also published the slides on slideshare.

Abstract:
As soon as an application becomes even moderately complex, CQRS and an Event Sourced architecture start making a lot of sense. The talk is focused on: - the challenges and tactics of separating the write model from the query model in a complex domain - how commands naturally lead to events and to an event based system, and - how events get projected into useful, eventually consistent views. Event Sourcing is one of those things that you really need to push through at the beginning (much like TDD) and that - once understood and internalized, will change the way you architect a system. This talk introduces you to the basic concepts and problem spaces to solve.

Thanks again for hosting me, CJUG! It was a real pleasure!

CJUG - 2016-06-28 - Markus Eisele on CQRS and Event Sourcing for Java Developers from Spantree Technology Group, LLC on Vimeo.

Read More

How Cloud, Containers, Microservices and you can help Charity!

No, this isn't a customer success story. And this isn't the typical post about some awesome stuff that you can do with technology. This time it is actually about knowledge in a different kind. Lightbend would love to know how you use the Cloud, Containers and Microservices in your daily work. How Enterprises are adopting these technologies and last but not least, how influential are the decisions that are made in development to the final system architecture choices being made.

Take the quick ten minute survey and don't forget to share the link with friends, colleagues and your contacts.

The results will be published next month (July) on the Lightbend tech blog!
Lightbend is donating to Charity for each completed survey! It's going to be either #yeswecode, SeniorNet or Mouse and you can select the cause at the end of the survey.  Make you voice heard and your answers count! We are aiming to break 3000 respondents, which unlocks a total donation of $1500!

Share the link
https://lightbend.qualtrics.com/SE/?SID=SV_aaXOYeDecDEMoRv

Read More

Modern Architecture with Containers, Microservices, and NoSQL

On Tuesday, May 10, 2016 I had the pleasure to join Arun Gupta (Couchbase), Mano Marks (Docker) and Shane Johnson (Couchbase) for a great webinar with ADTMag. You can watch the complete replay for free after registration. This blog highlights some of the most prominent findings and provides a brief writeup.

After a short introduction of the main business drivers behind the new architectures and the panel by Arun Gupta, it was time for Mano Marks (@manomarks) from Docker to give an overview of the container hype. With applications changing from centralised server installations which very rarely got updated to loosely coupled services with a high update frequency running on my small servers, containers provide a standard format which is easily portable between environments. Docker provides a great ecosystem around their products and is a solid foundation for applications following the new principles.

After Mano, it was my part. I did an overview from where we came from in terms of monolithic applications and why they survived so long including their advantages. With the introduction of microservices or better "right-sized" services we finally start to build systems for flexibility
and resiliency, not just efficiency and robustness. The relevant aspects for a successful microservices architecture are plenty and not easily to be achieved by using a single framework. You also have to respect the architecture, software design, methodology and organisation and also embrace the distributed systems thinking. I introduced the audience to some available decomposition strategies and also gave a very quick rundown about the Lightbend microservice framework Lagom.

Shane finished the presentation part of the webinar with an overview about the capabilities of the Couchbase server and how it supports application modernisation and microservice base architectures. The following FaQ with all the panelists tried to answer some of the most pressuring audience questions.

The whole webinar runs for an hour and it is packed with all the latest information around modern architectures. With the additional minutes spend on an hour, this is pretty much the most recent information you can get on the topic by the top speakers in the field. If you have nothing to do on this rainy weekend I highly recommend to watch it!

Read More