Upgrading to latest Mojarra in GlassFish 3.1.x

I was playing around with latest Mojarra and in order not to replace any existing versions at a server module level I tried to package it with my apps. I shouldn't have done that. Here is a short story.

I do love application servers. Especially GlassFish and WebLogic. You know that. But there is one thing that starts to bug me more and more with latest releases. It's the release cycle of bundled components. Do you remember the good old days where you simply threw a newer library into your app bundle and (nearly) everything worked fine? Those days belong to the past. As of today you experience a lot of trouble if you try to upgrade and replace single reference implementations or components of your appserver by bundling them to your apps. The recent example was Mojarra. Some bugs in the Mojarra 2.1.5 which is shipped with the latest GlassFish 3.1.2-b22 prevented me from upgrading an application. So I thought I give the latest 2.1.10 (05/31/2012) a shot which promised to solve those issues. I am very thankful for the frequent releases here and was very very looking forward having this solved in literally a few seconds.
Googling around a bit for the specifics of the upgrade process brought me to Arun's blog. A four year old entry pointed me to the basics. Bundle impl and api jars to your application. Add some lines to your sun-web.xml.
There you go. That should be everything. Fine. Obviously the name of the deployment descriptor changed. It's new name is glassfish-web.xml and also the property name changed from useMyFaces to useBundledJsf. Not a big surprise and very welcome.

<class-loader delegate="false"/>
<property name="useBundledJsf" 
value="false"/>

If you now try to find the jsf-impl and jsf-api dependencies for the 2.1.10 you start struggling. With 2.1.3 Mojarra started adopting the Oracle Java EE Maven and OSGi naming and versioning scheme. This basically means, you now have only one dependency for both:

<dependency>
  <groupId>org.glassfish</groupId>
  <artifactId>javax.faces</artifactId>
  <version>2.1.10</version>
  <scope>compile</scope>
</dependency>

Also fine to me. Let's give that a test-drive. Ups:

java.lang.Exception: java.lang.IllegalStateException: ContainerBase.addChild: start: org.apache.catalina.LifecycleException: java.lang.RuntimeException: com.sun.faces.config.ConfigurationException: CONFIGURATION FAILED! org.glassfish.weld.jsf.WeldFacesConfigProvider cannot be cast to com.sun.faces.spi.ConfigurationResourceProvider

That did not work!
What happened?
The issue seems to be that there are two versions of ConfigurationResourceProvider on the class path: one at the web app level (your bundled jar file) and a second at the parent class loader level (provided by GlassFish). WeldFacesConfigProvider extends ConfigurationResourceProvider at the parent class loader, but it ends up casting this to ConfigurationResourceProvider at the web app class loader.

Workaround: Replacing Mojarra in GlassFish
How to work around that? Easy if you like it to be easy. Simply download the latest stable version of Mojarra from it's java.net project and replace the glassfish3\glassfish\modules\javax.faces.jar with the downloaded version. Make sure to keep a backup of the original file and also don't forget to delete the osgi cache glassfish3\glassfish\osgi for the changes to take effect. Also make sure to only use a 2.1 version of Mojarra and don't try a 2.2 even if it is there. First of all because it implements the new EE 7 features and second because it is still a snapshot. Keep away here! For now.
Why you might not like this approach. Imagine you are a paying Oracle GlassFish Server Customer. What do you think? Do you still get support if you are replacing a core module with a newer version? I haven't checked the licensing and support terms. But I believe you will not. And that might be right from a product point of view because you simply can't test every possible combination of modules for every release. 

Workaround: Adding the missing dependency to your app
If you can't cast across class loaders you have to make the missing part available to the right class-loader. Not replacing the javax.faces.jar in your server's modules leads to adding the magic WeldFacesConfigProvider to your app. It is contained in the glassfish3\glassfish\modules\weld-integration.jar and by simply putting that as an additional library to your application you also solve the issue.
There is even a mavenized version on the web. But this is drawing in a lot of additional dependencies which you would have to exclude and it also seems to be a snapshot build only. So you will end up putting this jar to your local or corporate maven repository. Going this way you should still receive support for your commercial appserver version. Even if the weld-integration obviously was not thought of being deployed with your applications every time over and over again. But with 77KB this also isn't a too big issue.

The good news
The JSF and GlassFish team is working on this. And I hope they are getting this fixed with one of the upcoming GlassFish releases. Thanks to Andy and Ed for the support here and for taking care.

Modularization and Rolling Upgrades
As I said in the introduction, this is only one example. I believe you could come up with more. And the voices are getting louder which call for a better modularization everywhere. And they are right. If I or you would have developed a complex project with all those cross dependencies we would have been kicked hardly. Either by our customers or by our coworkers. So, even if we dropped that from the scope of EE 7 I am very looking forward having the possibility to integrate this into EE 8 latest. Another point here are the release cycles and the update policies for vendors. Even if GlassFish isn't broken by default and it is a very stable server in general you have other examples out there: Let's look at the latest WebLogic 12c release. It has been released quite late comparing to other EE 6 compliant servers. Bringing in some bugs and forcing Oracle to deliver a repackaged distribution later on. And even this isn't completely usable (Issue, Issue) forcing you to be either a paying customer to receive the fixes or stay off the product. Bottom line? I don't know. Something is wrong the way it is today. We need to work harder to make EE a specification which can be used, updated and worked with. And I believe the next release shouldn't focus on first level developer experience but on vendors and quality. What is a car worth that is easy to drive for everyone but breaks down frequently with bugs and missing spare parts?