JSF Component Libraries - Quality is more than zero bugs.

It has been a while since I last looked at the quality of the three major JSF component libraries. In December 2009 I started a comparison of the overall software quality of RichFaces, Primefaces and ICEfaces. Things have changed since than and I wanted to re-evaluate and update this since some time now. The tools I used back in 2009 are still valid but the tool-suite was a bit tricky to setup and I was simply missing the time for doing this. Thanks to the recent needs for a FAMIX 2.1 exporter I was looking again at inFusion.  It did the trick for GlassFish City posts (First, Second). But beside this it is far more. It's a tool to help with assessment of quality for your systems. It focuses on architecture and design quality and allows for quality assurance of multi-million LOC systems.
Before I give you an idea about what inFusion can do for you (implicitly by analyzing the candidates, I don't do advertising :-D) I have to thank Dr. Radu Marinescu and Dr. Adrian Trifu for providing a full functional test and evaluation license of their product to me. Without this I would not be able to show you the great software cities or blog about quality of open source projects in general like today! Please look at the resources underneath this post for further links about inFusion and the principles behind it. If you would like me to do a complete product post, let me know in the comments!

Focus of this Article
PrimeFaces, RichFaces and ICEfaces are the three mostly used JSF component libraries. Looking at the communities using it I always get the feeling that there is kind of a competition for the one and only. This is absolutely driven by the PrimeFaces lead. You can think about what he is doing and like it or not. With this post I am not trying to blame anybody about political correct behavior but trying to bring this back to some more objective views on the different projects by looking at the delivered quality.

Introduction
Before we get to the results I need to introduce you to some basics. If you feel you have seen enough of this before and everything down below is simple, feel free to proceed to the single results.
InFusion assesses software quality in a way that is build around but not centered on metrics. So it is introducing a special kind of quality model (QM) which expresses the quality of a software system in terms of some of its measurable characteristics. Quality itself can mean a couple of different things (External, Process, Internal quality). inFusion defines the notion of quality as “internal quality”, in other words the quality of the system’s architecture and design. The inFusion QM defines two decompositional layers: a layer of "quality attributes", and a layer of "design properties". The higher level overview contains a set of five "design properties" which is build up on a couple of well known "design principles" (e.g. DRY Principle and the Law of Demeter). With these principles in mind, inFusion measures deviations from most of these principles and design rules. By taking into account also the "bad smells" these deviations are quantified. All this together with the right mapping (which could be looked up in inFusion itself or the publications mentioned below) a "Quality Deficit Index" (QDI) is computed. The QDI is a positive, upwards unbound value, which is a measure of "badness" of the analyzed system's design quality respecting the overall size of the system.
Beside those high-level measures, inFusion also presents visualizations like coupling, encapsulation and design flaws on different levels (package, inheritance, class and modules).
I also like the metrics pyramid. It somehow answers the question "How does my project compare to others?".It generates a pyramid, showing key metrics for your project along with comparisons to industry-standard ranges for those numbers.It is separated into three different categories (inheritance, size and communication).

Overview Pyramid

The numbers indicate the ratios; the colors indicate where the ratios fit into the industry-standard ranges (derived from numerous open source projects). Each ratio
is either green (close to average range), blue (close to low range), or red (close to high range). The generated numbers serve a couple of purposes. First, they allow you to compare your code base to others along several dimensions. Second, these numbers indicate places where you might want to expend effort to improve code hygiene and design. However, you must understand these numbers in context.

PrimeFaces (QDI: 30,8)

Design Flaws on PrimeFaces

Founded in 2009 and having a growing user base. Head of development is Çağatay Çivici. The following analysis was run on the latest development trunk.

The total number of lines of code in the system is 44.123 (includings comments and whitespace). The Quality Deficit Index for primefaces is 30,8.
InFusion detected 12 different Design Flaws. Most impacting ones are the 24 Data Classes and the 23 Refused Parent Bequest classes. Followed by three God classes. There are quite a few duplication flaws but no cyclomatic dependencies.

Class hierarchies tend to be tall and wide (i.e. inheritance trees tend to have many depth-levels and base-classes with many directly derived sub-classes)
Classes tend to contain an average number of methods; be organized in rather fine-grained packages (i.e. few classes per package)
Methods tend to be rather long and having an average logical complexity ; call many methods (high coupling intensity) from few other classes (low coupling dispersion);

Metrics Pyramid for PrimeFaces

Given the fact, that this is a component library the NDD (Number of direct descendants) and HIT (Height of inheritance tree) might be acceptable. Complex inheritance is something that make understanding and predicting behavior more complex. Deeper trees constitute greater design complexity, since more methods and classes are involved, but enhance the potential reuse of inherited methods. NOM refers to the number of methods. This is a simple metric showing the complexity of a class in terms of responsibilities but not in terms of size of the methods.

RichFaces (QDI: 9.1)

Design Flaws on RichFaces

RichFaces originated from Ajax4jsf in late 2005. It is the widely used component library on JBoss.
The analysis was using the latest development trunk and only includes the core and the components parts.

The total number of lines of code in the system is 134.037 (includings comments and whitespace).
The Quality Deficit Index for RichFaces is 9.1.


Class hierarchies tend to be tall and of  average width  (i.e. inheritance trees tend to have many depth-levels and base-classes with several directly derived sub-classes)
Classes tend to contain an average number of methods; and are organized in rather fine-grained packages  (i.e. few classes per package);
Methods tend to: be average  in length and having an average logical complexity; call  many methods  (high coupling intensity)  from  few other classes  (low coupling dispersion);

Metrics Pyramid for RichFaces

RichFaces is doing a better job with hierarchies in general. Only the inheritance tree height is close to high range. The NOM for communication classes also is close to high. The rest is within the defined ranges which actually leads to this good QDI.

ICEfaces (QDI: 16.6)

Design Flaws on ICEfaces

ICEfaces is there since ... and 

The analysis was done against the 3.1.0-tag and includes the core, push and the components. 

The total number of lines of code in the system is 153.843 (includings comments and whitespace).
The Quality Deficit Index for ICEfaces is 16.6.
InFusion detected 16 different design flaws with 35 Data Classes, 13 God Classes, 20 SAP Breakers followed by 21 Refused Parent Bequest classes and 35 Cyclic Dependencies.We have a fair amount duplication in there, too.

Class hierarchies tend to be tall and of average width i.e. inheritance trees tend to have many depth-levels and base-classes with several directly derived sub-classes)

Classes tend to contain an average number of methods; be organized in rather fine-grained packages (i.e. few classes per package).

Methods tend to be rather long and having an average logical complexity; call many methods (high coupling intensity) from few other classes (low coupling dispersion).

Metrics Pyramid for ICEfaces

As expected, we also find a close to high inheritance tree height. Beside that only the number of methods is something to worry about.

Interpretation
This analysis is different from the one I did a few years back. I skipped looking at all the obvious stuff (e.g. checkstyle, findbugs) because everybody is running a different approach here and to me this simply isn't a comparable base for system quality in general.
Before we draw the conclusion here, let me first express that the results are no indication about weather you should use any of the candidates or not. The system design quality doesn't influence the quality of the code you produce using them. Also it shouldn't be any indicator about if the candidates are stable or bug-free. It simply focuses on the issues the developers building the products might face. On a long range this also might have an impact on you as a user. Because design problems are expensive, frequent and unavoidable. So having a lot of quality defects in a code base could influence the number of new features a team is able to deliver over time or the time for fixing bugs raises significantly. At the end, in combination with a small team this might lead to the end of a product.

All three candidates share the same problems in terms of inheritance. The reason for that is, that they are all frameworks which provide a good set of features to their clients. In combination with the size of the candidates, PrimeFaces seems to have the biggest design flaws at the moment doing the analysis. RichFaces is the leader in terms of quality far in front of any of the other two. This is what I expected to see from a RedHat community driven project. Another indicator, that working software communities are vital, skilled and kicking! ICEfaces is the only project with cyclic dependencies and an unusual amount of duplicated code. So they might end up having to fix the same error a couple of times.

I don't have a price to give away here but like to send my congratulations to the RichFaces team for a high quality product! Keep up the good work!

Here is your RichFaces-City (core & components). The green area is the old org.ajax4jsf.* bungalow :)

RichFaces-City

Resources:
inFusion Product Page
Object-Oriented Metrics in Practice (Springer, 2006)
iPlasma: An Integrated Platform for Quality Assessment of Object-Oriented Design (PDF)
Pragmatic Design Quality Assessment (Slideshare presentation)