This post shows how to render a list of product objects with associated category object using the webframeworks Rails, Wicket, Grails, Play, Tapestry, Lift, Context and JSP/Servlets. I’ve also benchmarked theresponse times.
Framework: JSP, JSTL, Servlets
Because of a bug ( https://issues.apache.org/jira/browse/WICKET-3740 ) in Wicket, which might impact performance, Wicket is also tested for the latest snapshot version of Wicket from trunk for 1.5-RC3. (Called “Wicket-trunk” in the test results.)
Framework: Ruby On Rails
-WEBrick 1.3.1 (webserver)
-mongrel 1.2.0.pre2 (webserver)
-Trinidad 1.2.0 (embedded Tomcat for JRuby)
Static file test
The static file test is rendering of plain HTML files served directly by the webserver (Tomcat). So no framework was used in this test.
Note: that the more component based frameworks – such as Wicket and Lift – have multiple ways of doing things. Where the simpler MVC push frameworks – such as Rails, JSP, Play, Grails – usually have one way of doing it. Therefore, because there are multiple solutions in Wicket and Lift, there may be faster (and slower) solutions than I have used. However, I did try to use common solutions.
Wicket-1.5rc3 and Wicket-1.5r5trunk use a ListView to iterate over the categories, but Wicketb-1.5rc5trunk uses a RepeatingView.
Test Remarks CPU Utilization is inaccurate
Effects of page size by unneeded whitespaces
JMeter impacts the test results. This might be due to JMeter using RAM or CPU. I should retest the tests with JMeters Distribution Graph disabled. Package sizes
What suprised me is that even though JSP/JSTL, Rails, Grails and Play use about the same MVC model, the differences in performance are big. Simply switching from template system in Play framework from the default to Japid, has a huge impact (in this test 27 times faster).
Most posts about JRuby on Rails tell that it is faster than Ruby on Rails, however in this test, Ruby on Rails was clearly faster in both the rendering of products and concurrent-user test.
Also, most posts about Rails tell that Webrick is slower than Mongrel, but in this test, Webrick was a bit faster.
JRuby on Rails running on Trinidad does perform faster than other Rails configurations when tested with concurrent users.
Play-Scala using Netty NIO server is faster than Tomcat webserver. I wouldn’t expect the differences to be that big. It’s unclear whether these fast results are caused by Play-Scala being optimized for Netty, or that Netty is simply faster than Tomcat (and thus that other frameworks could also have faster results under Netty).
Note that the memory usage of Play-Scala under Netty is a lot higher than under Tomcat.
The section “Effects of page size by unneeded whitespaces” shows that unneeded whitespaces impact the response time. Frameworks can probably be optimized by stripping unneeded whitespaces from the templates during boot, even when HTTP gzip compression is enabled. (During the test, compression was disabled in Tomcat and Netty.) Questions
-Does anybody know why the static file test under Tomcat is using so much memory?
Blog post updates
-30 May 2011: Added Play-scala (under Tomcat webserver) and Play-scala-netty (under Play’s builtin Netty webserver). Added the effects of unneeded whitespaces. Added Wicket-trunk, which is the latest Wicket snaphot from trunk (for 1.5-RC5), and info about a performance bug in Wicket. Project files
-31 May 2011: Added Play-Japid and JRails.
-3 June 2011: Added Grails trunk.
-13 June 2011: Added Tapestry, Context-framework and Grails-1.3.7-freemarker
The files (code, project, html) of this post can be found here: https://github.com/jtdev/blogpost_files