JSP and Servlets can outperform mod_perl and mod_php. Java's outdated reputation for performance problems is no longer deserved. Fast servlet engines like Resin and fast JDK's like IBM's 1.1.8 JDK now make Java the language of choice for web applications.

Java has always been more maintainable than Perl or PHP scripts. With JSP, Java also becomes easy for web page design. Where Java has had trouble in the past is performance. That unfortunate perception is now obsolete. Java servers can now outperform the leading contenders for web site design.

Note: The benchmark has been rerun with an additional "Loop" benchmark. Also, several reviewers have suggested ways to improve the performance of Perl and PHP.

New Benchmark Comparison

All results are in operations per second. The results are sorted by Hello performance.

Single Client

Engine	File	Servlet	Hello	Loop	Big	DB	Cache
Resin/IBM	571	550	558	36	83	131/268	603
Resin/Apache/IBM	505	373	386	36	65	122/202	382
Resin/IIS	730	348	364	32	46	--	--
mod_perl/Apache	497	n/a	322	11	74	98	--
ASP/IIS	727	n/a	317	1.5	31	--	--
mod_php/Apache	504	n/a	308	13	36	130	--
Resin JS/Apache/IBM	484	n/a	254	16	61	33/118	376

The following table shows four HTTP clients and HTTP/1.1 keepalive. The fourth keepalive request closes the connection.

Four Clients with Four Keepalive

Engine	File	Servlet	Hello	Loop	Big	DB	Cache
Resin/IBM	1455	1187	1553	46	96	176/367	1561
Resin/Apache/IBM	880	564	575	38	91	135/237	586
Resin/IIS	1619	417	422	35	48	--	--
mod_perl/Apache	801	n/a	385	11	97	93	--
ASP/IIS	1654	n/a	385	1.4	80	--	--
mod_php/Apache	869	n/a	342	13	44	141	--

Benchmark Description

Each engine was tested against seven benchmarks.

File

Servlet

Hello

Loop

Big

DB

For the new Java benchmark, the first number uses the org.gjt MySql driver. The second number uses an experimental Caucho driver, available on request. So the two numbers gives a good idea of how important JDBC drivers are to the performance.

Cache

Old Data

The new version of the benchmark is incomplete. It leaves out several servlet engines. FYI, here's the old data. All results are in operations per second. The results are sorted by Hello performance.

Benchmark comparison

Engine	File	Servlet	Hello	Big	DB	Cache
Resin/IBM	615	557	510	96	143	572
Resin/JDK1.2	451	400	426	66	75	420
Resin/Apache/IBM	488	346	370	49	92	388
Orion/JDK1.2	433	165	319	61	78	64
mod_php/Apache	497	n/a	258	46	104	104
mod_perl/Apache	488	n/a	324	82	98	98
ServletExec/Apache/IBM	475	87	91	--	52	52
Tomcat/IBM	114	89	83	18	44	44
CGI perl/Apache	488	n/a	59	25	6	6
JRun/Apache/JDK1.2	471	61	33	--	22	22
Tomcat/JDK1.2	33	55	33	11	21	21

Web Content Engines

• Resin 1.1 b2 (internal web server)
• Orion 0.7.6b (internal web server)
• Tomcat (internal web server)
• Resin 1.1 b2/Apache 1.3.9
• mod_php 4.0b2
• mod_perl 1.21
• JRun 2.3.3/Apache 1.3.9
• ServletExec 2.2/Apache 1.3.9
• CGI using Perl

Configuration

The server is a 266 Mhz Pentium II running RedHat 6.0 with 64 meg of ram. The client is a 300 Mhz Celeron also running RedHat 6.0 with 32 meg. The two machines are connected with a 100bT ethernet connection.

The client is a simple C program acting as a browser. It sends requests, including typical header fields, and retrieves the results. The results are hashed and compared to an expected value.

Before each test, the client sends and discards a single request. This takes care of startup costs. In the test, 1000 requests were sent sequentially; the client waited for the request to complete before sending another request. Each test was repeated three times and the median result chosen.

Apache configurations used Apache 1.3.9. All modules were compiled with DSO support.

Where possible, the benchmark used the IBM 1.1.8 JDK. When the servlet engine didn't work with the IBM JDK, the Blackdown 1.2 was substituted instead. For example, Orion needs JDK 1.2 and JRun spun using the IBM JDK.

Database benchmarks used Mysql using the mm.mysql JDBC driver.

Discussion

First, these benchmarks are 'toy' benchmarks. To fully compare Resin to other servlet engines or Perl or PHP, you need to write your full application in both languages and compare the two. These benchmarks are only a starting point and should be taken with a grain of skepticism.

This benchmark didn't include a full application for two reasons. First, we're somewhat crunched for time. Second, what would be a representative benchmark? The home page of slashdot? Also, should be application be written as most people would write it, or the most maintainable, or the fastest hacked up version the best programmers can come up with? So a full application would raise more questions than it answered.

For example, compare the "Big" numbers with the "Loop" numbers. They both send the same amount of data back to the client, but the performance is difference is large. Based on this information alone, an evil benchmarker could tweak the benchmark to the benefit of any engine.

Based on this benchmark, the Perl and PHP database drivers are faster than the JDBC driver. This appears to be a JDBC implementation issue, not anything intrinsic to Java. The internal Caucho prototype JDBC driver gives much faster results, even faster than Perl and PHP. Since real projects can't use internal JDBC prototypes, the DB performance difference may matter to some projects.

Also, the 'Big' benchmark also gives the advantage to Perl. It may be interesting to note that changing the benchmark slightly makes Resin faster than Perl. If the benchmark is a simple for loop printing 1024 lines, with the same output size, Resin is faster than Perl. The benchmarks will be updated in a few weeks including the new loop test.

So is Resin or PHP or Perl faster? It really depends on your application. As a first cut, it's probably best to assume they have about the same performance unless you can benchmark your specific application. A difference of only 30% in a toy benchmark should be interpreted as having little or no difference.

These results are primarly important because previous servlet engines were much slower than PHP or Perl. With Resin, servlet performance is now about the same. Projects should use other considerations, like maintainability or ease of implementation to decide between the languages.

Conclusion

JSP can approach static page performance. A perception still exists that Java is a great programming environment but hobbled by performance. The perception is no longer valid.

The near-static performance means that many sites, who never considered JSP or servlets because of performance, can now take advantage of the Java platform's reliability and ease of programming benefits.

Copyright © 1998-2005 Caucho Technology, Inc. All rights reserved. Resin® is a registered trademark, and HardCoretm and Quercustm are trademarks of Caucho Technology, Inc.