How the tests were performed:
I wanted to simulate a VPS environment similar to a basic Linode. A base install of Ubuntu Server 10.04 was installed in VMWare with an older/slower 7200RPM sata drive for storage. This drive was not in use by any other system during the test, nor were any other VMs active on the host. The guest was given 512MB of RAM and 1 CPU core of the host's 8 cores. The host's CPUs are dual Xeon X5365 @ 3Ghz.
Apache 2.2 was installed along with nginx 0.7.65, later Apache 2.4 was compiled on this same system.
Testing was performed using Apache JMeter on an 8 core Xeon workstation running Windows 7 and 32GB of RAM with a 1Gb/s link to the VMWare host. Requests per second were determined by rounding off the throughput displayed in the Summary Report listener. Each test was run until the requests per second stabilized.
The Apache 2.2 server was only tested with the Prefork MPM, while the Apache 2.4 server was tested with both Prefork and Event. Apache's KeepAlive setting was on throughout the testing and set at 2 seconds.
Update:
I received several requests to post memory usage statistics so I've updated the Jquery test with memory results. Again, care was not taken to keep the Apache builds consistent. It concerns me that the Prefork build of Apache 2.4 was using so much memory compared to the other Apache builds. Take these results with a grain of salt, but trust that Nginx definitely uses significantly less memory than Apache.
Update 2 - Nginx 1.0.12:
I received some flak for using an older version of Nginx, so I tested with Nginx 1.0.12 and it was around 4% slower than the results shown here.
Test 1 - 21KB text file
| HTTP Server | Req/s |
|---|---|
| Apache 2.2 Prefork | 2220 |
| Apache 2.4 Prefork | 2250 |
| Apache 2.4 Event | 2300 |
| Nginx | 2600 |
Test 2 - 2B text file consisting of a single period.
| HTTP Server | Req/s |
|---|---|
| Apache 2.2 Prefork | 4400 |
| Apache 2.4 Prefork | 4700 |
| Apache 2.4 Event | 4810 |
| Nginx | 6650 |
Test 3 - jquery.min.js (92KB)
| HTTP Server | Req/s | Memory Usage |
|---|---|---|
| Apache 2.2 Prefork | 650 | 12MB |
| Apache 2.4 Prefork | 770 | 72MB |
| Apache 2.4 Event | 820 | 20MB |
| Nginx | 1000 | 2MB |
Test 4 - PHP output of phpinfo()
| HTTP Server | Req/s |
|---|---|
| Apache 2.2 Prefork | 525 |
| Apache 2.4 Prefork | 575 |
| Nginx FastCGI | 450 |
8 comments:
On 1 core it makes no sense. Run the tests on many CPUs, because this is what Apache's supposed to be good at: scalability as opposed to simple efficiency.
"this is what Apache's supposed to be good at: scalability as opposed to simple efficiency"
No. Apache's scalability is so incredibly poor it's laughable when compared with lighthttpd, nginx, and even IIS.
Apache's strength is its market share. Everything was made to work with it, so it's highly compatible. That's all it has going for it.
Great stuff, Andy.
If would be perfect if you could include Nginx 1.2 benchmarks, though.
I look at the comments and it appears that people are thinking hat nginx is ahead in the performance department in this benchmark but this looks to me to be the opposite for what I am looking for in performance... here's why:
In the Environments I am working in PHP processing is much more of a bottle neck than static file serving especially if 75% of the static files (js,css,pngs,gifs) are already cached by the browser.
The Concurrency for the environments I am working in is between 50 & 200 requests per second so not too high, but there is a lot of php processing going on in the system.
Mass Media file serving appears faster in nginx, there are not many dynamic images, no video or other files in my environments.
In the environments I am evaluating to upgrade from apache 2.2 to either apache 2.4 or nginx, I don't think nginx will be an improvement based on these benchmarks.
Arguments can be made for tweaking either nginx or apache but if the compiled options for php are similar between apache and nginx then apache is ahead for the purposes I need it for.
I would like to see a benchmark between (nginx + php-fpm) vs (apache + statically compiled php) as I would assume the statically compiled apache php is supposedly the fastest.
So far I haven't seen a good case for switching to nginx for my purposes yet as it doesn't look like performance will improve at all, so I am leaning towards apache 2.4.
In the past I have tried lighttpd and for serving media files and it works well but there was no noticeable performance increase over apache 2.2 when using php.
Has anybody seen any other good php benchmarks comparing nginx 1.2 and apache 2.4?
Obviously Apache/mod_php is faster than Nginx+php-fpm... if you just look at pure php performance. However websites are not just php. If you look at a more realistic situations, you'll see nginx outperforms apache in any application, be it an ecommerce or a CMS or whatever.
I've choosen nginx over apache when setting up my webserver coz i made my own benchmarks with both and nginx was really much faster even if my application depends heavily on PHP. It allowed me to halve the budget for my production webserver (cloud based) at the cost of spending few hours RTFMing coz i know nothing about nginx.
Thanks to dotdeb.org i now run the latest version of nginx, php and mysql on my debian server and i'm completely satisfied by the performance.
could you update the benchmark with apache 2.2 + fastCGI + php-fpm?
At this scenario, the right way could be.
Use nginx in the front as proxy, and serving all static content (css, images, js) and leaving the php process to Apache.
So that's an Apache loaded with unnecessary modules (default config), directives, scanning and loading changes to .htaccess files and loads of features which Nginx will never have for years to come vs Nginx.
I just benchmarked the 2 (nginx 1.6). The problem is Nginx is designed for benchmarks. If you tune Apache for benchmarks, it'll outperform Nginx and it did in my case.
Post a Comment