Ruby EventMachine :gt Python Tornado, Twisted

I was working on a project looking to see if Ruby was good enough for responding quickly to HTTP requests. Good thing it, along with Python, and every other language, plays well with C/C++. Anyways, EventMachine apparently blows away Tornado and Twisted. I only tested Tornado because it’s faster, right? What I really wanted to test was if either of these would fall apart under high concurrency or load. For the “Hello World!”, they both survived although as you can see for Tornado, response times became an issue earlier. I’ve also provided ‘ab’ for reference – it’s a little more specific with regard to response times. Clearly both of these are hitting a CPU ceiling – with Tornado hitting it faster. Ftr, I tested on a dual-core 2.33ghz xeon w/RHEL5, python2.6, and ruby1.8.5.

Along my adventure in this hnews thread, I came along this most awesome post: Twisted vs. Tornado: You’re Both Idiots

Anyways, what I’m happy about is there’s a Ruby option for a fast little server which pumps out a bajillion requests per second if you’ve got a farm of servers and it won’t fall on it’s face. Also, I don’t have to use Python and EventMachine is a BREEZE to use. What does suck is the EM HTTP server isn’t RFC compliant but that’s probably just a matter of time and I won’t be using HTTP anyways. ymmv

httperf: Tornado

[root@mail ~]# httperf --port=3002 --num-conns=1000 --num-calls=500 --rate 100 -v
httperf --verbose --client=0/1 --server=localhost --port=3002 --uri=/ --rate=100 --send-buffer=4096 --recv-buffer=16384 --num-conns=1000 --num-calls=500
httperf: maximum number of open descriptors = 1024
reply-rate = 5045.8
reply-rate = 4868.5
reply-rate = 4905.4
reply-rate = 4846.9
reply-rate = 4938.4
reply-rate = 4747.3
reply-rate = 4800.2
reply-rate = 4795.6
reply-rate = 4595.3
reply-rate = 4591.1
reply-rate = 4784.6
reply-rate = 4775.9
reply-rate = 4563.3
reply-rate = 4872.3
reply-rate = 4948.8
reply-rate = 4853.0
reply-rate = 4551.3
reply-rate = 4587.3
reply-rate = 4885.7
reply-rate = 4900.2
Maximum connect burst length: 1
Total: connections 1000 requests 500000 replies 500000 test-duration 104.059 s
Connection rate: 9.6 conn/s (104.1 ms/conn, <=1000 concurrent connections)
Connection time [ms]: min 34704.2 avg 93867.4 max 97177.5 median 95862.5 stddev 6293.6
Connection time [ms]: connect 0.0
Connection length [replies/conn]: 500.000
Request rate: 4805.0 req/s (0.2 ms/req)
Request size [B]: 62.0
Reply rate [replies/s]: min 4551.3 avg 4792.8 max 5045.8 stddev 144.4 (20 samples)
Reply time [ms]: response 187.7 transfer 0.0
Reply size [B]: header 156.0 content 12.0 footer 0.0 (total 168.0)
Reply status: 1xx=0 2xx=500000 3xx=0 4xx=0 5xx=0
CPU time [s]: user 2.97 system 99.60 (user 2.9% system 95.7% total 98.6%)
Net I/O: 1079.2 KB/s (8.8*10^6 bps)
Errors: total 0 client-timo 0 socket-timo 0 connrefused 0 connreset 0
Errors: fd-unavail 0 addrunavail 0 ftab-full 0 other 0

httperf: Ruby EventMachine

[root@mail ~]# httperf --port=3001 --num-conns=1000 --num-calls=500 --rate 100 -v
httperf --verbose --client=0/1 --server=localhost --port=3001 --uri=/ --rate=100 --send-buffer=4096 --recv-buffer=16384 --num-conns=1000 --num-calls=500
httperf: maximum number of open descriptors = 1024
reply-rate = 11631.7
reply-rate = 9769.5
reply-rate = 9352.3
reply-rate = 10086.1
reply-rate = 8899.4
reply-rate = 9759.3
reply-rate = 9985.1
reply-rate = 10152.8
reply-rate = 10383.9
Maximum connect burst length: 1
Total: connections 1000 requests 500000 replies 500000 test-duration 49.590 s
Connection rate: 20.2 conn/s (49.6 ms/conn, <=984 concurrent connections)
Connection time [ms]: min 229.8 avg 39130.7 max 42870.4 median 41409.5 stddev 6775.5
Connection time [ms]: connect 0.0
Connection length [replies/conn]: 500.000
Request rate: 10082.7 req/s (0.1 ms/req)
Request size [B]: 62.0
Reply rate [replies/s]: min 8899.4 avg 10002.2 max 11631.7 stddev 756.9 (9 samples)
Reply time [ms]: response 78.3 transfer 0.0
Reply size [B]: header 65.0 content 12.0 footer 0.0 (total 77.0)
Reply status: 1xx=0 2xx=500000 3xx=0 4xx=0 5xx=0
CPU time [s]: user 2.20 system 46.84 (user 4.4% system 94.4% total 98.9%)
Net I/O: 1368.7 KB/s (11.2*10^6 bps)
Errors: total 0 client-timo 0 socket-timo 0 connrefused 0 connreset 0
Errors: fd-unavail 0 addrunavail 0 ftab-full 0 other 0

ab: Tornado

[root@mail ~]# ab -c1000 -n100000 http://127.0.0.1:3002/
This is ApacheBench, Version 2.0.40-dev <$Revision: 1.146 $> apache-2.0
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright 2006 The Apache Software Foundation, http://www.apache.org/

Benchmarking 127.0.0.1 (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Finished 100000 requests

Server Software:        TornadoServer/0.1
Server Hostname:        127.0.0.1
Server Port:            3002

Document Path:          /
Document Length:        12 bytes

Concurrency Level:      1000
Time taken for tests:   27.996766 seconds
Complete requests:      100000
Failed requests:        0
Write errors:           0
Total transferred:      16800336 bytes
HTML transferred:       1200024 bytes
Requests per second:    3571.84 [#/sec] (mean)
Time per request:       279.968 [ms] (mean)
Time per request:       0.280 [ms] (mean, across all concurrent requests)
Transfer rate:          586.00 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0  197 1102.8      0   20998
Processing:     1   50  37.3     45    5234
Waiting:        0   49  37.4     44    5234
Total:         18  247 1109.2     45   21253

Percentage of the requests served within a certain time (ms)
  50%     45
  66%     48
  75%     52
  80%     57
  90%     77
  95%   1237
  98%   3074
  99%   3112
 100%  21253 (longest request)

ab: EventMachine

[root@mail ~]# ab -c1000 -n100000 http://127.0.0.1:3001/
This is ApacheBench, Version 2.0.40-dev <$Revision: 1.146 $> apache-2.0
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright 2006 The Apache Software Foundation, http://www.apache.org/

Benchmarking 127.0.0.1 (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Finished 100000 requests

Server Software:
Server Hostname:        127.0.0.1
Server Port:            3001

Document Path:          /
Document Length:        12 bytes

Concurrency Level:      1000
Time taken for tests:   15.238117 seconds
Complete requests:      100000
Failed requests:        0
Write errors:           0
Total transferred:      7700077 bytes
HTML transferred:       1200012 bytes
Requests per second:    6562.49 [#/sec] (mean)
Time per request:       152.381 [ms] (mean)
Time per request:       0.152 [ms] (mean, across all concurrent requests)
Transfer rate:          493.43 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0   76 603.3      0    9000
Processing:     0   32 264.1     15   10627
Waiting:        0   31 264.1     14   10625
Total:          9  108 752.8     15   14642

Percentage of the requests served within a certain time (ms)
  50%     15
  66%     15
  75%     15
  80%     22
  90%     33
  95%     35
  98%   2999
  99%   3015
 100%  14642 (longest request)
Ruby EventMachine :gt Python Tornado, Twisted