Additionally, with the help of the author of Wrk, we've been able to change the methodology to use time-limited tests (1 minute) rather than request-limited tests (100,000 requests). This means all frameworks are exercised for the same amount of time.
We've migrated the results to a stand-alone site so we have a little more screen real-estate for the charts and tables.
I look forward to any feedback, comments, questions, or criticism. Thanks!
The "raw" suffix indicates that an ORM is not used. This can give you an idea of the cost of using an ORM (or, in some cases, the cost of having more framework code in general).
The "servlet-raw" test uses raw JDBC to connect to and query the database. The "php-raw" test uses PHP's raw MySQL connectivity and no ORM. The "php" test with no "raw" suffix is using PHP ActiveRecord.
One strange thing is that Phreeze rocks on the multi-query test on the EC2 instance but on dedicated hardware does poorly. Would anyone have a clue why that would be? At the PHP level I have to admit I don't really factor in performance tuning for specific hardware - I was very surprised to see such a difference. I almost suspect the Nginx setup or something in the include path searching for too many files on one platform but not the other. Does anybody have any clues on where to look for something like that without having access to the testing hardware?
As for your question, I'm not certain why the physical hardware turned in a lower score than EC2. That suggests a configuration problem since we know the physical hardware is in fact quite a bit higher-performance. Pat (pfalls) may be able to find some time to help you diagnose it further.
Have you had a chance to benchmark Phreeze on some of your own hardware to fine-tune its configuration?
Color coding per language would be fantastic too!
If your use case actually matches these benchmarks then yes, Lift would be a poor choice.
[1] http://benchmarksgame.alioth.debian.org/u64/benchmark.php?te...
Also of interest: https://code.google.com/p/go/issues/detail?id=5323
1: https://groups.google.com/forum/?fromgroups=#!topic/golang-n...
2: https://github.com/go-sql-driver/mysql/pull/52 https://github.com/go-sql-driver/mysql/pull/55
Others from the Perl community submitted pull requests for some Perl framework subsequent to Round 3 and we will be including them going forward.
Look to see that data included soon!
