In response to Wil’s comment regarding the PHP framework performance comparisons I made in my previous post, I have decided to post the results I got.
My decision not post them initially was due to the benchmarks not being done in complete isolation (a seperate client and server machine) but the scores relative to each should still be accurate and that’s what I am testing for.
Furthermore, while I only used requests/sec as a performance yardstick, I do realize that other factors do affect a web application’s response in the real world.
The Tools
The tests were run on a 1.8Ghz AMD sempron computer with 512Mb RAM running Ubuntu Gutsy (7.10) Desktop.
I used httperf to run the tests although I did run some of them again using apachebench and the results were consistent.
I created the same front page with CakePHP, CodeIgniter, Zend Framework and Ruby on Rails. I also duplicated the same functionality in HTML and in PHP using procedural code (aka. spaghetti code) to act as baselines.
I decided to include data access in the tests and the tests involve the application fetching four rows from the database and rendering the results.
View the sample page.
The test files are also available for download for those interested in them. The sql file for the mysql database is also included. The code assumes a database root user with no password.
The Results
All scores are requests per second. Higher is better.
- No PHP code cache
All frameworks used an ORM (of sorts, in the case of codeigniter). The Zend Framework used Zend_DB_Table and CodeIgniter used ActiveRecord.
|
Run 1
|
Run 2
|
Run 3
|
Run 4
|
Average
|
| Baseline HTML |
1327.5
|
1326.5
|
1328.6
|
1329.1
|
1327.9
|
| Baseline PHP |
331.6
|
332.1
|
331.4
|
332.0
|
331.8
|
| CakePHP |
3.6
|
3.7
|
3.8
|
3.5
|
3.7
|
| CodeIgniter |
21.5
|
21.2
|
21.7
|
21.7
|
21.5
|
| Zend Framework |
9.3
|
9.1
|
9.2
|
9.3
|
9.2
|
- With eAccelerator PHP code cache
All frameworks used an ORM (of sorts, in the case of CodeIgniter). The Zend Framework used Zend_DB_Table and CodeIgniter used ActiveRecord.
|
Run 1
|
Run 2
|
Run 3
|
Run 4
|
Average
|
| CakePHP |
8.0
|
8.0
|
8.0
|
7.2
|
7.8
|
| CodeIgniter |
98.2
|
98.1
|
98.3
|
98.3
|
98.2
|
| Zend Framework |
33.2
|
33.3
|
33.5
|
33.6
|
33.4
|
- With APC PHP code cache
All frameworks used an ORM (of sorts, in the case of CodeIgniter). The Zend Framework used Zend_DB_Table and CodeIgniter used ActiveRecord.
|
Run 1
|
Run 2
|
Run 3
|
Run 4
|
Average
|
| CakePHP |
7.3
|
7.3
|
7.3
|
7.3
|
7.3
|
| CodeIgniter |
97.5
|
98.0
|
96.6
|
98.3
|
97.6
|
| Zend Framework |
32.8
|
33.3
|
31.8
|
32.7
|
32.7
|
- With APC PHP code cache, No ORM
The Zend Framework used Zend_DB and I disabled ActiveRecord in CodeIgniter. I couldn’t figure out how to disable the ORM in cakePHP so I left it out.
|
Run 1
|
Run 2
|
Run 3
|
Run 4
|
Average
|
| CodeIgniter |
106.4
|
105.6
|
106.3
|
106.6
|
106.2
|
| Zend Framework |
42.6
|
42.8
|
42.9
|
43.0
|
42.8
|
- With APC PHP code cache, No database calls
This tests the impact of the database call on the overall performance. I sent an empty result set to the view directly from the controller, thus bypassing the model.
|
Run 1
|
Run 2
|
Run 3
|
Run 4
|
Average
|
| CodeIgniter |
118.1
|
118.3
|
117.3
|
118.2
|
118.0
|
| Zend Framework |
51.9
|
52.1
|
52.1
|
52.2
|
52.0
|
- Ruby on Rails comparison
I added this test to see how the PHP frameworks stcked up against Ruby on Rails. I also used the chance to try out Passenger (mod_rails) and Ruby Enterprise.
|
Run 1
|
Run 2
|
Run 3
|
Run 4
|
Average
|
| ROR with 1 Mongrel |
88.1
|
85.1
|
84.9
|
84.8
|
85.7
|
| ROR with Passenger |
85.2
|
97.3
|
86.3
|
84.2
|
88.2
|
| ROR with Passenger and Ruby Enterprise |
89.0
|
99.2
|
98.9
|
98.6
|
96.4
|
Conclusion
CodeIgniter is over twice the speed of the Zend framework in all cases and CakePHP is a lot slower than the other two PHP frameworks. I do admit that considering my experience with CodeIgniter, I might have inadvertently set it up optimally without doing the same to the other two frameworks. I have posted the files and I welcome comments from “the experts”.
Update
We have decided to use the Zend framework so obviously outright performance is not the only factor in the choice of framework. The results we are currently getting are fast enough for us and in our existing application, the database is the bottleneck not the PHP code.
I have also done a few more tests on a production-grade dual-core server with 2Gb of RAM running Centos 5. This time, all benchmarks were from a seperate client computer on the same network. I used the exact same files as the previous results except for CakePHP where I used the recently released RC2. The results follow.
- No PHP code cache
All frameworks used an ORM (of sorts, in the case of codeigniter). The Zend Framework used Zend_DB_Table and CodeIgniter used ActiveRecord.
|
Run 1
|
Run 2
|
Run 3
|
Run 4
|
Average
|
| Baseline HTML |
3431.2
|
3311.8
|
3427.7
|
3395.0
|
3391.4
|
| Baseline PHP |
1912.1
|
1932.3
|
1983.3
|
1911.3
|
1934.7
|
| CakePHP |
15.6
|
15.6
|
15.6
|
15.6
|
15.6
|
| CodeIgniter |
83.5
|
83.0
|
82.0
|
83.2
|
82.9
|
| Zend Framework |
34.7
|
34.6
|
34.6
|
34.6
|
34.6
|
- With eAccelerator PHP code cache
All frameworks used an ORM (of sorts, in the case of CodeIgniter). The Zend Framework used Zend_DB_Table and CodeIgniter used ActiveRecord.
|
Run 1
|
Run 2
|
Run 3
|
Run 4
|
Average
|
| CakePHP |
36.0
|
36.1
|
36.1
|
36.2
|
36.1
|
| CodeIgniter |
383.3
|
377.9
|
371.8
|
385.2
|
379.5
|
| Zend Framework |
129.2
|
128.5
|
129.0
|
128.9
|
128.9
|
- With the Zend Platform
All frameworks used an ORM (of sorts, in the case of CodeIgniter). The Zend Framework used Zend_DB_Table and CodeIgniter used ActiveRecord.
|
Run 1
|
Run 2
|
Run 3
|
Run 4
|
Average
|
| CakePHP |
24.7
|
24.8
|
24.9
|
24.8
|
24.8
|
| CodeIgniter |
255.0
|
256.6
|
254.1
|
253.8
|
254.9
|
| Zend Framework |
83.5
|
84.4
|
83.8
|
83.4
|
83.8
|
From AVNet Labs
