Performance benchmark of popular PHP frameworks

Posted by on Apr 23, 2013 in PHP Programming | 68 Comments

There are many assumptions around performance of different PHP frameworks. I frequently hear strong opinions about superiority X over Y in this context. There are companies writing new PHP frameworks from scratch because available solutions are too slow for them. What does it really mean? Does the framework performance matters? Before answering this questions lets check how slow is your framework!

Performing a representative benchmark across different framework is not an easy task. There are multiple ways to use each of them. Every use case will give different reading. Lets take routing as an example. Zend1 by default doesn’t need a routing file. It’s happy to use “/controller/action” pattern. On the other hand Symfony2 comes with a routing configuration. The file has to be read and parsed. That obviously takes some additional CPU cycles but does it mean Symfony2 routing is slower then Zend1? The answer is (obviously) no.

I benchmarked “quick start” projects. That gives some idea on what is the base line for every framework and makes it possible to reproduce my tests (and argue against them).
Code was hosted on Amazon EC2 medium instance. I installed PHP-APC to avoid disc access and code parsing. I also made sure there is no I/O on Apache2 or application level. I set logs and cache paths to “/dev/shm/”. I tweaked projects to make them return roughly the same amount of data (10KB). All virtual hosts had the same mod_rewrite rules. AllowOveride was set to None.

Benchmarked frameworks:

Requests per second from Apache Benchamrk with c=20 and n=500.

Framework Req/Sec
Phalcon 822.96
Slim 399.83
Kohana 217.34
Code Igniter 187.78
Silex 179.01
Laravel 135.9
YII 123.5
Fuel PHP 116.34
Hazaar MVC 103.53
Zend 1 103.02
Cake PHP 54.97
Nette 53.48
Symfony2 39.22
Zend 2 36.1

PHP Frameworks benchmark

I’m not surprised, Slim is the fastest because it’s a micro framework. The Quick Start project didn’t use any templates or layout which obviously contributed to the reading.
Zend1 is twice faster than Symfony2 and Zend2 but in my experience the number will quickly go down in a real live setup.

Frameworks should speed up development, performance is a secondary concern. Zend 2 and Symfony2 could do better but it’s not bad. There are ways to improve those numbers on production servers. Don’t reinvent the wheel, learn and use frameworks. There are various options which balance between performance and features.

ps. If you have space for more I would like to recommend you great research on performance of web frameworks by techempower.

ps2. Thank you for your suggestion Ron. Phalcon is the new winner.

68 Comments

  1. Wojtek
    23/04/2013

    It’s hard to agree on your last sentence “If you need more performance buy another web server. It’s cheeper then development.” – you can not be more wrong.

    You can’t make your app to run faster by adding more servers, you can just make it less slow. If ab -c 1 -n 1000 www_yourapp_com tells you that average response time is 300ms, you can’t bring it down to 30ms by adding more servers :(

    You need to design your app with performance in mind from day 1, and if you want to use frameworks, you have to accept additional costs (performance, dev and infra).

    PS. I’m not talking about home/small projects that runs on +/-20 cores in total.
    PSS. For small project/teams, I’m all for frameworks like symfony2, as long as it’s not adding 10 layers of abstraction around everything.

    Reply
    • Lukasz Kujawa
      23/04/2013

      Well spotted! It obviously won’t magically drop down from Xms to Yms by adding more servers. That will only improve capacity. I will rephrase the sentence to avoid confusion ;-) Thank you for that.

      With reagards to the design you need a good caching strategy. In my opinion, it doesn’t matter that much what framework are you using. Most of them offer enough entry points. High traffic environments introduce unique challenges which are difficult to mitigate with “the best framework”. It’s often a combination of different technologies.

      Reply
  2. EllisGL
    23/04/2013

    Would like to see more frameworks in review here.
    Kohana
    Yii
    Laravel
    Fat-Free
    Hazaar MVC
    Fuel PHP
    Zikula

    Reply
    • Lukasz Kujawa
      23/04/2013

      Thank you Ellis for your brilliant suggestions. I will try to add all of them.

      Reply
    • Tom
      16/01/2014

      Would like to see you benchmark the Lithium framework through the same process. Keep in mind everyone who goes to run a benchmark is going to end up with different results…But for poops and giggles, it’d be interesting to see. I do agree with the point made that the primary concern is not performance. You are hitting the nail on the head there. It is development speed and maintainability. Hardware gets faster and faster and cheaper and cheaper. So to the point of Wojtek there, yes you can magically take the same app from 300ms to 30ms. Not by adding more servers perhaps, but by changing your code and/or your hardware. Of course your already fast codebase will run faster.

      Reply
  3. Jacques Marques
    23/04/2013

    Hi,

    I suggest you test the YII Framework, It is a very fast and complete framework.

    Reply
  4. Phil Kingston
    23/04/2013

    I can see the attraction of comparing the performance when writing systems that do “basic” tasks, but I’m always concerned about not being able to see the bigger picture. I have experience of four of the frameworks tested here and can state that all of them vary in functionality drastically. My personal favourite framework from the list is certainly not at the top, but would much rather use it as I know about the functionality.

    Basically, don’t judge a book by it’s cover! Don’t pick a framework JUST because it’s fast. Do your research. Get to know what each framework is good at and understand it’s functionality (and limitations) before rushing in.

    Reply
    • Lukasz Kujawa
      23/04/2013

      I agree 100%. This test is not about judging which framework is better. I was curious what is the performance difference between them. As I said in the post – those values can move up and down depends on how the framework is set. My favourite frameworks are at the bottom but I expected them to be there. It’s logical – more features, more code, more CPU cycles consumed.

      On the other hand there are some occasions where performance is in the first place. When that is the case it’s good to have a rough idea what options are available. I usually go for Zend but now I’m considering two frameworks in parallel: Zend for the web application and Slim for API access.

      Reply
  5. Melissa
    24/04/2013

    How about including Silex to the list as competition for Slim? It is also a micro framework and uses symfony 2 components.

    Reply
    • Lukasz Kujawa
      24/04/2013

      Thank you for your suggestion Melissa. I wasn’t aware of this framework but it’s definitely worth checking.

      Reply
    • Lukasz Kujawa
      24/04/2013

      Ok, I’ve checked it and it gave me 187.78 req/sec. I was expecting more so I triple checked everything (including another Slim benchmark). It looks like it’s what it’s.

      Reply
  6. Martin Hlaváč
    24/04/2013

    Have you properly configured production settings for quick starts projects? As Symfony 2 is much faster with production settings and APCClassLoader… On my websites the difference is almost 5 times faster performance up to 160 requests/s. Same goes for any framework that uses configurations and caches things.

    Have you considered putting these quick starts codes on github? It would be nice to see what each on of them does in time. If you would automatize these you could say if framework is getting faster or slower in time.

    Reply
    • Lukasz Kujawa
      24/04/2013

      Hello Martin. I’m not a Symfony expert but I believe I benchmarked against the production setup. All logs and cache were directed to “/dev/shm” so there wasn’t any IO during testing. I’ve done some test against app_dev and it was much slower. 160 req/sec is quite impressive performance for such a big framework. I’m guessing you got those results on a dedicated server? EC2 medium instance has limited CPU. I don’t think it would be possible to squeeze that much from Symfony in this environment without extensive caching. That would give inaccurate results because I wanted to see how long does it take to actually execute the code.

      Most of the code I tested didn’t require much configuration. It was basically an “out of the box” skeleton project adjusted to log to memory and return 10k page. I must admit that Symfony required little bit more attention. If you can provide me with a setup you believe should run fast I’m happy to test it. We can also link it here so anybody can have a look.

      Reply
      • Leric
        25/04/2013

        Symfony2 include too many bundles by default(in standard dist), something like Doctrine, Aop, DiExtra, SecurityExtra, other framework done even have such features, and many application might never need them.

        I have a ‘Hello World’ benchmark earlier, on my laptop with an i5 cpu, running php5.4 and nginx, I got 928 rps running symfony2, about 43% of codeigniter’s score. I wonder why your test is so slow.

        Reply
        • Lukasz Kujawa
          25/04/2013

          Send me your benchmark and we can have a look. My guess is the difference is in caching? Perhaps your test wasn’t executing too much of the Symfony code? This is only guess.

          Reply
          • Leric
            16/05/2013

            I did the test again, and got roughly the same ratio of speed of different frameworks.
            The link on my name is to the post of my test result.

  7. bazo
    24/04/2013

    Could you test the nette framework?

    Reply
    • Lukasz Kujawa
      24/04/2013

      With a pleasure ;) I got 53.48 req/sec.

      Reply
      • Vojta
        02/10/2013

        Hi Lukasz!
        Could you specify which router you used in Nette fw? thx

        Reply
  8. parjo
    25/04/2013

    Would you kindly divide frameworks that can be use in personal project, small, medium or big projects?

    Reply
    • Lukasz Kujawa
      25/04/2013

      Interesting question but I’m afraid that would be highly subjective exercise. I don’t think you match a framework with a project size. I work with people who love Symfony 2 and they use it for everything. I’m using Zend and I would also go for it with any project size. For some special tasks I might use Slim Framework. I also recommended it to my sys-admin friends to help them structure their small projects.

      Reply
  9. Chris
    25/04/2013

    Ouch ZF 2 so slow :( i experienced this too in mine benchmarks..

    Reply
  10. Ron
    25/04/2013

    How would Phalcon stack up in the comparison?

    Reply
    • Lukasz Kujawa
      28/04/2013

      Thank you for that. I wasn’t aware of this framework. It’s great to know somebody made an effort to create it purely in C. I’m going to use it with some future project!

      Reply
  11. Alexander Makarov
    28/04/2013

    Looks like Yii was run in debug mode. Have you set YII_DEBUG to false and followed performance guidelines?

    Reply
    • Lukasz Kujawa
      28/04/2013

      Hi Alexander. I looked into index.php and it doesn’t look like the debug was on

      // change the following paths if necessary
      $yii=dirname(__FILE__).'/../../framework/yii.php';
      $config=dirname(__FILE__).'/protected/config/main.php';

      // remove the following line when in production mode
      // defined('YII_DEBUG') or define('YII_DEBUG',true);

      require_once($yii);
      Yii::createWebApplication($config)->run();

      I benchmarked YII and it gave me 155.68 this time but Laravel got me higher results as well. Perhaps the virtual machine is less busy Today.

      Reply
  12. Matt Robinson
    28/04/2013

    A few questions:
    With the composer-dependent frameworks, did you run ‘composer dumpautoload –optimize’?
    Have you got these benchmarks in a git repository? Not sure what a “quick start” means in this context.
    Is the Laravel benchmark 3 or 4? Versions on all of them would be appreciated.

    Reply
    • Lukasz Kujawa
      29/04/2013

      Hello Matt, the version of tested Laravel was 3.2.1 (it’s returned by the download button on the official website). With regards to the dumpupload I didn’t do it. I checked it now with Zend1 which is heavily using the autoloader. There wasn’t any significant difference. Actually the reading gone down but I blame EC2 for it ;) With regards to the “quickstart” it was the root page of freshly installed framework.

      Reply
  13. Brian
    29/04/2013

    Hello, could you test my framework please? I wanted to know the performance compared to other frameworks. (you don’t have to add it to list).

    It’s in github: luminairre/Crayd2 (for some reason I can’t put links, so I put it up there)

    I made it for my friends that says Zend Framework is difficult to learn, so I made something simple, and having controllers / views work like Zend, so it’s easy for them to use Zend Framework at later time.
    But those guys ends up using my framework instead of Zend so..

    Anyway, I’m sorry there are no documentations, but it should be ready to use.

    Thanks!

    Reply
  14. Andrea Cardinale
    03/05/2013

    Hello, I am a Yii developer and if you want I can take a look at the code used.

    Reply
    • Lukasz Kujawa
      03/05/2013

      Hi Andrea, thank you for your offer. I will let you know once/if I find enough people to cover for the rest of the frameworks. Once again thank you and have a nice day!

      Reply
  15. Steve
    06/05/2013

    including phalcon is a bit of a cheat with it being C, but a very interesting project that I am going to look into. I feel sorry for anybody on restricted shared hosting that can’t install any modules!

    My main issue is with ZF2. I have done similar hello world comparisons and also found ZF2 to be a lot slower than ZF1.

    Is anyone else concerned about the future of ZF when the new version is one half or even one third of the speed, am I missing something? I know you say that the primary role of frameworks is to speed dev time, but this still feels like an incredible step backwards, especially as ZF1 was widely criticised for being bloaty and slow… am I missing something?

    Reply
  16. Marte
    17/05/2013

    What about Seagull? seagullproject (dot) org It’s a mature player. It will be interesting to have it benchmarked against the new ones.

    Reply
  17. Gary Brooks
    17/05/2013

    Would you be willing to review Joomla Framework?

    Joomla framework is newer “framework” as compared to what is on this list, but I believe we have some good engineers building off the base Joomla already had in motion, could you add us to your list for testing?

    Reply
  18. Roman
    19/05/2013

    Could you please put these quick starts codes on github?

    Reply
  19. nirleka
    08/06/2013

    Well, because you testing phalcon, maybe you can also test yaf framework

    Same as falcon, yaf also php framework as c extension.
    Yaf take more minimalistic approach that Phalcon (no template, no ORM)

    Reply
  20. Antonin Januska
    12/06/2013

    Can you throw Laravel 4 into the mix? :)

    And can you detail your benchmarking process?

    Reply
  21. George Jempty
    14/06/2013

    I briefly worked somewhere 6 years ago where performance was critical as they needed to support two of the top 10 or 20 visited websites in the world if I’m not mistaken. Problem was our application couldn’t support 10 hits a second and these websites said it was possibly they could get 10 THOUSAND hits a second. We were using Symfony and I did some profiling with XDebug, and found that for the one site 10′s of thousands of function calls were being made, the other 100′s of thousands, merely to render the landing page. I later heard that after I abandoned that sinking ship, the CTO blamed it on shoddy development by the offshore team. However I know that Symfony was largely to blame, I don’t know if they’ve changed this, but as it recursed through directories it was idiotically trying to determine if those directories were associate with a half-dozen or ten version control systems, and we had over a hundred directories, so that was about 1000 unnecessary function calls right there.

    Reply
  22. George Jempty
    15/06/2013

    Sorry for the second post in as many days, but maybe you should consider including “jackrabbitmvc”? I don’t know if it’s “popular” enough, but in it’s previous incarnation as “barebonesmvc-php” it’s been downloaded from google code over 1000 times per year on average; see: I can provide a link per WP but if you google “barebonesmvc-php downloads list” the download stats page was the first one that came up for me.

    I just completed some rough benchmarking on my home system this morning and the summary is, with or without templates, jackrabbitmvc can serve over 300 requests per second, requests that entail calling json_encode on about 5K of data I had laying around ;) The system specs are as follows:

    Windows 7 Home Premium
    Intel i3-3220 CPU @3.3 Ghz
    6 GB Ram
    Xampp-Lite, PHP Version 5.4.7

    I’m sure your tests are *way* more scientific than mine, but mine would place jackrabbitmvc in a close 3rd place. Thanks for your consideration and let me know if I can provide any usage help beyond that at: “use dot jackrabbitmvc dot com”

    Reply
    • Lukasz Kujawa
      20/06/2013

      Thank you for your suggestion. I would love to give it a try but I have so much work this month I can’t do anything else :(

      Reply
      • George Jempty
        29/06/2013

        OK, thanks for your consideration, maybe later this summer?

        Reply
  23. Patrick Forget
    07/07/2013

    So from my understanding this basically tests how the MVC components compare.

    I would much like to see more comparisons between other components of each framework. Like memory consumption, numbers of files included.

    Because it might be interesting to use Slim’s MVC speed and some of ZF 2 modules, or would Zend still slow down things.

    Any thoughts on this?

    Reply
  24. Tim van Steenbergen
    16/07/2013

    Hi, Same question for Zend Framework as for Yii:
    Was Zend run in debug mode?

    Reply
  25. krishnaTORQUE
    21/07/2013

    I would like to know more about CAKEPHP.
    why people not talking about it?
    my another question is — If ZEND is that much slow then why facebook and other big sites are using this?
    please let me know, thank you.

    Reply
    • Lukasz Kujawa
      23/07/2013

      Is Facebook really using Zend Framework? I don’t know for sure but I assume they actually don’t. Zend is not compatible with HipHop they use to compile PHP code into binaries.

      Frameworks should speed up development not the execution time (with some exceptions). In most cases performance is a secondary concern.

      Reply
  26. Romain L
    28/07/2013

    Hi Lukasz!
    First of all, thanks for this really useful article, as well as for your effort in trying to stay as objective as possible!

    A had a little request for you actually, as the new major Laravel update (Laravel 4) was released about 2 months ago: would it be possible for you to update your benchmark to this new version? Backward compatibility should not be a problem, so I bet you won’t even have to create a new “quick start” project ;)

    Thanks in advance!

    Reply
    • Lukasz Kujawa
      29/07/2013

      Hello Romain, thank you for your suggestion and I would love to do it but I’ve lost access to the test environment. Obviously not only Laravel has been updated in the last few months so I would have to retest everything else.

      Sorry for not being able to help you.

      Reply
  27. Philippe
    02/08/2013

    Zend is too damn slow! Never gonna use it!

    Reply
    • Lukasz Kujawa
      03/08/2013

      I use Zend every day despite it’s slowness. There are few tricks to make it faster and generally speaking it’s a good framework.

      Reply
      • Patrick Forget
        13/08/2013

        Lukasz, when you say you use Zend every day.

        Are you talking about it’s MVC components? Or which modules do you use?

        I personally, avoid any of its database abstraction and now trying out Slim to replace Zend’s MVC component…. I think that’s one of the strenghts of Zend you can pick and choose what you use, therefore should not be ignored completely :)

        Reply
        • Lukasz Kujawa
          14/08/2013

          Hello Patrick, I use PropelORM instead of Zend’s database abstraction. With regards to the components I use the MVC, Tables (with custom validators and decorators) and all sort of helpers and plugins.

          Reply
  28. Tinku Pandey
    25/08/2013

    Is phalconphp faster than vanila php?

    Reply
    • Jason
      31/12/2013

      No, but it’s roughly as close as you’re going to get. Phalcon compiles in as a PHP module, effectively providing “native” PHP functions, the same way PHP modules like php5-gd and php5-mcrypt, etc. do. It “becomes part of PHP”, as it were; you don’t have to include anything. It’s just there waiting to be used.

      Reply
  29. Mika
    01/09/2013

    Don’t forget french framework as the mkframework
    It’s a security and easily framework, you can generate CRUD, menu, private access… in few clics ;)

    Reply
  30. Pragnesh Karia
    02/09/2013

    Nice article , I really appreciate this.

    Is any one here can , suggest me how to improve performance on ZF2.

    Thanks in Advance..!!

    Reply
  31. Nick Koster
    09/09/2013

    You should update the Laravel benchmark with the new Laravel 4. It’s faster.

    Reply
  32. Fernando Uchiyama
    26/09/2013

    I can be totally wrong, but I have the impression that development time is also much slower when you go down the list. So you loose performance and also take longer (much longer) to develop. In the other hand, most of people may say you’d gain in terms of code quality or tests…

    All of this stuff is difficult to say… kind of a philosophical matter.

    I appreciate simple and clean code, less development time and elaborated code and system documentation. I like to concentrate on business rules instead of talking about code and code design. But I’m kinda of an old guy…

    Reply
    • Lukasz Kujawa
      26/09/2013

      I disagree. More robust (slow) frameworks = more code to reuse and less wheels to reinvent.

      Reply
  33. Sarim Khan
    28/09/2013

    Great job man (y)
    Bigger frameworks like symfony2 can be tweaked to provide faster performance IMO. I’m trying to learn symfony2 and i benchmarked again project i’m working on. This page uses doctrine orm for do some query and also use a twig template.

    Result: Requests per second: 235.50 [#/sec] (mean)

    I think this i can still improve this by implementing symfony book’s tweaks for performance.

    Again Thank you for going through this trouble for the php community.

    Reply
  34. Piotr Pasich
    31/10/2013

    Hi,

    really nice article. I’d like to ask which exactly version of symfony 2 did you test.

    Reply
  35. RaviTeja.K
    17/03/2014

    hey can you add F3 and laravel framework to this list. I feel that i f3 might go second but fall behind phalcon, and i even want to know did you used opcode cache for all these tests

    Reply
  36. Luca
    24/04/2014

    Hi, good article!

    CAM you please add also to this benchmark the PHPixie framework?

    Reply
  37. Luca
    24/04/2014

    Hi, good article!

    Can you please add also to this benchmark the PHPixie framework?

    Reply
    • CJ
      02/05/2014

      Another vote for adding Fat Free Framekwork (F3) and PHPixie to this benchmark.

      Reply
  38. Alan Seiden
    10/06/2014

    You’ve made a sincere effort here. Could you tell us more about your methodology? As you say, it’s difficult to compare framework speed when there are so many variables, yet, perhaps contrary to your intention, people are relying on your results as the full truth.

    A basic question to start: what version of each framework did you use? (Some frameworks get faster as they mature; others may slow down. An example of speeding up was Zend Framework 2 [one that I know pretty well], which started slow with their DI component, then later sped up without it.)

    Did you use each framework’s “quick start” application? It seems that you tweaked each one to try to make them somewhat equal. Nevertheless, each quick start application may have been created to demonstrate different features, whether performance, ease of use, or enterprise capabilities. For example, ZF 2′s quick start application enables a translation service to demonstrate a multilingual setup. This will slow down the application, but as an optional setting, doesn’t mean the framework itself has to be slow. What I’m getting at: despite your best efforts, your results may actually be testing the speed of the quick start applications, not the frameworks alone–or perhaps a combination of both.

    You also touched on the fact that your framework choice, Zend, can be accelerated by production-ready performance configurations that are not reflected in the quick start application. Other frameworks may have similar techniques and recommendations not reflected here. (Perhaps framework makers should produce two quick start apps: one simple setup and one tweaked for performance, as a guide for experienced users and for benchmark testers.)

    I’ll be interested to hear your thoughts.

    Reply
  39. Waleed Gadelkareem
    24/06/2014

    Can you add Yii2 to the list?

    Reply

Leave a Reply