DI Container Benchmark

Author:Máté Kocsis (@kocsismate90)
Repository:https://github.com/kocsismate/php-di-container-benchmarks
Generated:2020-12-02 22:58:48 UTC

Table of Contents


Introduction

In 2014, a really interesting benchmark about DI Containers for PHP was published on Sitepoint. Unfortunately, the implementation of the tests turned out to be quite controversial, so the benchmark itself wasn't very insightful.

I have been interested in the topic since then - and my curiosity was just growing after I had started to develop my own DI container, Zen - so I wanted to conduct another benchmark that also tries to measure real-life performance, while being as unbiased and reliable as possible. So here is my take! If you have any suggestion in mind about the benchmark or you want to add your container to the list, please create an issue or a pull request.

The examined containers are listed below along with some of their attributes:

Name Version Compiled/Dynamic Autowiring
aura/di 4.1.0 dynamic supported
chubbyphp/chubbyphp-container 1.3.1 dynamic not supported
level-2/dice 4.0.2 dynamic supported
joomla/di 1.5.1 dynamic supported
laminas/laminas-servicemanager 3.5.1 dynamic not supported
illuminate/container v8.17.0 dynamic supported
php-di/php-di 6.3.0 compiled supported
symfony/dependency-injection v5.2.0 compiled supported
yiisoft/yii2 2.0.39.3 dynamic supported
woohoolabs/zen 3.0.x-dev compiled supported

I'll try to give a vague definition below for some of the aforementioned notions:

A DI Container is compiled if it can be generated into a new class for production usage from where container entries then can be fetched. It means that your dependency graph is resolved during build time. This technique usually results in a very fast DIC, because there is no need for any Reflection or configuration when consuming the container. Dynamic containers however resolve your dependency graph Just-In-Time thus they are by design slower compared to the compiled ones.

A DI Container supports autowiring if it can be configured to automatically inspect and resolve at least some non-trivial subgraphs of the full dependency graph - no matter if the resolution takes place build time or run time. Otherwise all dependencies have to be resolved manually which is probably done as configuration. In this case, a DI Container does not support autowiring.

Essentially, dynamic containers usually need less attention during development than compiled ones, while containers which support autowiring usually need much less configuration than the ones without autowiring capabilities.


Method

Each container is given 6 tasks (Test Suites) where they have to create or fetch object graphs of different sizes (100, 1, or 1000 objects, respectively). For this purpose, containers are configured either to always instantiate objects (this is usually called as Prototype scope) or to instantiate objects only at the first retrieval and return the same instance on the subsequent calls (which is usually referred to as Singleton scope or shared services).

There are 3 main types of Test Cases: "Cold" ones measure performance including autoloading and bootstrap time of containers. "Warm" ones measure performance excluding autoloading and bootstrap time equally, while "Hot" ones additionally can warm up their caches before the measurements. The time of script compilation is always excluded from the results due to OPcache. Retrieved objects are always autoloaded (via preloading) prior to running the tests.

Each Test Suite contains three Test Cases which define how many times the main task has to be repeated in order to simulate different usage patterns. This number ranges from 10 to 100 000. Furthermore, all Test Cases are performed 30 times (this is referred to as "runs") in order to improve the accuracy of measurements. The median of these results is displayed in the final results.


Setup

The benchmark is run on AWS EC2, using a c5.large instance. The operating system on the host is Ubuntu 20.04. PHP 8.0 is running in a Docker container with OPcache enabled and autoloader optimized (using authoritative mode). Preloading is used for loading all classes besides the container-specific ones in order to keep the memory and autoloading overhead as small as possible. The code which performs the measurements is automatically generated in order to minimize the number of unnecessary instructions. During the measurements, a PHP-FPM script served by nginx is executed each time. This is needed because a production environment is simulated much better this way than in the CLI.

The examined DI Containers are configured for production usage as if it was probably done in case of a big project. That's why I took advantage of autowiring capabilities when possible.


Results

Test Suite 1: Instantiating an object graph of moderate size - Singleton scope

In this Test Suite, containers have to instantiate the same object graph of 100 objects (defined as Singletons) 1000 and 10 000 times. The first test case includes autoloading and bootstrap time of the containers in the measurements. The second test case excludes these. The third one warms up the container before the measurements.

1 000 iterations, container bootstrap time included
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Zen 0.09 100% 0.356 100%
2 Symfony 0.204 227% 0.947 266%
3 PHP-DI 0.364 404% 1.594 448%
4 Dice 0.427 474% 0.722 203%
5 ServiceManager 0.582 647% 1.388 390%
6 Chubbyphp 0.758 842% 1.353 380%
7 Yii2 Container 0.771 857% 1.782 501%
8 Joomla 1.021 1134% 2.088 587%
9 Laravel 1.156 1284% 1.618 455%
10 Aura 1.64 1822% 1.386 389%

1 000 iterations, container bootstrap time excluded
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Zen 0.076 100% 0.356 100%
2 Symfony 0.079 104% 0.947 266%
3 Chubbyphp 0.12 158% 1.353 380%
4 PHP-DI 0.145 191% 1.594 448%
5 ServiceManager 0.16 211% 1.388 390%
6 Joomla 0.204 268% 2.088 587%
7 Yii2 Container 0.318 418% 1.782 501%
8 Dice 0.41 539% 0.722 203%
9 Laravel 0.567 746% 1.618 455%
10 Aura 0.653 859% 1.386 389%

100 000 iterations, container already warmed up
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Zen 3.811 100% 1.868 100%
2 Symfony 4.099 108% 2.459 132%
3 ServiceManager 4.938 130% 2.9 155%
4 PHP-DI 4.993 131% 3.107 166%
5 Chubbyphp 5.071 133% 2.865 153%
6 Aura 5.446 143% 2.897 155%
7 Yii2 Container 5.453 143% 3.294 176%
8 Dice 6.657 175% 2.344 125%
9 Joomla 13.151 345% 3.599 193%
10 Laravel 26.907 706% 3.13 168%

Test Suite 2: Instantiating an object graph of moderate size - Prototype scope

In this Test Suite, containers have to instantiate the same object graph of 100 objects (defined as Prototypes) 1000 and 10 000 times. The first test case includes autoloading and bootstrap time of the containers in the measurements. The second test case excludes these. The third one warms up the container before the measurements.

100 iterations, container bootstrap time included
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Zen 0.561 100% 0.342 100%
2 Symfony 0.726 129% 0.933 273%
3 Chubbyphp 2.093 373% 1.339 392%
4 Joomla 3.298 588% 2.074 606%
5 ServiceManager 3.538 631% 1.374 402%
6 Dice 4.224 753% 0.601 176%
7 Yii2 Container 8.127 1449% 1.612 471%
8 Laravel 19.968 3559% 1.611 471%
9 Aura N/A N/A N/A N/A
10 PHP-DI N/A N/A N/A N/A

100 iterations, container bootstrap time excluded
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Zen 0.545 100% 0.342 100%
2 Symfony 0.559 103% 0.933 273%
3 Chubbyphp 1.501 275% 1.339 392%
4 Joomla 2.508 460% 2.074 606%
5 ServiceManager 3.143 577% 1.374 402%
6 Dice 4.204 771% 0.601 176%
7 Yii2 Container 7.694 1412% 1.612 471%
8 Laravel 19.376 3555% 1.611 471%
9 Aura N/A N/A N/A N/A
10 PHP-DI N/A N/A N/A N/A

100 iterations, container already warmed up
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Zen 0.515 100% 0.342 100%
2 Symfony 0.523 102% 0.933 273%
3 Chubbyphp 1.456 283% 1.339 392%
4 Joomla 2.47 480% 2.074 606%
5 ServiceManager 3.065 595% 1.374 402%
6 Dice 3.961 769% 0.672 197%
7 Yii2 Container 7.552 1466% 1.648 482%
8 Laravel 19.302 3748% 1.611 471%
9 Aura N/A N/A N/A N/A
10 PHP-DI N/A N/A N/A N/A

Test Suite 3: Instantiating a large number of small object graphs, Singleton scope

In this Test Suite, containers have to instantiate 1000 different objects (defined as Singletons) 100 times. The first test case includes autoloading and bootstrap time of the containers in the measurements. The second test case excludes these. The third one warms up the container before the measurements.

100 iterations, container bootstrap time included
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Zen 4.406 100% 2.053 100%
2 Symfony 5.189 118% 2.61 127%
3 ServiceManager 6.274 142% 3.011 147%
4 Chubbyphp 6.307 143% 2.994 146%
5 Yii2 Container 6.952 158% 3.482 170%
6 PHP-DI 7.938 180% 3.563 174%
7 Dice 8.383 190% 3.779 184%
8 Aura 10.419 236% 3.162 154%
9 Joomla 14.671 333% 3.747 182%
10 Laravel 29.28 665% 3.276 160%

100 iterations, container bootstrap time excluded
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Zen 4.374 100% 2.053 100%
2 Symfony 5.028 115% 2.61 127%
3 Chubbyphp 5.695 130% 2.994 146%
4 ServiceManager 5.948 136% 3.011 147%
5 Yii2 Container 6.478 148% 3.482 170%
6 PHP-DI 7.69 176% 3.563 174%
7 Dice 8.277 189% 3.779 184%
8 Aura 9.486 217% 3.162 154%
9 Joomla 13.981 320% 3.747 182%
10 Laravel 28.583 653% 3.276 160%

100 iterations, container already warmed up
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Zen 4.202 100% 2.076 100%
2 Symfony 4.486 107% 2.632 127%
3 PHP-DI 5.33 127% 3.586 173%
4 Chubbyphp 5.431 129% 3.028 146%
5 ServiceManager 5.453 130% 3.027 146%
6 Yii2 Container 5.564 132% 3.875 187%
7 Aura 5.879 140% 3.555 171%
8 Dice 7.027 167% 4.161 200%
9 Joomla 13.886 330% 3.781 182%
10 Laravel 27.221 648% 3.299 159%

Test Suite 4: Instantiating a large number of small object graphs - Prototype scope

In this Test Suite, containers have to instantiate 1000 different objects (defined as Singletons) 10 times. The first test case includes autoloading and bootstrap time of the containers in the measurements. The second test case excludes these. The third one warms up the container before the measurements.

10 iterations, container bootstrap time included
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Zen 0.917 100% 0.61 100%
2 Chubbyphp 2.022 221% 1.605 263%
3 Symfony 2.614 285% 1.52 249%
4 Dice 3.214 350% 1.437 235%
5 Joomla 3.344 365% 2.34 383%
6 ServiceManager 3.524 384% 1.64 269%
7 Yii2 Container 7.042 768% 1.909 313%
8 Laravel 14.935 1629% 1.839 301%
9 Aura N/A N/A N/A N/A
10 PHP-DI N/A N/A N/A N/A

10 iterations, container bootstrap time excluded
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Zen 0.903 100% 0.612 100%
2 Chubbyphp 1.429 158% 1.605 262%
3 Symfony 2.416 268% 1.52 248%
4 Joomla 2.508 278% 2.34 382%
5 ServiceManager 3.111 345% 1.64 268%
6 Dice 3.183 352% 1.437 235%
7 Yii2 Container 6.614 732% 1.909 312%
8 Laravel 14.406 1595% 1.839 300%
9 Aura N/A N/A N/A N/A
10 PHP-DI N/A N/A N/A N/A

10 iterations, container already warmed up
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Zen 0.806 100% 0.624 100%
2 Chubbyphp 1.337 166% 1.617 259%
3 Symfony 1.743 216% 1.539 247%
4 Joomla 2.44 303% 2.351 377%
5 Dice 2.588 321% 1.815 291%
6 ServiceManager 3.01 373% 1.652 265%
7 Yii2 Container 6.337 786% 2.298 368%
8 Laravel 14.152 1756% 1.85 297%
9 Aura N/A N/A N/A N/A
10 PHP-DI N/A N/A N/A N/A

Test Suite 5: Instantiating a single, large sized object graph - Singleton scope

In this Test Suite, containers have to instantiate the same object graph of 1000 objects (defined as Singletons) 100 and 10 000 times. The first test case includes autoloading and bootstrap time of the containers in the measurements. The second test case excludes these. The third one warms up the container before the measurements.

100 iterations, container bootstrap time included
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Zen 0.35 100% 0.376 100%
2 Symfony 0.461 132% 0.933 248%
3 PHP-DI 0.994 284% 2.136 568%
4 ServiceManager 1.184 338% 1.775 472%
5 Chubbyphp 1.188 339% 1.598 425%
6 Joomla 1.471 420% 2.351 625%
7 Yii2 Container 2.88 823% 3.708 985%
8 Laravel 3.312 946% 3.029 805%
9 Dice 3.566 1019% 4.768 1267%
10 Aura 6.847 1956% 4.242 1127%

100 iterations, container bootstrap time excluded
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Symfony 0.299 100% 0.933 100%
2 Zen 0.334 112% 0.376 40%
3 Chubbyphp 0.618 207% 1.598 171%
4 Joomla 0.678 227% 2.351 252%
5 PHP-DI 0.754 252% 2.136 229%
6 ServiceManager 0.774 259% 1.775 190%
7 Yii2 Container 2.451 820% 3.708 397%
8 Laravel 2.7 903% 3.029 325%
9 Dice 3.569 1194% 4.768 511%
10 Aura 5.843 1954% 4.242 455%

10 000 iterations, container already warmed up
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Zen 0.385 100% 0.531 100%
2 Symfony 0.415 108% 1.088 205%
3 ServiceManager 0.49 127% 1.93 364%
4 PHP-DI 0.505 131% 2.29 432%
5 Chubbyphp 0.507 132% 1.752 330%
6 Yii2 Container 0.55 143% 3.862 728%
7 Aura 0.551 143% 4.396 828%
8 Dice 0.753 196% 5.255 990%
9 Joomla 1.258 327% 2.505 472%
10 Laravel 2.697 701% 3.183 600%

Test Suite 6: Instantiating a single, large sized object graph - Prototype scope

In this Test Suite, containers have to fetch an object graph of 100 objects (defined as Singletons) 1000, 10 000 and 100 000 times. Neither autoloading time, nor bootstrap time is included in the measurements.

10 iterations, container bootstrap time included
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Zen 0.878 100% 0.375 100%
2 Symfony 1.046 119% 0.939 251%
3 Chubbyphp 2.87 327% 1.565 418%
4 ServiceManager 4.3 490% 1.754 468%
5 Joomla 4.341 494% 2.318 619%
6 Dice 7.507 855% 3.492 932%
7 Yii2 Container 10.225 1165% 3.55 947%
8 Laravel 21.517 2451% 3.066 818%
9 Aura N/A N/A N/A N/A
10 PHP-DI N/A N/A N/A N/A

10 iterations, container bootstrap time excluded
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Zen 0.858 100% 0.375 100%
2 Symfony 0.871 102% 0.939 251%
3 Chubbyphp 2.254 263% 1.565 418%
4 Joomla 3.546 413% 2.318 619%
5 ServiceManager 3.904 455% 1.754 468%
6 Dice 7.497 874% 3.492 932%
7 Yii2 Container 9.805 1143% 3.55 947%
8 Laravel 20.815 2426% 3.066 818%
9 Aura N/A N/A N/A N/A
10 PHP-DI N/A N/A N/A N/A

100 iterations, container already warmed up
Rank Container Time (ms) Time (%) Peak Memory (MB) Peak Memory (%)
1 Zen 5.72 100% 0.376 100%
2 Symfony 5.76 101% 0.941 250%
3 Chubbyphp 18.833 329% 1.567 416%
4 Joomla 30.994 542% 2.32 617%
5 ServiceManager 34.588 605% 1.756 467%
6 Dice 49.543 866% 4.211 1119%
7 Yii2 Container 82.83 1448% 3.91 1039%
8 Laravel 201.605 3525% 3.067 815%
9 Aura N/A N/A N/A N/A
10 PHP-DI N/A N/A N/A N/A


Conclusion

Keep in mind that in a well-architected application you won't call your DI Container hundreds or even thousands of times because ideally there should be as few composition roots as possible (but there is a good chance of needing the container in other places of the application layer - e.g. in your middleware or bootstrap files). That's why most results are exaggerated - you probably won't see milliseconds of difference between the fastest and the slowest DIC in the real life.

To sum up, when choosing a container it only depends on your needs which one suits your project best: if you have a performance-critical application then you probably want to choose a compiled container. If maximum performance is not required, but you develop a big and complex system then I would recommend a dynamic container with autowiring capabilities. Otherwise you can go with simpler containers.