DI Container Benchmark

Table of Contents

Introduction

In 2014, a really interesting benchmark about DI Containers for PHP was published on Sitepoint. Unfortunately, the implementation of the tests turned out to be quite controversial, so the benchmark itself wasn't very insightful.

I have been interested in the topic since then - and my curiosity was just growing after I had started to develop my own DI container, Zen - so I wanted to conduct another benchmark that also tries to measure real-life performance, while being as unbiased and reliable as possible. So here is my take! If you have any suggestion in mind about the benchmark or you want to add your container to the list, please create an issue or a pull request.

The examined containers are listed below along with some of their attributes:

I'll try to give a vague definition below for some of the aforementioned notions:

A DI Container is compiled if it can be generated into a new class for production usage from where container entries then can be fetched. It means that your dependency graph is resolved during build time. This technique usually results in a very fast DIC, because there is no need for any Reflection or configuration when consuming the container. Dynamic containers however resolve your dependency graph Just-In-Time thus they are by design slower compared to the compiled ones.

A DI Container supports autowiring if it can be configured to automatically inspect and resolve at least some non-trivial subgraphs of the full dependency graph - no matter if the resolution takes place build time or run time. Otherwise all dependencies have to be resolved manually which is probably done as configuration. In this case, a DI Container does not support autowiring.

Essentially, dynamic containers usually need less attention during development than compiled ones, while containers which support autowiring usually need much less configuration than the ones without autowiring capabilities.

Method

Each container is given 6 tasks (Test Suites) where they have to create or fetch object graphs of different sizes (100, 1, or 1000 objects, respectively). For this purpose, containers are configured either to always instantiate objects (this is usually called as Prototype scope) or to instantiate objects only at the first retrieval and return the same instance on the subsequent calls (which is usually referred to as Singleton scope or shared services).

There are 3 main types of Test Cases: "Cold" ones measure performance including autoloading and bootstrap time of containers. "Warm" ones measure performance excluding autoloading and bootstrap time equally, while "Hot" ones additionally can warm up their caches before the measurements. The time of script compilation is always excluded from the results due to OPcache. Retrieved objects are always autoloaded (via preloading) prior to running the tests.

Each Test Suite contains three Test Cases which define how many times the main task has to be repeated in order to simulate different usage patterns. This number ranges from 10 to 100 000. Furthermore, all Test Cases are performed 30 times (this is referred to as "runs") in order to improve the accuracy of measurements. The median of these results is displayed in the final results.

Setup

The benchmark is run on AWS EC2, using a c5.large instance. The operating system on the host is Ubuntu 20.04. PHP 8.0 is running in a Docker container with OPcache enabled and autoloader optimized (using authoritative mode). Preloading is used for loading all classes besides the container-specific ones in order to keep the memory and autoloading overhead as small as possible. The code which performs the measurements is automatically generated in order to minimize the number of unnecessary instructions. During the measurements, a PHP-FPM script served by nginx is executed each time. This is needed because a production environment is simulated much better this way than in the CLI.

The examined DI Containers are configured for production usage as if it was probably done in case of a big project. That's why I took advantage of autowiring capabilities when possible.