How I Run a Laravel Web Server and Its Queue Workers in One Container

Introduction

A web application that only answers requests is a simple thing to package. Something arrives, the application works for a few hundred milliseconds, it replies, and it is done. One program, one job, one container. That is the arrangement container tooling was designed around, and when your software fits it, everything downstream — deployment, scaling, restarts, logging — falls into place without argument.

The application I had to package does not fit it. It answers HTTP requests, and it also sends email, calls back into client systems that are waiting on it, signs documents, and renders thumbnails of the files people upload. The first kind of work is shaped like a request. The second kind is not shaped like a request at all, and trying to make it one is how you get a user staring at a spinner while a PDF is signed.

So there are two kinds of long-lived work in one codebase, and the obvious container arrangement has no room for the second. That is the challenge this article is about: how both of them ended up running inside a single container image, why that was harder than it sounds, and what it cost.

A. Two processes that both refuse to end

Work that should not happen during a request gets deferred. The application writes down what needs doing and returns a response immediately; something else picks the work up afterwards and performs it. In this codebase there are twenty-one distinct kinds of deferred work, and the pattern in them is consistent: sending email to a user whose password just changed, notifying a client system that a certificate was approved, signing an uploaded document, generating a thumbnail. Almost all of it waits on something external — a mail server, another organisation's HTTP endpoint, a disk.

Performing that work requires a process that is running before the work arrives, because the work arrives at unpredictable times. It cannot be started by a request, since the whole point is that no request is waiting. It has to already be there, asking repeatedly whether anything needs doing.

Meanwhile, the web server is also a process that is running before any request arrives, waiting on a socket. Both processes are long-lived. Both must survive crashes. Both must be shut down carefully. But they are long-lived in different ways, and that difference is the whole problem. A web server that is killed mid-request loses one response, which the client will retry. A queue worker killed mid-job may have already charged something, already sent something, already written half a signature into a file.

B. Why one container makes this awkward

A container is a way of packaging a program together with everything it needs to run — the operating system libraries, the language runtime, the configuration — into a single image that runs the same way on any machine that can run containers. The strong convention, and it is a good one, is that a container runs one program. When the program exits, the container is finished. That is what makes containers easy to reason about: the health of the container is the health of the program.

Two programs break that equivalence immediately. Which one is the container? If the web server is healthy and every queue worker has died, is the container healthy? Nothing in the container runtime can answer that, because the runtime only watches one process.

The honest alternative is to build two images, or to run the same image twice with different commands — one deployment serving HTTP, another running workers. Many teams do exactly this, and for a large system it is the right answer. I decided it was not the right answer here, because the two deployments would share everything that actually matters: the same codebase, the same dependencies, the same configuration, the same database credentials, the same version. Every release would mean building and shipping the same code twice and hoping both halves landed together. A worker running last week's code against this week's database schema is a genuinely bad afternoon.

Against that, one image means one build, one version number, one thing to deploy, one stream of logs, and no possibility of the two halves drifting apart. The cost is that the container no longer supervises itself, and something inside it has to take that job.

C. The words this article uses

These terms appear throughout, and two of them collide in an unhelpful way, so they are worth settling first.

A queue is a list of work waiting to be performed. A job is one item on that list — in practice a small record saying which class to run and with what data. A worker is a process that repeatedly takes the next job off the queue, runs it, and asks for another. Here the queue is a table in the database, which means a job is a row and taking a job is a transaction.

Worker mode is an unrelated idea that unfortunately reuses the word. Normally a PHP application is loaded from scratch on every request: the framework boots, configuration is read, routes are registered, and all of it is discarded when the response is sent. Worker mode instead boots the application once inside a long-lived process and reuses it across many requests. The word "worker" here refers to a process holding a booted application, not a process running queued jobs. When this article says queue worker it means the second; worker mode always means the first.

PID 1 is the first process started inside a container. It matters more than its name suggests: when PID 1 exits, the container stops, whatever else is still running. It is also the process that receives the shutdown signal when something asks the container to stop, and it is responsible for passing that signal on.

A process manager, or supervisor, is a program whose only purpose is to start other programs, watch them, and restart them when they die. It does no work of its own.

A reverse proxy is a server placed in front of an application that receives requests from the outside world and forwards them inward. It usually terminates HTTPS, so the application behind it only ever speaks plain HTTP.

A layer is the result of one instruction in a build file; images are stacks of layers. The build cache reuses a layer when its instruction and inputs have not changed. A multi-stage build is one build file containing several independent builds, where a later stage copies selected files out of an earlier one and leaves the rest behind.

D. The shape of the answer

The container's first process is not the web server. It is a short script that prepares state — permissions, caches, a check that the database is reachable — and then replaces itself with a process manager. That manager owns three programs: the web server, seventeen queue workers, and a small monitor watching the backlog. The rest of this article is that arrangement, the build that produces it, and the trade-offs it makes on purpose.

E. Five obstacles in the way

Stated as a goal: I wanted to run one application that both answers HTTP requests and performs deferred work, out of a single image, and have both halves be genuinely supervised rather than merely present. Without building the same code twice, and without the container reporting itself healthy while half of what it contains is dead.

Easy to state, awkward to build. Five obstacles stood in the way, and they are not all failures — the first two are constraints that had to be designed around, and the last three are decisions that look arbitrary until the reasoning is written down.

The first is that PHP's traditional execution model rebuilds the world on every request. The framework boots, service providers register, configuration and routes are parsed, dependency wiring is constructed — and then the response is sent and all of it is discarded so the next request can build it again. For an application of any size that is most of the time spent on a request that does nothing but read a row and return JSON. The rejected alternative is the ordinary PHP runtime everyone reaches for first; what replaces it is worker mode, where the application is booted once and kept in memory between requests. That choice constrains everything after it, which is why it comes first.

The second is that a container assumes one process, and this application is two kinds of long-lived process at once. A web server that outlives every request and a pool of workers that outlive every job are both processes that refuse to end, but they fail differently and they stop differently. The container runtime can only watch one of them. If the web server is healthy while every queue worker is dead, the runtime still calls the container healthy, because the thing it is watching is still running. The rejected alternative is two images — which means building identical code twice and accepting that the halves will drift apart. What replaces it is a process manager owning three programs, so that supervision happens inside the container rather than above it.

The third is that the number of workers cannot be derived from the number of cores. Seventeen processes on a sixteen-core host looks like a mistake, and it would be if the work were computation — more processes than cores normally buys nothing but context switching. This work is not computation. Almost all of it is sending mail, calling an external endpoint and waiting, or reading and writing files, and a process blocked on a socket holds a process slot without holding a core. Matching workers to cores is the rejected alternative, and it would leave the machine idle while the queue grew.

The fourth is that two timeouts living in two different files have an ordering that is a correctness requirement. One belongs to the worker and stops it after a job has run too long; the other belongs to the queue and decides when an unfinished job becomes available to somebody else. Set them independently — the obvious thing to do, since they are configured in different places — and the queue can hand the same job to a second worker while the first is still working on it. The relationship between the numbers is the point, not the numbers themselves.

The fifth is that build order decides whether the cache is worth anything at all. An image is a stack of layers, a layer is reused only when its instruction and inputs are unchanged, and a changed layer invalidates every layer after it. Ordering instructions by convenience is the alternative I rejected, and it throws the cache away on every build. Ordering them from least to most frequently changing — system packages, then dependencies, then application code — is what makes a rebuild after a one-line change cost seconds instead of minutes.

The order is one of dependency rather than severity. The first two are constraints: they decide what the container is allowed to look like, and nothing later makes sense without them. The third, fourth and fifth are tuning decisions made inside that shape — how many workers, how they give up, and how the thing gets built. Each of the last three would be a reasonable default in isolation; what makes them worth writing down is that in this arrangement the defaults are all wrong.

Method

A. Speed first: why this runtime and not the usual one

This is the answer to the first obstacle, and it is the one the other four are built on top of.

PHP's traditional execution model rebuilds the world on every request. The framework boots, service providers register, configuration and routes are parsed, dependency wiring is constructed — and then the response is sent and all of it is thrown away, so the next request can build it again. For a framework of any size this is most of the time spent on a request that does nothing but read a row and return JSON.

Worker mode removes that. The application is booted once and held in memory, and each request runs against the already-booted application. The runtime I chose is FrankenPHP, a PHP runtime built into the Caddy web server, which supports this directly. I considered the alternatives and rejected them on the same grounds: classic PHP-FPM behind a separate web server keeps the boot-per-request cost entirely, and while OpenLiteSpeed is faster than a naive setup, it still treats the application as something to be re-entered rather than something to be kept warm. Keeping the application warm is the entire gain, and it is worth an unfamiliar base image.

What makes this notable is how little configuration it takes. There is no Laravel Octane in this project's dependencies — no package, no service provider, nothing in the application at all. Worker mode is one environment variable.

FROM dunglas/frankenphp:php8.3

# Listen on a plain port; the value of FRANKENPHP_CONFIG is what turns on
# worker mode, holding the booted application between requests.
ENV SERVER_NAME=":8000"
ENV FRANKENPHP_CONFIG="worker ./public/index.php"

The web server's own configuration is correspondingly small, and most of it is switching things off.

{
    # HTTPS is terminated by the reverse proxy in front of this container,
    # so the server must not try to obtain certificates of its own.
    auto_https off
    admin off
    frankenphp
}

:8000 {
    root * /app/public
    encode gzip
    php_server
    file_server
}

Both switches are deliberate. This container is never reachable from the internet; a reverse proxy sits in front of it and holds the certificate. A web server that tried to obtain its own certificate would fail, repeatedly and noisily, for a hostname it does not own. The administration endpoint is disabled for the same reason a door is locked: it allows live reconfiguration, nothing inside the container needs it, and it is one fewer surface to reason about.

B. The entrypoint prepares, then gets out of the way

The container's entrypoint is a shell script, but it does not stay running as the container's main process. Its job is to get the application into a known state, and then hand the role of PID 1 to something better suited to holding it.

# A database that is briefly unreachable must not stop the container.
php artisan db:show --database=pgsql || echo "database not ready yet, continuing"

php artisan config:clear && php artisan route:clear && php artisan view:clear
php artisan config:cache && php artisan route:cache && php artisan view:cache

# Tell any worker still running from a previous container to finish and exit.
php artisan queue:restart

# Replace this shell with the process manager, which becomes PID 1.
exec /usr/bin/supervisord -n -c /etc/supervisor/supervisord.conf

Three decisions in those lines are worth drawing out.

The database check is deliberately non-fatal. It reports whether the database can be reached and then continues regardless. This looks careless and is the opposite: the container is configured to restart automatically, so a container that refused to start because the database was unreachable for four seconds would enter a restart loop and turn a condition that heals itself into an outage that does not. The application will fail its own health check until the database returns, which is the correct place for that failure to appear.

The caches are cleared before they are rebuilt. Configuration and route caches are compiled files; an image carrying a stale one from build time would serve stale configuration forever, and clearing first costs a second at startup.

The last line is the important one. exec replaces the shell with the process manager rather than starting it as a child, which means the manager becomes PID 1 itself. Without exec, the shell would remain PID 1 with the manager beneath it, and the shutdown signal sent to the container would arrive at a shell that does not know what to do with it — so the workers would never be told to stop, and would be killed outright when the grace period expired. One keyword is the difference between a clean shutdown and a violent one.

C. Three programs under one supervisor

This is the answer to the second obstacle: I moved supervision inside the container, where it can see both halves.

The process manager I chose is Supervisor, and its configuration file is the closest thing this setup has to a design document. It declares three programs and, for each, the policy for starting it, restarting it, and stopping it.

[program:frankenphp]
command=frankenphp run --config /etc/caddy/Caddyfile
user=www-data
autostart=true
autorestart=true
priority=10
stdout_logfile=/dev/stdout
stdout_logfile_maxbytes=0
stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0
stopwaitsecs=10

priority sets start order, lowest first, so the web server is running before the workers begin competing for the database. autorestart is the restart policy that the container runtime can no longer provide, now that it is watching a process manager instead of the application. If the web server dies, Supervisor starts it again and the container never notices.

The logging lines are less obvious and matter more than they look. Both streams are pointed at the container's own standard output and error, and the rotation size is set to zero, which disables Supervisor's log rotation entirely. This is intentional. Anything written to those two streams is collected by the container runtime, which is where every other container on the host already sends its logs. Letting Supervisor rotate logs into files inside the container would put them somewhere nothing collects and nothing rotates, and they would quietly fill the disk.

stopwaitsecs is the grace period between asking a program to stop and killing it. Ten seconds is generous for a web server, which need only finish the requests in flight. It is nowhere near enough for the queue workers, which is why they are configured separately.

[program:laravel-queue]
command=php /app/artisan queue:work --sleep=3 --tries=3 --timeout=90 --max-jobs=100 --max-time=3600 --memory=512
user=www-data
autorestart=true
startretries=10
priority=20
numprocs=17
process_name=%(program_name)s_%(process_num)02d
stopwaitsecs=60
stopasgroup=true
killasgroup=true

numprocs=17 is the line that makes this work: one program declaration produces seventeen independent processes, distinguished by the numbered naming pattern below it. Supervisor watches all of them individually and restarts each on its own.

The stop behaviour is where the difference between the two kinds of process becomes concrete. Sixty seconds, rather than ten, because a worker asked to stop finishes its current job first — and a job that signs a document is not something to interrupt at an arbitrary instant. stopasgroup and killasgroup exist because a worker is not always a single process: running a job may fork a child, and signalling only the parent would leave orphans behind that keep running work nobody is waiting for. The signal goes to the whole process group instead.

startretries=10 is higher than the web server's three for the same reason the database check is non-fatal. A worker that cannot reach the database fails immediately, so a transient outage burns through retries much faster than it does for a server that fails only when a request arrives.

D. Why seventeen workers on a sixteen-core machine

This is the answer to the third obstacle, and it deserves an argument rather than an assertion.

That reasoning holds for work that uses a core. This work does not. Of the twenty-one kinds of deferred work in the application, the overwhelming majority are sending an email, calling an external HTTP endpoint and waiting for its answer, or reading and writing files. A process doing any of those is not computing — it is blocked, waiting on a socket or a disk, holding a process slot but no core. Sizing the pool to the core count would leave the machine idle while sixteen processes all wait on other people's servers.

So I sized the pool to expected concurrent waiting rather than to cores, with the memory ceiling as the real limit: each worker is capped at 512 MB and exits if it exceeds that, which bounds the worst case to something the host can survive. Seventeen is a judgement about the shape of this workload, not a measured optimum. It has been comfortable in practice, and it is the number most likely to change if the mix of work changes — document signing is genuinely CPU-bound, and if it ever became the common case rather than the rare one, this number would need to come down.

Two flags keep the pool honest over time. --max-jobs=100 and --max-time=3600 make each worker exit after a hundred jobs or an hour, whichever comes first, and Supervisor immediately starts a replacement. This is a deliberate use of the restart policy: a long-lived PHP process accumulates memory it never quite releases, and rather than hunting every leak, the workers are recycled often enough that leaks never matter.

# The pool is bounded by memory, not by cores, and recycled on purpose.
php artisan queue:work \
  --memory=512 \        # hard ceiling per worker; exits if exceeded
  --max-jobs=100 \      # recycle after 100 jobs...
  --max-time=3600 \     # ...or an hour, whichever comes first
  --timeout=90          # see the next section: this number is not free

E. Two timeouts that have to be ordered correctly

This is the answer to the fourth obstacle, and the numbers matter far less than the relationship between them.

A job that hangs must eventually be abandoned, or one worker sits on it forever and the pool shrinks by one every time it happens. Two separate mechanisms handle this, they live in different files, and the relationship between their numbers is a correctness requirement rather than a preference.

The first is the worker's own limit, --timeout=90 from the configuration above. A worker that has spent ninety seconds on a single job stops itself. This is enforced by an alarm signal from the pcntl extension, which is why that extension appears in the build at all.

# pcntl is what makes --timeout enforceable; without it a hung job is
# never interrupted and the worker holding it never returns to the pool.
RUN docker-php-ext-install zip pdo_pgsql pgsql pcntl bcmath intl gd

The second mechanism belongs to the queue itself, and it exists because the first one cannot be trusted alone. If a worker is killed outright — the container is stopped, the machine loses power — nothing gets the chance to put its job back. So the queue records when a job was handed out, and after retry_after seconds it considers that job abandoned and makes it available again.

'database' => [
    'driver' => 'database',
    'table' => env('DB_QUEUE_TABLE', 'jobs'),
    // Must stay GREATER than the worker's --timeout, so a job is never
    // handed to a second worker while the first is still stopping.
    'retry_after' => (int) env('DB_QUEUE_RETRY_AFTER', 120),
],

These two numbers must be ordered, and the order is not arbitrary. retry_after has to be larger than the worker timeout. Consider what happens if they are equal: at ninety seconds the worker begins stopping its job, and at exactly ninety seconds the queue decides the job was abandoned and offers it to another worker. The same job then runs twice, concurrently, while the first copy is still unwinding. For an email that means the user gets it twice. For a document signature it means something considerably worse.

Giving retry_after the larger value closes that window: the worker always gets to finish stopping, and release the job itself, before the queue is willing to hand it to anyone else. The gap between the two numbers is the safety margin.

One consequence is worth stating plainly, because it is easy to miss when reading the job classes. None of the twenty-one declare a timeout of their own, so this single pair of numbers governs every kind of deferred work in the application. Raising a timeout for one slow job means raising it for all of them, and raising the queue's value to match. That is a constraint worth knowing before adding a job that takes two minutes.

F. Ordering the build so the cache actually helps

This is the answer to the fifth obstacle, and the only one whose payoff is measured in minutes rather than correctness.

Ranked by how often they actually change, this application has three groups: the system packages and PHP extensions, which change perhaps twice a year; the PHP dependencies, which change when a package is added or upgraded; and the application code, which changes many times a day. So I put the extensions first, the dependencies second, and the code last.

The dependencies need one extra trick, because installing them normally requires the application code to be present — which would defeat the ordering entirely. A separate build stage solves it.

# Stage one: dependencies only. Nothing here depends on application code,
# so this layer is reused until the lock file itself changes.
FROM composer:2 AS vendor
WORKDIR /app
COPY composer.json composer.lock ./
RUN composer install --no-dev --no-scripts --no-autoloader --prefer-dist

# Stage two, after the heavy extension layer: take the finished dependencies
# from stage one, then copy the application code LAST.
COPY --from=vendor --chown=www-data:www-data /app/vendor ./vendor
COPY --chown=www-data:www-data . .
RUN composer dump-autoload --no-dev --optimize

The first stage copies only the two dependency manifests and installs from them, with the scripts and autoloader generation switched off because both need application code that is deliberately absent. The second stage copies the finished result across and only then brings in the code, generating the optimised autoloader once everything is present.

One line in the ignore file is load-bearing rather than tidy. The build excludes vendor from the copy of the working directory, which matters because COPY . . would otherwise overwrite the dependencies that came from the first stage with whatever happens to be on the machine running the build. That is how a local development dependency ends up in a production image, and it is a difficult thing to notice afterwards.

The ownership handling is a smaller optimisation with a large effect. Setting ownership during the copy is free, whereas changing ownership recursively across the whole application afterwards is a separate layer that rewrites every file — slow to build, and it doubles the space the files occupy in the image.

Results and Discussion

A. What the build ordering was worth

I adopted the ordering described above to replace an earlier build that copied everything in first and installed dependencies afterwards. The difference shows up on the only operation that happens constantly — changing a line of application code and rebuilding.

The second and third rows matter as much as the first. Nothing was made faster in general; the work was moved so that the case which happens fifty times a day is cheap and the cases which happen twice a year are not. A dependency change still pays the full cost, correctly, because the layer holding dependencies genuinely has to be rebuilt.

A build that relies on the dependency folder being present on the machine that runs it puts whatever happens to be installed on a developer's laptop into the image. The stage-based build derives it from the lock file instead, so the same commit produces the same image anywhere — which is what makes building it automatically on every push possible at all.

B. A ten-megabyte file that would vanish without an error

One consequence of this base image is worth writing down, because nothing about it is visible in any log.

The FrankenPHP image ships without a PHP configuration file at all. Not an empty one — none, so PHP's built-in compiled defaults apply: a two-megabyte limit on uploaded files and eight megabytes on a request body. The reverse proxy in front of the container allows twenty megabytes, and the application is written expecting that.

Left that way, files between two and twenty megabytes pass the proxy, reach PHP, and are discarded before the application ever sees them. The upload array carries an over-limit error code with an empty temporary filename, which the framework's validation reports as the field being invalid. The user is told their document is not acceptable. The logs record a validation failure. Nothing anywhere mentions a size limit, because as far as the application is concerned it has simply been handed a broken field.

A ten-megabyte PDF sent to a document signing endpoint would fail with nothing more informative than the name of the field. The guard is two lines.

; The base image ships no php.ini at all, so without this file PHP's
; defaults (2M upload, 8M body) silently discard anything larger.
upload_max_filesize = 20M
post_max_size = 20M

Both are set to twenty megabytes deliberately, matching the proxy exactly rather than exceeding it. The reasoning is about which component fails first: when the limits are equal, the proxy rejects an oversized request with a clear status code before PHP is involved. If PHP's limit were higher, the proxy would still reject it; if PHP's limit were lower, the failure would be silent again. Matching the numbers makes the behaviour predictable, and it is the sort of thing that should be written in the image rather than remembered.

C. What the container looks like from outside

From the host's perspective the container remains a single unit, and its declaration is unremarkable in a way that is rather the point.

    ports:
      - ${API_PORT}:8000          # bound to one gateway address, not 0.0.0.0
    restart: always
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8000/api/up"]
      interval: 30s
      start_period: 40s
    extra_hosts:
      # Resolve the identity provider to the host, so the container's
      # server-to-server calls stay inside the machine.
      - "<sso-host>:${GATEWAY_IP}"

Two details are worth drawing out. The published port is bound to a specific gateway address rather than every interface, so the container is reachable from the proxy and from nothing else. And the host mapping exists because the application makes server-to-server calls to an identity provider whose public name resolves, from outside, to a content delivery network that answers an automated client with a challenge page rather than a token. Mapping the name to the host's own address keeps those calls internal, where they are answered properly.

The start period is the part that acknowledges everything above: the container is given forty seconds before its health is judged at all, because the entrypoint clears caches, rebuilds them, and starts eighteen processes before anything can answer.

Conclusion

Two kinds of long-lived process now run in one container because I put a process manager where the application would normally be, and the container's health is defined by that manager rather than by any single program. The build that produces the image is ordered by how often each part changes, which is what makes the arrangement tolerable to work on day to day.

What this buys is coherence. One image means the web server and the workers are the same code, the same dependencies and the same configuration by construction, not by agreement. There is one version number to reason about, one thing to deploy, and one stream of logs in which a request and the job it queued appear in order.

What it costs is independent scaling. The workers cannot be scaled without also restarting the web server, because they are the same container. Changing seventeen to twenty-five means a restart that drops connections, which is a poor trade when only the queue is under pressure. If the two ever need to scale on different schedules, this arrangement is the wrong one and should be split — the image already supports it, since running the same image with a different command is all it would take.

Alongside the health check, a backlog monitor runs beside the workers and escalates when the queue grows beyond a threshold, so a pool that has shrunk shows up as a queue that keeps growing — the signal that matters when the question is whether the work is being done.

The part that generalises beyond PHP is the ordering principle. Processes were separated by how long they live and how they must be stopped, not by what language they are written in or which team owns them. A web server and a queue worker differ in ways that matter — one can be interrupted, one cannot — and once that difference is written down as configuration, the arrangement mostly explains itself.

Terms

Queue

A list of work waiting to be performed later. Here it is a database table, so one item of work is one row.

Job

A single item on the queue: a record naming the code to run and the data to run it with.

Queue worker

A long-running process that repeatedly takes the next job from the queue, performs it, and asks for another.

Worker mode

A way of running a PHP application in which the framework is booted once and reused across many requests, instead of being rebuilt for each one. Unrelated to queue workers despite the shared word.

PID 1

The first process started inside a container. When it exits the container stops, and it receives the signal asking the container to shut down.

Process manager

A program whose only purpose is to start other programs, watch them, and restart them when they fail. Supervisor is the one used here.

Reverse proxy

A server placed in front of an application that receives outside requests and forwards them inward, usually terminating HTTPS so the application behind it speaks only plain HTTP.

Layer

The result of one instruction in an image build. An image is a stack of them.

Build cache

The reuse of a previously built layer when its instruction and inputs are unchanged. A changed layer invalidates every layer after it.

Multi-stage build

One build file containing several independent builds, where a later stage copies selected files out of an earlier one and discards the rest.

retry_after

How long the queue waits before deciding a handed-out job was abandoned and offering it again. Must be larger than the worker's own timeout.