Laravel Queues Explained: Jobs, Workers and Horizon

A web request should finish in a fraction of a second, and some work cannot: sending mail through a slow provider, building an export, resizing uploads, calling a third-party API that answers when it feels like it. Laravel queues take that work out of the request. The controller records what has to be done and responds, and a separate process does the work a moment later.

The idea is simple. The trouble comes later, in production: jobs that run twice, workers still running last week's code, a failed-jobs table nobody reads. Below are the parts, the choices and the settings that prevent those problems.

Three parts: the job, the connection and the worker

A job is a class in app/Jobs that implements ShouldQueue and has a handle() method. You create one with php artisan make:job SendInvoice and send it to the queue with SendInvoice::dispatch($order).

A connection is where waiting jobs are stored. Connections are defined in config/queue.php, and QUEUE_CONNECTION in .env picks the default. One connection can hold several named queues, which is how you give password-reset emails priority over a nightly import.

A worker is a long-running PHP process, started with php artisan queue:work, that takes jobs off the queue and runs them. If no worker is running, jobs wait forever. That is the first thing to check when "the queue does not work".

Choosing a driver

Driver	Good for	What you have to run
database	Low to moderate volume. The usual starting point	Nothing new. Jobs sit in a jobs table in your own database
redis	Higher volume, many workers, and anything that will use Horizon	A Redis server and the phpredis extension or the predis package
sqs	Applications on AWS that want a managed queue	An AWS account and the AWS SDK package. Retry timing is set on the SQS side
sync	Local development and tests	Nothing. The job runs inside the request, so it removes the benefit in production

Start with database unless you already run Redis. Changing the driver later is a configuration change, and the job classes stay as they are. The Laravel queue documentation lists what each driver needs.

Writing a job that is safe to run twice

A queue promises that a job runs at least once, not exactly once. A worker can be killed after the email went out and before the job was marked as done, and the job then runs again. Every job needs to cope with that.

<?php

namespace App\Jobs;

use App\Models\Order;
use App\Services\InvoiceMailer;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Queue\Queueable;
use Throwable;

class SendInvoice implements ShouldQueue
{
    use Queueable;

    public function __construct(public Order $order) {}

    public function tries(): int
    {
        return 3;
    }

    public function backoff(): array
    {
        return [10, 60, 300];
    }

    public function handle(InvoiceMailer $mailer): void
    {
        if ($this->order->invoice_sent_at !== null) {
            return; // an earlier attempt already sent it
        }

        $mailer->send($this->order);

        $this->order->update(['invoice_sent_at' => now()]);
    }

    public function failed(?Throwable $exception): void
    {
        // Alert someone and flag the order for manual follow-up.
    }
}

Four habits matter here:

• Check before acting. The job looks at invoice_sent_at first. For payments, pass the provider an idempotency key so a repeated call cannot charge twice.
• Pass models, not copies of data. Laravel stores only the model's ID in the queue and reloads the record when the job runs, so the job sees current data and the payload stays small.
• Dispatch after the transaction commits. A job dispatched inside a database transaction can be picked up before the commit, and the worker will not find the new record. Call SendInvoice::dispatch($order)->afterCommit() or set after_commit to true on the connection.
• Prevent duplicates where they hurt. Implement ShouldBeUnique to keep a second copy of the same job off the queue, and use the WithoutOverlapping job middleware when two jobs for the same record must not run at the same time.

Jobs in applications created on older Laravel versions list several traits where this example has one. Both forms work.

Retries, backoff and timeouts

By default a job is attempted once. Allow more attempts for all jobs with queue:work --tries=3, or per job with a tries() method as above. Older code sets a public $tries property, and recent releases also accept PHP attributes on the class. backoff() sets the wait between attempts, and an array gives each retry a longer pause, which helps when the cause is a third-party outage.

Two time limits work together, and getting them wrong is the classic cause of jobs that run twice:

• The worker timeout (60 seconds by default, changed with --timeout or per job) stops a job that runs too long. It needs the PCNTL extension.
• retry_after in config/queue.php is how long the queue waits before it assumes a job was lost and hands it to another worker.

The timeout must be several seconds shorter than retry_after. If a job is allowed to run for 120 seconds while retry_after is 90, a second worker starts the same job while the first is still busy. For long jobs, raise both, or split the work into smaller jobs.

Some errors should not be retried at all. If the input is invalid, call $this->fail($exception) so the job fails at once and does not sit through three pointless attempts.

Failed jobs are a to-do list

When a job has used all its attempts, Laravel writes it to the failed_jobs table with the exception and calls the job's failed() method. Each row is a customer who did not get an invoice or an order that did not sync, so someone has to look.

php artisan queue:failed                  # list failed jobs
php artisan queue:retry <id>              # put one back on the queue
php artisan queue:retry all               # retry everything
php artisan queue:forget <id>             # delete one
php artisan queue:prune-failed --hours=168

Send an alert from failed(), or register a listener with Queue::failing() in a service provider, so failures reach a person and not only a table. Schedule queue:prune-failed to remove old rows, and queue:monitor to raise an event when a queue grows past a threshold.

Keeping workers running in production

A worker started in a terminal stops when the terminal closes. In production a process manager keeps workers alive. Supervisor is the common choice on Linux:

[program:app-worker]
process_name=%(program_name)s_%(process_num)02d
command=php /var/www/app/current/artisan queue:work --sleep=3 --tries=3 --max-time=3600
autostart=true
autorestart=true
stopasgroup=true
killasgroup=true
user=www-data
numprocs=4
redirect_stderr=true
stdout_logfile=/var/www/app/shared/logs/worker.log
stopwaitsecs=3600

What the important lines do:

• numprocs is the number of workers. Start low and raise it while watching database connections and memory.
• --max-time=3600 makes each worker exit after an hour so Supervisor starts a fresh one. Workers keep the application in memory, and this clears whatever has built up.
• stopwaitsecs must be longer than your longest job, or Supervisor will kill a job halfway through during a restart.

Because workers hold the code in memory, they keep running the old version after a deployment. Add php artisan queue:restart to the end of every deploy. It tells each worker to exit after its current job, and Supervisor starts replacements on the new code. The signal is stored in the cache, so on more than one server the cache store has to be a shared one.

When Horizon is worth adding

Laravel Horizon is a dashboard and a configuration layer for Redis queues. It only works with the redis connection, and its documentation notes that Redis Cluster is not supported. You install it with composer require laravel/horizon and php artisan horizon:install.

With Horizon, the worker setup moves into config/horizon.php: which queues exist, how many processes each environment may run, and a balancing strategy that shifts processes toward the busiest queue. Supervisor then runs one php artisan horizon process, and deployments call php artisan horizon:terminate in place of queue:restart.

The dashboard at /horizon shows throughput, wait time per queue, recent jobs and failed jobs with their exceptions and a retry button. Outside the local environment, access is decided by the viewHorizon gate in app/Providers/HorizonServiceProvider.php, so define who may see it before you deploy. The metrics graphs fill in only if horizon:snapshot runs on the scheduler every five minutes.

Horizon earns its place when you run several queues and people need to see what is happening without logging in to a server. A single worker on the database driver does not need it.

Do you need Laravel queues yet, and who should set them up

You need them when requests are slow because of work the user does not have to wait for, when exports time out, or when a webhook sender expects an answer faster than your processing takes. Writing jobs is ordinary work for any Laravel developer. The production side is where projects stumble: Supervisor, Redis, deploy hooks, failure alerts and timeouts that match. If outgoing API calls are your main reason for a queue, designing integrations that survive third-party failures covers the retry design.

We set this up as part of Laravel application development and look after running workers under Laravel maintenance and support. Multi-tenant products usually need queues from the first release, so we plan them in from the start of Laravel SaaS development.

Frequently asked questions

Do I need Redis to use Laravel queues?

No. The database driver stores jobs in a table in your existing database and is enough for many applications. Redis becomes worth running when job volume is high, when you want Horizon, or when you already use Redis for cache and sessions.

Why did my queued job run twice?

The usual cause is a job that runs longer than retry_after, so the queue hands it to a second worker. A worker killed during a deployment has the same effect. Fix the timing, and write the job so that a second run does no harm.

Can Laravel queues run on shared hosting?

Often, yes, if the host allows cron. A cron entry that runs php artisan queue:work --stop-when-empty every minute processes whatever is waiting and exits. Jobs may wait up to a minute and runs must not overlap, so it suits low volumes only.

What is the difference between the queue and the scheduler?

The scheduler starts work at set times, such as a report every night at two. The queue runs work as soon as a worker is free after something triggers it. They are often combined: a scheduled command dispatches many small jobs, and the workers process them in parallel.