Mastering Laravel Workers: A Step-by-Step Guide to Setting Up a Laravel Worker Server on Ubuntu

Introduction

In modern web applications, handling time-consuming tasks synchronously degrades both performance and user experience. Laravel workers provide a powerful solution by offloading these tasks to background processes, keeping the application responsive and efficient.

This guide covers everything needed to set up a Laravel worker server on Ubuntu, from understanding how workers operate to configuring and supervising them in production.

1. Understanding Laravel Workers

What Are Laravel Workers?

Laravel workers are background processes that listen to a queue for new jobs and process them asynchronously. This is especially valuable for tasks that take significant time to execute, such as sending emails, processing images, generating reports, or interacting with external APIs.

Common Use Cases

• Sending transactional emails and notifications
• Processing uploaded files (images, videos, documents)
• Syncing data with third-party services
• Generating PDF reports or exports
• Running scheduled cleanup tasks

2. Prerequisites

System Requirements

• Ubuntu 20.04 or later
• PHP 8.0+ with required extensions
• Composer installed globally
• A Laravel application deployed on the server
• Redis (recommended) or another supported queue driver

Environment Setup

We start by installing the necessary PHP packages:

sudo apt update
sudo apt install php-cli php-mbstring php-xml php-zip php-mysql unzip

Then install Composer globally:

curl -sS https://getcomposer.org/installer | php
sudo mv composer.phar /usr/local/bin/composer

3. Setting Up Queue Configuration

Laravel supports several queue drivers. For this guide, we use Redis for its performance and reliability.

Installing and Configuring Redis

Install Redis on Ubuntu:

sudo apt install redis-server

Enable Redis to start on boot and start the service:

sudo systemctl enable redis-server.service
sudo systemctl start redis-server.service

Install the Redis PHP extension:

sudo apt install php-redis

Restart the PHP service:

sudo systemctl restart php7.4-fpm.service
# or for Apache
sudo systemctl restart apache2.service

Configuring the .env File

Update the Laravel application's .env file to use Redis as the queue connection:

QUEUE_CONNECTION=redis

We should also verify that the Redis settings in config/database.php and config/queue.php are correct.

4. Creating and Dispatching Jobs

Generating a Job Class

We use Artisan to create a new job:

php artisan make:job ProcessData

This creates a new job class at app/Jobs/ProcessData.php.

Implementing the Job Logic

Edit the handle method in the job class:

namespace App\Jobs;

use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;

class ProcessData implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable;

    public function handle()
    {
        // Your job logic here
    }
}

Dispatching Jobs to the Queue

In a controller or service class, we dispatch the job:

use App\Jobs\ProcessData;

class DataController extends Controller
{
    public function process()
    {
        // Dispatch the job
        ProcessData::dispatch();

        return response()->json(['status' => 'Job dispatched']);
    }
}

5. Running the Worker Server

Starting the Worker

We start processing jobs with the Artisan command:

php artisan queue:work

This starts a long-running worker process that listens to the queue and processes jobs as they arrive.

Worker Options

We can customize worker behavior with flags:

• --queue=default -- Specify which queue to listen to
• --sleep=3 -- Seconds to wait before polling for new jobs
• --tries=3 -- Maximum number of attempts before a job is marked as failed
• --timeout=60 -- Maximum seconds a job can run

Daemonizing the Worker

In production, workers should run as daemon processes. However, daemon mode requires careful memory management and regular restarts to prevent memory leaks.

6. Supervising the Worker Process

To ensure worker processes stay alive after server reboots or crashes, we use Supervisor.

Installing Supervisor

sudo apt install supervisor

Configuring Supervisor for Laravel

Create a Supervisor configuration file:

sudo nano /etc/supervisor/conf.d/laravel-worker.conf

Add the following configuration:

[program:laravel-worker]
process_name=%(program_name)s_%(process_num)02d
command=php /path/to/your/laravel/artisan queue:work redis --sleep=3 --tries=3
autostart=true
autorestart=true
user=www-data
numprocs=1
redirect_stderr=true
stdout_logfile=/path/to/your/laravel/storage/logs/worker.log

Replace /path/to/your/laravel with the actual path to the Laravel application.

Reloading Supervisor

Update Supervisor to recognize the new configuration and start the worker:

sudo supervisorctl reread
sudo supervisorctl update

sudo supervisorctl start laravel-worker:*

7. Managing and Monitoring Workers

Handling Failed Jobs

Jobs can fail due to exceptions. Laravel provides built-in mechanisms for handling failures.

Setting Up the Failed Job Table

Create a migration for the failed jobs table:

php artisan queue:failed-table
php artisan migrate

We can then view failed jobs with php artisan queue:failed and retry them with php artisan queue:retry.

Configuring Retry Mechanisms

We can configure the maximum number of retries in config/queue.php or by using the --tries option when starting the worker.

Monitoring Worker Performance

Using Laravel Horizon (Optional)

For advanced monitoring, Laravel Horizon provides a dashboard to monitor queues, job throughput, and failure rates. It requires Redis.

Install Horizon:

composer require laravel/horizon

Publish the configuration and assets:

php artisan horizon:install

Run migrations:

php artisan migrate

We can then update the Supervisor configuration to run Horizon instead of the default worker for more sophisticated queue management.

8. Scaling Workers

Running Multiple Workers

To handle a larger volume of jobs, we increase the numprocs value in the Supervisor configuration:

numprocs=5

This starts five worker processes, allowing parallel job processing.

Optimizing Worker Performance

• Use separate queues for different job types (e.g., emails, reports, notifications).
• Tune the --sleep and --timeout values based on workload characteristics.
• Monitor memory usage and configure --max-jobs or --max-time to restart workers periodically.
• Use Horizon's auto-scaling features to adjust worker count based on queue depth.

9. Troubleshooting Common Issues

Debugging Workers

• Check the worker log file configured in Supervisor.
• Use php artisan queue:failed to inspect failed jobs.
• Run php artisan queue:work --verbose for detailed output during development.

Common Errors

• "No handler registered for command" -- Ensure the job class implements ShouldQueue and is properly namespaced.
• Memory exhaustion -- Add --max-jobs=1000 or --max-time=3600 to restart workers before memory builds up.
• Redis connection refused -- Verify Redis is running with sudo systemctl status redis-server and that the PHP Redis extension is installed.

Best Practices

• Always use Supervisor or a similar process manager in production.
• Keep worker code stateless where possible.
• Log job progress for visibility into long-running tasks.
• Set reasonable retry limits to avoid infinite retry loops.
• Monitor queue depth and worker health as part of the overall application observability stack.

Conclusion

By setting up Laravel workers on Ubuntu, we take a significant step toward optimizing application performance and scalability. Offloading tasks to background processes improves user experience and allows the application to handle more concurrent requests. With Supervisor managing worker lifecycles and Horizon providing queue visibility, we have a robust foundation for production-grade asynchronous processing.