How to Enable Asynchronous Indexing in Magento 2 on Ubuntu -- Setting Up Beanstalk

Why Asynchronous Indexing?

If you've ever had a store grind to a halt mid-day because someone triggered a full reindex from the admin panel, you already know why synchronous indexing is a problem. It blocks other operations, locks tables, and your customers feel it immediately.

Async indexing pushes those updates to a background queue. The storefront stays responsive while index jobs churn through data at their own pace.

Key Benefits

• No more storefront slowdowns -- index updates happen in the background, so peak-traffic hours aren't affected
• Database locks go away -- background processing means your store doesn't freeze during large catalog updates
• Resource usage stays even -- instead of a massive spike when reindexing kicks off, the work gets spread out over time
• Scales with your catalog -- we've run this on stores with 500k+ SKUs and frequent inventory changes without issues
• Happier customers -- they never notice the indexing happening at all

When You Need It

• You're adding or updating products frequently throughout the day
• Your catalog has thousands (or millions) of SKUs
• Orders, catalog edits, and customer traffic all need to coexist -- which is basically every production store

Step 1: Install and Configure Beanstalk

Beanstalkd is a lightweight work queue. It's dead simple compared to RabbitMQ, and for Magento's indexing needs, it gets the job done.

sudo apt update && sudo apt upgrade -y
sudo apt install beanstalkd -y

Enable and start it:

sudo systemctl enable beanstalkd
sudo systemctl start beanstalkd

Now configure it for persistent storage so jobs survive a restart:

sudo nano /etc/default/beanstalkd

Update these lines:

START=yes
OPTIONS="-l 127.0.0.1 -p 11300 -b /var/lib/beanstalkd -f 5 -z 1048576"

Save and restart:

sudo systemctl restart beanstalkd

Step 2: Install the Magento Asynchronous Operations Module

You'll need the magento/module-asynchronous-operations package:

composer require magento/module-asynchronous-operations
php bin/magento module:enable Magento_AsynchronousOperations
php bin/magento setup:upgrade
php bin/magento cache:clean

Step 3: Configure the Message Queue in Magento

Magento dispatches async tasks through its message queue system. You need to point it at Beanstalkd:

sudo nano app/etc/env.php

Add this under the queue key:

'queue' => [
  'amqp' => [
    'host' => '127.0.0.1',
    'port' => '11300',
    'user' => '',
    'password' => '',
    'virtualhost' => ''
  ]
],

Step 4: Enable Asynchronous Indexing

Flip all indexers to schedule mode:

php bin/magento indexer:set-mode schedule

Confirm it took effect:

php bin/magento indexer:status

You should see every indexer listed as "Update by Schedule" rather than "Update on Save."

Step 5: Test the Setup

Go ahead and add a product or edit a category to trigger an index update. To see what's happening in the queue, install the Beanstalk Console:

composer require ptrofimov/beanstalk_console

Fire it up:

php -S 127.0.0.1:8000 vendor/ptrofimov/beanstalk_console/public/index.php

Open http://127.0.0.1:8000 in your browser -- you'll see jobs flowing through tubes in real time. It's a nice way to verify everything's wired up correctly.

Summary

Once async indexing is in place, your Magento store handles catalog growth and traffic spikes much more gracefully. Beanstalkd is easy to set up and pairs well with Magento's queue framework. We've been running this setup on production stores for years without any drama.

For help with Magento performance tuning or server management, contact us at Private DevOps.