Why Magento 2 Indexers Get Stuck and How to Unstick Them

Stuck indexers show up in one of two ways. The Index Management grid says Reindex required and never clears, or an indexer sits at Processing for hours while prices, stock or search results lag behind what was saved in the admin. The store keeps running either way, just on data that is out of date.

There are only a handful of causes, and each has its own fix. This guide explains just enough of how indexing works in Magento 2.4.8 and 2.4.9 to read the status output, then goes through the causes in the order worth checking. Everything below comes from Magento's source code at the 2.4.9 tag and from Adobe's documentation.

Read the Status Before Changing Anything

bin/magento indexer:status

The output has six columns: ID, Title, Status, Update On, Schedule Status and Schedule Updated. Status is the value stored in the indexer_state table under a friendlier name:

Update On is Save or Schedule. For indexers on Schedule, Schedule Status shows the state of the indexer's changelog view, which is idle, working or suspended, followed by how many entities are waiting, as in idle (0 in backlog).

How Update by Schedule Works

Adobe recommends Update by Schedule for every indexer as a best practice, because in Update on Save mode, saving objects that depend on each other can cause deadlocks. Since 2.4.8, newly installed indexers start on schedule, while existing ones keep the mode they had.

In schedule mode, MySQL triggers on the source tables record the ID of every changed entity in a changelog table named after the view with a _cl suffix. The triggers are named trg_<table>_after_insert, trg_<table>_after_update and trg_<table>_after_delete. Each changelog row gets an auto-increment version_id, and the mview_state table remembers, for every view, the last version_id it has applied.

Cron does the rest in the index group, which runs in its own process and writes its output to var/log/magento.cron.index.log:

A view is updated only when its state is idle and its changelog holds rows newer than the stored version_id. While the update runs the state is working, and when it succeeds the stored version_id moves forward and the state returns to idle. A failed update restores the previous state and leaves the version_id where it was.

Cause 1: Cron Is Not Running the Index Group

This is the most common cause, and the one the admin warns about. The banner "One or more indexers are invalid. Make sure your Magento cron job is running." appears when any indexer is invalid. It never appears for an indexer stuck at Processing, because it only looks for invalid ones.

Check that the crontab exists for the user Magento runs as, and what the index group did last:

crontab -l | grep -i magento
tail -n 50 var/log/magento.cron.index.log

SELECT job_code, status, messages, scheduled_at, executed_at, finished_at
FROM cron_schedule
WHERE job_code LIKE 'indexer_%'
ORDER BY scheduled_at DESC
LIMIT 20;

A missing crontab comes back with bin/magento cron:install, and bin/magento cron:run --group=index runs the group once by hand, so any error lands in the terminal. In the cron log, Could not acquire lock for cron job: indexer_update_all_views means the previous run of that job still holds its lock. If that line repeats for hours, the job is not slow but stuck, and the next cause applies.

Cause 2: A Process Died Halfway Through

A reindex killed by the out-of-memory killer, a deploy that restarted PHP, a server reboot: each can stop an indexer in the middle. What happens next depends on a setting most stores never change.

By default the only guard against two reindexes of the same indexer is the status row itself. While the stored status is working, a new full reindex does nothing, and bin/magento indexer:reindex prints process error during indexation process: followed by <Indexer> index is locked by another reindex process. Skipping. Cron retries only indexers marked invalid, so an indexer whose process died stays at Processing indefinitely. A changelog view killed mid-update fares the same. Its state stays working, it is never updated again, and because changelog cleanup only deletes rows the view has already applied, its backlog keeps growing.

First make sure nothing is actually still running:

ps aux | grep -E 'indexer:reindex|cron:run --group=index' | grep -v grep

Then mark the indexer invalid, so cron rebuilds it on its next pass:

bin/magento indexer:reset catalogsearch_fulltext
# or, on 2.4.7 and later
bin/magento indexer:set-status invalid catalogsearch_fulltext

A full reindex also moves a stuck changelog view back to idle, so a view left at working clears once its indexer has been rebuilt.

To stop this from happening again, turn on the application lock that Magento has offered since 2.4.3:

// app/etc/env.php
'indexer' => [
    'use_application_lock' => true,
],

With it, a reindex holds a real lock, by default a MySQL named lock that is released when the process's database connection closes. An indexer or view whose status says working while nobody holds its lock is then read as invalid or idle, and cron picks it up again without anyone stepping in. One caution for 2.4.8: 2.4.9 fixed a bug where, with this lock on and indexers running in parallel threads, the lock was lost and indexers stayed at Processing.

Cause 3: One Failing View Holds Up the Rest

The cron job that updates changelog views walks through them one after another, and nothing in that loop catches an error. When one view throws an exception, every view after it waits for the next cron run, and if the failure repeats, they wait indefinitely.

A common culprit is the search index. If OpenSearch is unreachable, the catalog search indexer fails with Indexer handler is not available: followed by the engine name, and a full reindex of it ends as invalid again, to be retried a minute later. When catalogsearch_fulltext is among the stuck indexers, check the search engine before anything else:

bin/magento config:show catalog/search/engine
bin/magento config:show catalog/search/opensearch_server_hostname
bin/magento config:show catalog/search/opensearch_server_port
# then, with that host and port
curl -s http://<host>:<port>/_cluster/health

If the search engine itself needs work, the OpenSearch setup guide for Magento covers installing and connecting it.

Cause 4: The Triggers Are Gone

If you save a product and no new row appears in its changelog table, the triggers that feed the changelog are missing, which tends to follow a database copied or restored from a dump. List what exists:

SELECT TRIGGER_NAME, EVENT_OBJECT_TABLE
FROM information_schema.TRIGGERS
WHERE TRIGGER_SCHEMA = DATABASE() AND TRIGGER_NAME LIKE 'trg\_%';

Adobe's documented fix is to switch the affected indexers to realtime and back to schedule, which removes and recreates their triggers:

bin/magento indexer:set-mode realtime catalog_product_price
bin/magento indexer:set-mode schedule catalog_product_price

Setting schedule alone does nothing for a view that is already on schedule, so both steps are needed. Do it in a quiet window. Since 2.4.8 the switch to realtime also drops the changelog table and invalidates the indexer, so a full reindex follows. A gentler route exists too, since bin/magento setup:upgrade recreates the triggers of every indexer on schedule without dropping their changelogs.

Cause 5: The Changelog Counter Ran Out or Was Reset

On very busy stores the changelog's version_id can reach the maximum value of its column. Adobe's 2.4.9 release notes describe index updates stopping when that happens, and 2.4.9 widens the column to BIGINT. On 2.4.8, a backlog that never moves while cron runs cleanly is worth checking against that limit.

The same symptom can be caused by hand. Emptying a changelog table with TRUNCATE resets its auto-increment counter, so new rows start again from 1. A view whose stored version_id is higher than every row in the table skips each update until the counter catches up, which on a quiet table can take a long time. Leave changelog tables to the cleanup job, and change indexer_state and mview_state through the commands above rather than with SQL.

Cause 6: Batches Too Big for the Memory

Several indexers build temporary tables in batches, and the batch size for each is set in app/etc/env.php since 2.4.3. The 2.4.9 defaults are 5,000 rows for the price index, 200 for stock and 100,000 for category products. Magento logs Memory size allocated for the temporary table is more than 20% of innodb_buffer_pool_size when a batch outgrows the database, and a reindex that ends with the single word Killed was stopped by the operating system, usually for running out of memory. Smaller batches trade speed for memory. Adobe's documentation uses this shape:

// app/etc/env.php
'indexer' => [
    'batch_size' => [
        'cataloginventory_stock' => ['simple' => 200],
        'catalog_category_product' => 666,
        'catalogsearch_fulltext' => [
            'partial_reindex' => 100,
            'mysql_get' => 500,
            'elastic_save' => 500,
        ],
        'catalog_product_price' => [
            'simple' => 200,
            'default' => 500,
            'configurable' => 666,
        ],
    ],
],

A Routine That Is Safe to Follow

1. Confirm cron runs the index group, and read var/log/magento.cron.index.log.
2. Run bin/magento indexer:status, and note what is invalid, what is Processing and which views have a backlog.
3. If catalogsearch_fulltext is involved, check the search engine first.
4. Make sure no reindex is still running, reset what is stuck and let cron rebuild it, or run bin/magento indexer:reindex <indexer> in a quiet hour. Naming one indexer also rebuilds the indexers that depend on it, which is why a price reindex also runs the search index.
5. Save a test product and confirm a row lands in its changelog. If none does, recreate the triggers.
6. Watch the backlog fall in indexer:status.
7. Turn on use_application_lock, so the next killed process clears itself.

Indexing also changes between releases, so the 2.4.9 upgrade guide is worth a read before the next upgrade. On large catalogs the search and price indexes are also where most of the speed goes, and sizing them is part of Magento 2 Speed Optimization.