Skip to main content
Server & DevOpsMay 7, 20267 min read

Your MySQL Upgraded Itself Overnight. Here's What Happened

cPanel's nightly updates silently upgraded MySQL 8.4 to 9.7 on thousands of servers. Here is what broke, why rollback was nearly impossible, and how to prevent it.

The Morning Many Server Owners Dreaded

On the morning of April 22, 2026, thousands of system administrators opened their laptops to find their sites down. PHP apps could not connect to the database. WordPress installations returned blank pages. Custom applications threw socket errors they had never seen before. Nothing in the codebase had changed. Nobody had touched the server. And yet MySQL was on a completely different major version than it was the night before.

This was not a hack. It was not a misconfiguration on the part of the server owner. It was a silent, automatic major-version upgrade triggered by cPanel's nightly update routine — and the root cause was bad repository metadata from Oracle.

What Actually Happened

On April 21, 2026, Oracle released MySQL 9.7.0 and simultaneously marked MySQL 8.0 as End of Life. As part of that release, the official MySQL APT and YUM repositories were updated — and the metadata inside those repositories incorrectly classified MySQL 9.7 as the direct successor to MySQL 8.4 LTS.

From the package manager's perspective, this looked like a routine minor upgrade. When cPanel's nightly job ran yum update or apt dist-upgrade, it saw a newer MySQL package available, resolved no conflicts, and installed it. No warning. No confirmation prompt. No rollback checkpoint. By morning, the server was running MySQL 9.7 — a version with breaking changes that no production server was prepared for.

cPanel acknowledged the issue in case CPANEL-52811 and published recovery guidance shortly after reports flooded in.

Why MySQL 9.7 Is Not a Drop-In Replacement for 8.4

MySQL has two parallel release tracks. The LTS track (8.4, and eventually 9.4) is designed for long-running production deployments — conservative, stable, supported for years. The Innovation track (8.1, 8.2, 8.3, 9.0, 9.1… 9.7) ships new features every quarter and is intended for development experimentation, not production workloads.

MySQL 9.7 removed the mysql_native_password authentication plugin. This plugin is the default in PHP 7.4 and PHP 8.0, which are still widely deployed on cPanel shared and VPS environments. Losing it means:

  • PHP applications using older MySQL drivers lose their ability to authenticate
  • WordPress fails to connect unless wp-config.php is updated with a compatible auth method
  • Any application using a hard-coded connection string without explicit auth plugin handling breaks silently

Beyond authentication, the data directory format changed. Socket paths shifted from their expected locations. System variables that applications relied on were renamed or removed. The result was cascading failures across the entire application layer.

Why Downgrading Was Not an Option

The most frustrating part of this incident was the recovery story. Rolling back a MySQL major version upgrade is not supported. You cannot run yum downgrade and have your databases return to normal. The data directory written by MySQL 9.7 uses internal formats incompatible with MySQL 8.4. The only reliable recovery path was:

  1. Stop MySQL 9.7
  2. Reinstall MySQL 8.4
  3. Restore from a database dump taken before the upgrade occurred

Most affected servers did not have a fresh dump from the hours just before the nightly update ran. Some had daily backups from the previous day. Others had weekly snapshots. A few had nothing recent at all. The gap between "last backup" and "point of failure" determined how much data was at risk.

How to Check If Your Server Was Affected

Log into your server and run:

mysql --version

If the output shows 9.7 or any 9.x version and you intended to be on 8.4, your server was affected. To check whether automatic updates are still enabled on cPanel:

cat /etc/cpanel/autoupdate

If the value is 1, nightly updates are active and the same scenario can recur any time repository metadata is updated incorrectly.

How to Prevent This From Happening Again

Pin the MySQL version

On RPM-based systems (AlmaLinux, CloudLinux, CentOS):

yum versionlock add mysql-community-server

On Debian/Ubuntu-based systems:

apt-mark hold mysql-community-server

Version pinning tells the package manager to ignore newer versions of that specific package during automatic updates. Security patches within the pinned major version still apply normally.

Separate security updates from version upgrades

Configure your update policy so that only security patches apply automatically. Any package update that crosses a major version boundary should require explicit human approval. On cPanel servers, WHM > Update Preferences gives you control over which update categories run automatically.

Set up pre-update database snapshots

Before any nightly update job runs, trigger an automated mysqldump of all databases to a remote or off-server location. Even a 24-hour-old dump is infinitely more useful than no dump at all when you need to roll back.

# Example: nightly dump before cPanel updates run (typically 02:00)
0 1 * * * mysqldump --all-databases --single-transaction | gzip > /backup/mysql-pre-update-$(date +\%F).sql.gz

Monitor MySQL version as a health check

Add a simple check to your monitoring stack that alerts if the MySQL major version changes unexpectedly. A version bump that no administrator initiated is a signal worth waking up to — before your users do.

The Broader Lesson

This incident is not a cPanel-specific problem. It is a systems management problem. Automatic updates are convenient right up until they are catastrophic. Every server that allows unattended package upgrades is making an implicit bet that upstream repositories will always behave correctly. On April 21, 2026, that bet lost.

The fix is not to disable all updates — unpatched security vulnerabilities are a worse risk than a rogue package upgrade. The fix is to be precise about what you allow automatically and what you require a human to approve. Critical infrastructure like your database engine belongs firmly in the second category.

If you are also evaluating whether a managed database service would remove this category of risk entirely, see our breakdown of AWS RDS vs self-managed MySQL.

If you manage cPanel servers and want a second opinion on your update policies, backup procedures, or MySQL configuration, our cPanel server management service covers exactly this kind of preventive infrastructure work.

Need help with this?

Our team handles this kind of work daily. Let us take care of your infrastructure.