CertGrid CertGrid
Troubleshooting·MySQL

MySQL Replication Failure and Failover

A local write on a replica collides with the source's next insert and replication stops with Last_SQL_Errno: 1062. The IO thread stays green throughout, which is why monitoring only one of the two threads misses it entirely.

Replication Guide 30 of 45 Advanced

Seconds_Behind_Source: NULL is the single most reliable signal that replication is not running. A lag alert that treats NULL as zero will never fire on this failure.

The failure is reproduced on db-c01. db-b01 is promoted to show the failover path.
Server NameIP AddressOSRolesCPURAMHDD
db-a01192.168.0.81Ubuntu 26.04 LTSPrimary / Source / Replica Set Member 12 Core4 GB50 GB
db-b01192.168.0.82Ubuntu 26.04 LTSStandby / Replica / Replica Set Member 22 Core4 GB50 GB
db-c01192.168.0.83Ubuntu 26.04 LTSReplica Set Member 3 / Cascading and Delayed Replica2 Core4 GB50 GB

This guide includes

Use this when replication has stopped and you need to know why. This matters because the usual cause is a local write on the replica colliding with the source - and proving where the offending transaction came from turns a mystery into a diagnosis.

Before you start

  1. Stop and start the replica deliberately

  2. Break it the way it really breaks

  3. Prove where the offending transaction came from

  4. Promote a replica

Official sources