Coordinator stopped because there were error(s) in the worker(s) mysql主从失败问题

1.主从不同步，查看主从状态显示

mysql> show slave status \G
*************************** 1. row ***************************
               Slave_IO_State: Waiting for master to send event
                  Master_Host: xx.xx.xx.xx
                  Master_User: slave
                  Master_Port: 3306
                Connect_Retry: 60
              Master_Log_File: mysql-bin.000023
          Read_Master_Log_Pos: 766919772
               Relay_Log_File: relay-bin.000002
                Relay_Log_Pos: 320
        Relay_Master_Log_File: mysql-bin.000023
             Slave_IO_Running: Yes
            Slave_SQL_Running: No
              Replicate_Do_DB: 
          Replicate_Ignore_DB: 
           Replicate_Do_Table: 
       Replicate_Ignore_Table: 
      Replicate_Wild_Do_Table: 
  Replicate_Wild_Ignore_Table: 
                   Last_Errno: 1032
                   Last_Error: Coordinator stopped because there were error(s) in the worker(s). The most recent failure being: Worker 1 failed executing transaction '0eb171ef-86ba-11ea-87b8-6c0b849b4343:2942675' at master log mysql-bin.000023, end_log_pos 766429911. See error log and/or performance_schema.replication_applier_status_by_worker table for more details about this failure or others, if any.
                 Skip_Counter: 0

2.根据上面报错中提示查看performance_schema.replication_applier_status_by_worker表中详细信息。

select * from performance_schema.replication_applier_status_by_worker\G
*************************** 1. row ***************************
         CHANNEL_NAME: 
            WORKER_ID: 1
            THREAD_ID: NULL
        SERVICE_STATE: OFF
LAST_SEEN_TRANSACTION: 0eb171ef-86ba-11ea-87b8-6c0b849b4343:2942675
    LAST_ERROR_NUMBER: 1032
   LAST_ERROR_MESSAGE: Worker 1 failed executing transaction '0eb171ef-86ba-11ea-87b8-6c0b849b4343:2942675' at master log mysql-bin.000023, end_log_pos 766429911; Could not execute Update_rows event on table aischedule.execution_flows; Can't find record in 'execution_flows', Error_code: 1032; handler error HA_ERR_KEY_NOT_FOUND; the event's master log mysql-bin.000023, end_log_pos 766429911
 LAST_ERROR_TIMESTAMP: 2020-06-02 12:42:01

3.检查发现是不能更新aischedule.execution_flows表中的数据。去主库根据binlog定位数据库操作记录。

mysqlbinlog --no-defaults -v -v --base64-output=decode-rows /data/logs/mysql-bin.000023 | grep -A 20 '766429911'
#200602 12:37:22 server id 65  end_log_pos 766429911 CRC32 0x95a2abf1   Update_rows: table id 111 flags: STMT_END_F
### UPDATE `aischedule`.`execution_flows`
### WHERE
###   @1=291046 /* INT meta=0 nullable=0 is_null=0 */
###   @2=16 /* INT meta=0 nullable=0 is_null=0 */
###   @3=1 /* INT meta=0 nullable=0 is_null=0 */
###   @4='basic' /* VARSTRING(512) meta=512 nullable=0 is_null=0 */
###   @5=20 /* TINYINT meta=0 nullable=1 is_null=0 */
###   @6='root' /* VARSTRING(256) meta=256 nullable=1 is_null=0 */
###   @7=1591072582036 /* LONGINT meta=0 nullable=1 is_null=0 */
###   @8=-1 (18446744073709551615) /* LONGINT meta=0 nullable=1 is_null=0 */
###   @9=-1 (18446744073709551615) /* LONGINT meta=0 nullable=1 is_null=0 */
###   @10=-1 (18446744073709551615) /* LONGINT meta=0 nullable=1 is_null=0 */
###   @11=2 /* TINYINT meta=0 nullable=1 is_null=0 */
###   @12='\x1f8\x00\x00\x00\x00\x00\x00\x0010D\x1c\x0dP*b9X\x10>($}b�=7H\x0a\x27\x19x180d\x0912O\x1612ߒb=A\x0e\x02)lA1517 089\x11i\x0dҖ,x19\x15\x06N\x05ccx0bZsv\x1a\x120c\x08ㆮ\x05x1305x5c@H\x03x11\x06\x19u�\x0bbvD1b$?\x108Ca302j\x07x0b{̒^\x5ce=ϱ1d\x100eliy%jxJ"06f\x15x097x11\x11*Īd\x09x1chS:;.8K9ڻ@nc>?\x0fˮxl8\x09rT2r$x039<<\x092�x19R\x1bS1fGU;>2x0ex1eI4\x1f&\x17v\x04\x00\x00' /* LONGBLOB/LONGTEXT meta=4 nullable=1 is_null=0 */
###   @13=2 /* INT meta=0 nullable=1 is_null=0 */
###   @14=NULL /* INT meta=0 nullable=1 is_null=1 */
###   @15=5 /* TINYINT meta=0 nullable=0 is_null=0 */
### SET
###   @1=291046 /* INT meta=0 nullable=0 is_null=0 */
###   @2=16 /* INT meta=0 nullable=0 is_null=0 */

4.根据@1=291046，在从库中发现没有这条记录。