一、级联时间计算方式
比如A->B->C那么C的延迟是计算B还是A的。因为Event的中timestamp没变,但是C计算时间差值却是以B为准。如果时钟同步的情况下,其实基本可以考虑为A的。
逻辑如下:
级联中的Event依然是主库的时间,因此其延迟还是相对主库而言。
虽然apply_event_and_update_pos函数中由设置为当前时间thd->set_time()
但是最终设置还是在Query_log_event::do_apply_event和Query_log_event::do_apply_event中的
的
THD::set_time (this=0x7ffe74007da0, t=0x7ffe74007828)
因为只有做了数据库修改才会触发记录Event的工作
设个设置就是设置为event header的timestamp,因此还是级联中记录的Event的时间还是
主库的时间,计算延迟就是相对主库的时间。
栈帧
#0 THD::set_time (this=0x7ffe74007da0, t=0x7ffe7493c4d0) at /root/mysqlall/percona-server-locks-detail-5.7.22/sql/sql_class.h:3526
#1 0x00000000018459ab in Query_log_event::do_apply_event (this=0x7ffe7493c3b0, rli=0x676be60, query_arg=0x7ffe740061dc "BEGIN", q_len_arg=5)
at /root/mysqlall/percona-server-locks-detail-5.7.22/sql/log_event.cc:4714
#2 0x0000000001845287 in Query_log_event::do_apply_event (this=0x7ffe7493c3b0, rli=0x676be60)
at /root/mysqlall/percona-server-locks-detail-5.7.22/sql/log_event.cc:4567
#3 0x00000000018420d4 in Log_event::apply_event (this=0x7ffe7493c3b0, rli=0x676be60) at /root/mysqlall/percona-server-locks-detail-5.7.22/sql/log_event.cc:3570
#4 0x00000000018bc078 in apply_event_and_update_pos (ptr_ev=0x7fffec094830, thd=0x7ffe74007da0, rli=0x676be60)
at /root/mysqlall/percona-server-locks-detail-5.7.22/sql/rpl_slave.cc:4766
#5 0x00000000018bd773 in exec_relay_log_event (thd=0x7ffe74007da0, rli=0x676be60) at /root/mysqlall/percona-server-locks-detail-5.7.22/sql/rpl_slave.cc:5300
#6 0x00000000018c46e4 in handle_slave_sql (arg=0x6675d30) at /root/mysqlall/percona-server-locks-detail-5.7.22/sql/rpl_slave.cc:7543
#7 0x0000000001945620 in pfs_spawn_thread (arg=0x7ffe7c02c6c0) at /root/mysqlall/percona-server-locks-detail-5.7.22/storage/perfschema/pfs.cc:2190
#8 0x00007ffff7bc6aa1 in start_thread () from /lib64/libpthread.so.0
#9 0x00007ffff6719bcd in clone () from /lib64/libc.so.6
#0 THD::set_time (this=0x7ffe74007da0, t=0x7ffe7493c4d0) at /root/mysqlall/percona-server-locks-detail-5.7.22/sql/sql_class.h:3526
#1 0x00000000018459ab in Query_log_event::do_apply_event (this=0x7ffe7493c3b0, rli=0x676be60, query_arg=0x7ffe740061dc "BEGIN", q_len_arg=5)
at /root/mysqlall/percona-server-locks-detail-5.7.22/sql/log_event.cc:4714
#2 0x0000000001845287 in Query_log_event::do_apply_event (this=0x7ffe7493c3b0, rli=0x676be60)
at /root/mysqlall/percona-server-locks-detail-5.7.22/sql/log_event.cc:4567
#3 0x00000000018420d4 in Log_event::apply_event (this=0x7ffe7493c3b0, rli=0x676be60) at /root/mysqlall/percona-server-locks-detail-5.7.22/sql/log_event.cc:3570
#4 0x00000000018bc078 in apply_event_and_update_pos (ptr_ev=0x7fffec094830, thd=0x7ffe74007da0, rli=0x676be60)
at /root/mysqlall/percona-server-locks-detail-5.7.22/sql/rpl_slave.cc:4766
#5 0x00000000018bd773 in exec_relay_log_event (thd=0x7ffe74007da0, rli=0x676be60) at /root/mysqlall/percona-server-locks-detail-5.7.22/sql/rpl_slave.cc:5300
#6 0x00000000018c46e4 in handle_slave_sql (arg=0x6675d30) at /root/mysqlall/percona-server-locks-detail-5.7.22/sql/rpl_slave.cc:7543
#7 0x0000000001945620 in pfs_spawn_thread (arg=0x7ffe7c02c6c0) at /root/mysqlall/percona-server-locks-detail-5.7.22/storage/perfschema/pfs.cc:2190
#8 0x00007ffff7bc6aa1 in start_thread () from /lib64/libpthread.so.0
#9 0x00007ffff6719bcd in clone () from /lib64/libc.so.6
query map event在修改的第一条数据 记录时间 这个时间肯定是Log_event::apply_event的时间,
GTID自然是在最后commit的时候 ,XID EVENT也是正确的主库时间。
实际上第一个THD::set_time(header->timestamp)后user_time 就正确,下面的逻辑不会设置为当前时间。
inline void set_time()
{
start_utime= utime_after_lock= my_micro_time();
if (user_time.tv_sec || user_time.tv_usec)
{
start_time= user_time;
}
else
my_micro_time_to_timeval(start_utime, &start_time);
#ifdef HAVE_PSI_THREAD_INTERFACE
PSI_THREAD_CALL(set_thread_start_time)(start_time.tv_sec);
#endif
}
但是主库dispatch_command的时候不会设置user_time user_time为0,因此设置为当前时间。
#0 THD::set_time (this=0x7ffedc0009c0) at /mysqldata/percona-server-locks-detail-5.7.22/sql/sql_class.h:3514
#1 0x00000000015c5fe8 in dispatch_command (thd=0x7ffedc0009c0, com_data=0x7fffec5bdd70, command=COM_QUERY)
at /mysqldata/percona-server-locks-detail-5.7.22/sql/sql_parse.cc:1247
#2 0x00000000015c58ff in do_command (thd=0x7ffedc0009c0) at /mysqldata/percona-server-locks-detail-5.7.22/sql/sql_parse.cc:1021
#3 0x000000000170e578 in handle_connection (arg=0x67d01a0) at /mysqldata/percona-server-locks-detail-5.7.22/sql/conn_handler/connection_handler_per_thread.cc:312
#4 0x0000000001945538 in pfs_spawn_thread (arg=0x67c9dc0) at /mysqldata/percona-server-locks-detail-5.7.22/storage/perfschema/pfs.cc:2190
#5 0x00007ffff7bcfaa1 in start_thread () from /lib64/libpthread.so.0
#6 0x00007ffff6b37c4d in clone () from /lib64/libc.so.6
二、header中的timestamp和query_event的exe time计算方式
common_header:中的时间来自 命令发起的时间。下面是其初始化
Log_event::Log_event(THD* thd_arg, uint16 flags_arg,
enum_event_cache_type cache_type_arg,
enum_event_logging_type logging_type_arg,
Log_event_header *header, Log_event_footer *footer)
: is_valid_param(false), temp_buf(0), exec_time(0),
event_cache_type(cache_type_arg), event_logging_type(logging_type_arg),
crc(0), common_header(header), common_footer(footer), thd(thd_arg)
{
server_id= thd->server_id;
common_header->unmasked_server_id= server_id;
common_header->when= thd->start_time;
common_header->log_pos= 0;
common_header->flags= flags_arg;
}
dispatch_command调用时间
Query_log_event::Query_log_event(THD* thd_arg, const char* query_arg,
size_t query_length, bool using_trans,
bool immediate, bool suppress_use,
int errcode, bool ignore_cmd_internals)
: binary_log::Query_event(query_arg,
thd_arg->catalog().str,
thd_arg->db().str,
query_length,
thd_arg->thread_id(),
thd_arg->variables.sql_mode,
thd_arg->variables.auto_increment_increment,
thd_arg->variables.auto_increment_offset,
thd_arg->variables.lc_time_names->number,
(ulonglong)thd_arg->table_map_for_update,
errcode,
thd_arg->db().str ? strlen(thd_arg->db().str) : 0,
thd_arg->catalog().str ? strlen(thd_arg->catalog().str) : 0),
Log_event(thd_arg,
(thd_arg->thread_specific_used ? LOG_EVENT_THREAD_SPECIFIC_F :
0) |
(suppress_use ? LOG_EVENT_SUPPRESS_USE_F : 0),
using_trans ? Log_event::EVENT_TRANSACTIONAL_CACHE :
Log_event::EVENT_STMT_CACHE,
Log_event::EVENT_NORMAL_LOGGING,
header(), footer()),
data_buf(0)
{
DBUG_EXECUTE_IF("debug_lock_before_query_log_event",
DBUG_SYNC_POINT("debug_lock.before_query_log_event", 10););
/* save the original thread id; we already know the server id */
slave_proxy_id= thd_arg->variables.pseudo_thread_id;
if (query != 0)
is_valid_param= true;
/*
exec_time calculation has changed to use the same method that is used
to fill out "thd_arg->start_time"
*/
struct timeval end_time;
ulonglong micro_end_time= my_micro_time();//这里获取时间 query event
my_micro_time_to_timeval(micro_end_time, &end_time);
exec_time= end_time.tv_sec - thd_arg->start_time.tv_sec;//这里计算时间
三、级联中的query event exe time和延迟从库的延迟区别
这个问题主要是出现如下,
A<-B<-C
B库中出现如下query event time
# at 997
#230517 14:50:19 server id 1623306 end_log_pos 1060 CRC32 0x68c7ee95 Query thread_id=16 exec_time=4283165325 error_code=0
SET TIMESTAMP=1684306219/*!*/;
BEGIN
/*!*/;
# at 1060
#230517 14:50:19 server id 1623306 end_log_pos 1110 CRC32 0x5d25f8d5 Table_map: `myppt`.`mytest` mapped to number 120
# at 1110
#230517 14:50:19 server id 1623306 end_log_pos 1150 CRC32 0x146308df Write_rows: table id 120 flags: STMT_END_F
注意这里 exec_time=4283165325,为什么会这么大呢,主要还是回到计算方式,
exec_time= end_time.tv_sec - thd_arg->start_time.tv_sec;//这里计算时间
其中end_time.tv_sec为从库执行的本地服务器时间,而thd_arg->start_time.tv_sec为主库执行第一个语句第一条数据返回的时间,因此正常情况下end_time.tv_sec - thd_arg->start_time.tv_sec,可以反应语句在主库执行了多久,C库计算延迟的时候就会去掉这部分。
但是当B库时间设置不对的时候end_time.tv_sec取值就可能出现问题,如果B库所在服务器的时间比A的服务器的时间要小一些,那么,
end_time.tv_sec - thd_arg->start_time.tv_sec
会出现负值,而 exec_time是一个非负整数,也就把负值当做非负整数输出了所以特别大。
那天我们看到 A B C 库C 库的级联延迟比较小,这个是因为B库中的exec_time这个值比较大,因为B库修复后才继续跑的,本身延迟就比较大。那么B库和A库的延迟较大在计算这个值的时候用的算法是。
exec_time= B库当前时间 - A库执行命令的时间 = 比较大的一值
而B库当前时间明显落后了很多,因此计算出来的值比较大,当C库计算延迟的时候,
C库当前时间 - A库执行命令时间 - (B库当前时间 - A库执行命令的时间)
因此C库延迟看到的是900多秒吧我记得,而B库延迟比较大。这个时间其实大概就是减去了B库延迟的时间,这个是正常的,当主从没什么延迟的时候C库的延迟还是能大概反应其和A库的延迟的,因为这个时候exec_time在B库生成是比较正常的。
四、从库中xid event和gtid event时间和主库的区别
从库中看起来没有显示提交一说,且
- xid event 为最后一个语句执行时间
- gtid event为第一个语句执行时间
这个应该和从库生成xid event和gtid event的时机不一样有关系,不做代码分析,如下:
主库binlog:
# at 4556
#230517 14:50:19 server id 1623306 end_log_pos 4621 CRC32 0xe1af1c46 GTID last_committed=19 sequence_number=20 rbr_only=yes
/*!50718 SET TRANSACTION ISOLATION LEVEL READ COMMITTED*//*!*/;
SET @@SESSION.GTID_NEXT= '952a6f72-f58e-11eb-9722-000c29b1d522:323'/*!*/;
# at 4621
#230517 14:50:19 server id 1623306 end_log_pos 4694 CRC32 0x87472ea7 Query thread_id=16 exec_time=0 error_code=0
SET TIMESTAMP=1684306219/*!*/;
BEGIN
/*!*/;
# at 4694
#230517 14:50:19 server id 1623306 end_log_pos 4747 CRC32 0xca697216 Rows_query
# insert into mytest values(10)
# at 4747
#230517 14:50:19 server id 1623306 end_log_pos 4797 CRC32 0x9189ab34 Table_map: `myppt`.`mytest` mapped to number 121
# at 4797
#230517 14:50:19 server id 1623306 end_log_pos 4837 CRC32 0x34791d6a Write_rows: table id 121 flags: STMT_END_F
BINLOG '
K3lkZB0KxRgANQAAAIsSAACAAB1pbnNlcnQgaW50byBteXRlc3QgdmFsdWVzKDEwKRZyaco=
K3lkZBMKxRgAMgAAAL0SAAAAAHkAAAAAAAEABW15cHB0AAZteXRlc3QAAQMAATSriZE=
K3lkZB4KxRgAKAAAAOUSAAAAAHkAAAAAAAEAAgAB//4KAAAAah15NA==
'/*!*/;
### INSERT INTO `myppt`.`mytest`
### SET
### @1=10 /* INT meta=0 nullable=1 is_null=0 */
# at 4837
#230517 14:50:19 server id 1623306 end_log_pos 4868 CRC32 0x2b2debff Xid = 88
COMMIT/*!*/;
# at 4868
#230517 14:55:49 server id 1623306 end_log_pos 4933 CRC32 0x3e3ac306 GTID last_committed=20 sequence_number=21 rbr_only=yes
/*!50718 SET TRANSACTION ISOLATION LEVEL READ COMMITTED*//*!*/;
SET @@SESSION.GTID_NEXT= '952a6f72-f58e-11eb-9722-000c29b1d522:324'/*!*/;
# at 4933
#230517 14:55:49 server id 1623306 end_log_pos 5006 CRC32 0x3de25fe9 Query thread_id=17 exec_time=0 error_code=0
SET TIMESTAMP=1684306549/*!*/;
BEGIN
/*!*/;
# at 5006
#230517 14:55:49 server id 1623306 end_log_pos 5059 CRC32 0xa36440da Rows_query
# insert into mytest values(10)
# at 5059
#230517 14:55:49 server id 1623306 end_log_pos 5109 CRC32 0xc861f68e Table_map: `myppt`.`mytest` mapped to number 121
# at 5109
#230517 14:55:49 server id 1623306 end_log_pos 5149 CRC32 0xabed3a2d Write_rows: table id 121 flags: STMT_END_F
BINLOG '
dXpkZB0KxRgANQAAAMMTAACAAB1pbnNlcnQgaW50byBteXRlc3QgdmFsdWVzKDEwKdpAZKM=
dXpkZBMKxRgAMgAAAPUTAAAAAHkAAAAAAAEABW15cHB0AAZteXRlc3QAAQMAAY72Ycg=
dXpkZB4KxRgAKAAAAB0UAAAAAHkAAAAAAAEAAgAB//4KAAAALTrtqw==
'/*!*/;
### INSERT INTO `myppt`.`mytest`
### SET
### @1=10 /* INT meta=0 nullable=1 is_null=0 */
# at 5149
#230517 14:55:49 server id 1623306 end_log_pos 5180 CRC32 0xaa879e65 Xid = 98
COMMIT/*!*/;
# at 5180
#230517 15:33:18 server id 1623306 end_log_pos 5245 CRC32 0x3bb31784 GTID last_committed=21 sequence_number=22 rbr_only=yes
/*!50718 SET TRANSACTION ISOLATION LEVEL READ COMMITTED*//*!*/;
SET @@SESSION.GTID_NEXT= '952a6f72-f58e-11eb-9722-000c29b1d522:325'/*!*/;
# at 5245
#230517 15:33:05 server id 1623306 end_log_pos 5318 CRC32 0x4ac753db Query thread_id=17 exec_time=0 error_code=0
SET TIMESTAMP=1684308785/*!*/;
BEGIN
/*!*/;
# at 5318
#230517 15:33:05 server id 1623306 end_log_pos 5371 CRC32 0x7f6c3b32 Rows_query
# insert into mytest values(10)
# at 5371
#230517 15:33:05 server id 1623306 end_log_pos 5421 CRC32 0x96928c6b Table_map: `myppt`.`mytest` mapped to number 121
# at 5421
#230517 15:33:05 server id 1623306 end_log_pos 5461 CRC32 0x910d9c78 Write_rows: table id 121 flags: STMT_END_F
BINLOG '
MYNkZB0KxRgANQAAAPsUAACAAB1pbnNlcnQgaW50byBteXRlc3QgdmFsdWVzKDEwKTI7bH8=
MYNkZBMKxRgAMgAAAC0VAAAAAHkAAAAAAAEABW15cHB0AAZteXRlc3QAAQMAAWuMkpY=
MYNkZB4KxRgAKAAAAFUVAAAAAHkAAAAAAAEAAgAB//4KAAAAeJwNkQ==
'/*!*/;
### INSERT INTO `myppt`.`mytest`
### SET
### @1=10 /* INT meta=0 nullable=1 is_null=0 */
# at 5461
#230517 15:33:08 server id 1623306 end_log_pos 5514 CRC32 0x317c44a4 Rows_query
# insert into mytest values(10)
# at 5514
#230517 15:33:08 server id 1623306 end_log_pos 5564 CRC32 0x5ad8ee61 Table_map: `myppt`.`mytest` mapped to number 121
# at 5564
#230517 15:33:08 server id 1623306 end_log_pos 5604 CRC32 0xaf5d421a Write_rows: table id 121 flags: STMT_END_F
BINLOG '
NINkZB0KxRgANQAAAIoVAACAAB1pbnNlcnQgaW50byBteXRlc3QgdmFsdWVzKDEwKaREfDE=
NINkZBMKxRgAMgAAALwVAAAAAHkAAAAAAAEABW15cHB0AAZteXRlc3QAAQMAAWHu2Fo=
NINkZB4KxRgAKAAAAOQVAAAAAHkAAAAAAAEAAgAB//4KAAAAGkJdrw==
'/*!*/;
### INSERT INTO `myppt`.`mytest`
### SET
### @1=10 /* INT meta=0 nullable=1 is_null=0 */
# at 5604
#230517 15:33:11 server id 1623306 end_log_pos 5657 CRC32 0x12581f8f Rows_query
# insert into mytest values(10)
# at 5657
#230517 15:33:11 server id 1623306 end_log_pos 5707 CRC32 0x8b4fbaba Table_map: `myppt`.`mytest` mapped to number 121
# at 5707
#230517 15:33:11 server id 1623306 end_log_pos 5747 CRC32 0x8e9ab826 Write_rows: table id 121 flags: STMT_END_F
BINLOG '
N4NkZB0KxRgANQAAABkWAACAAB1pbnNlcnQgaW50byBteXRlc3QgdmFsdWVzKDEwKY8fWBI=
N4NkZBMKxRgAMgAAAEsWAAAAAHkAAAAAAAEABW15cHB0AAZteXRlc3QAAQMAAbq6T4s=
N4NkZB4KxRgAKAAAAHMWAAAAAHkAAAAAAAEAAgAB//4KAAAAJriajg==
'/*!*/;
### INSERT INTO `myppt`.`mytest`
### SET
### @1=10 /* INT meta=0 nullable=1 is_null=0 */
# at 5747
#230517 15:33:18 server id 1623306 end_log_pos 5778 CRC32 0x5b217f89 Xid = 100
COMMIT/*!*/;
SET @@SESSION.GTID_NEXT= 'AUTOMATIC' /* added by mysqlbinlog */ /*!*/;
从库binlog:
# at 1181
#230517 14:55:49 server id 1623306 end_log_pos 1246 CRC32 0xb51a3922 GTID last_committed=3 sequence_number=4 rbr_only=yes
/*!50718 SET TRANSACTION ISOLATION LEVEL READ COMMITTED*//*!*/;
SET @@SESSION.GTID_NEXT= '952a6f72-f58e-11eb-9722-000c29b1d522:324'/*!*/;
# at 1246
#230517 14:55:49 server id 1623306 end_log_pos 1309 CRC32 0x0d138e4f Query thread_id=17 exec_time=4283165417 error_code=0
SET TIMESTAMP=1684306549/*!*/;
BEGIN
/*!*/;
# at 1309
#230517 14:55:49 server id 1623306 end_log_pos 1359 CRC32 0x3a3a3464 Table_map: `myppt`.`mytest` mapped to number 120
# at 1359
#230517 14:55:49 server id 1623306 end_log_pos 1399 CRC32 0x78945a88 Write_rows: table id 120 flags: STMT_END_F
BINLOG '
dXpkZBMKxRgAMgAAAE8FAAAAAHgAAAAAAAEABW15cHB0AAZteXRlc3QAAQMAAWQ0Ojo=
dXpkZB4KxRgAKAAAAHcFAAAAAHgAAAAAAAEAAgAB//4KAAAAiFqUeA==
'/*!*/;
### INSERT INTO `myppt`.`mytest`
### SET
### @1=10 /* INT meta=0 nullable=1 is_null=0 */
# at 1399
#230517 14:55:49 server id 1623306 end_log_pos 1430 CRC32 0x9406879e Xid = 135
COMMIT/*!*/;
# at 1430
#230517 15:33:11 server id 1623306 end_log_pos 1495 CRC32 0xa3b13979 GTID last_committed=4 sequence_number=5 rbr_only=yes
/*!50718 SET TRANSACTION ISOLATION LEVEL READ COMMITTED*//*!*/;
SET @@SESSION.GTID_NEXT= '952a6f72-f58e-11eb-9722-000c29b1d522:325'/*!*/;
# at 1495
#230517 15:33:05 server id 1623306 end_log_pos 1558 CRC32 0x2d7d12fc Query thread_id=17 exec_time=4283165634 error_code=0
SET TIMESTAMP=1684308785/*!*/;
BEGIN
/*!*/;
# at 1558
#230517 15:33:05 server id 1623306 end_log_pos 1608 CRC32 0x7849212c Table_map: `myppt`.`mytest` mapped to number 120
# at 1608
#230517 15:33:05 server id 1623306 end_log_pos 1648 CRC32 0x1feef7e5 Write_rows: table id 120 flags: STMT_END_F
BINLOG '
MYNkZBMKxRgAMgAAAEgGAAAAAHgAAAAAAAEABW15cHB0AAZteXRlc3QAAQMAASwhSXg=
MYNkZB4KxRgAKAAAAHAGAAAAAHgAAAAAAAEAAgAB//4KAAAA5ffuHw==
'/*!*/;
### INSERT INTO `myppt`.`mytest`
### SET
### @1=10 /* INT meta=0 nullable=1 is_null=0 */
# at 1648
#230517 15:33:08 server id 1623306 end_log_pos 1698 CRC32 0xe3137337 Table_map: `myppt`.`mytest` mapped to number 120
# at 1698
#230517 15:33:08 server id 1623306 end_log_pos 1738 CRC32 0x57e2539c Write_rows: table id 120 flags: STMT_END_F
BINLOG '
NINkZBMKxRgAMgAAAKIGAAAAAHgAAAAAAAEABW15cHB0AAZteXRlc3QAAQMAATdzE+M=
NINkZB4KxRgAKAAAAMoGAAAAAHgAAAAAAAEAAgAB//4KAAAAnFPiVw==
'/*!*/;
### INSERT INTO `myppt`.`mytest`
### SET
### @1=10 /* INT meta=0 nullable=1 is_null=0 */
# at 1738
#230517 15:33:11 server id 1623306 end_log_pos 1788 CRC32 0xb1863d89 Table_map: `myppt`.`mytest` mapped to number 120
# at 1788
#230517 15:33:11 server id 1623306 end_log_pos 1828 CRC32 0xc5e25a36 Write_rows: table id 120 flags: STMT_END_F
BINLOG '
N4NkZBMKxRgAMgAAAPwGAAAAAHgAAAAAAAEABW15cHB0AAZteXRlc3QAAQMAAYk9hrE=
N4NkZB4KxRgAKAAAACQHAAAAAHgAAAAAAAEAAgAB//4KAAAANlrixQ==
'/*!*/;
### INSERT INTO `myppt`.`mytest`
### SET
### @1=10 /* INT meta=0 nullable=1 is_null=0 */
# at 1828
#230517 15:33:11 server id 1623306 end_log_pos 1859 CRC32 0xbf1c38a4 Xid = 137
COMMIT/*!*/;