背景
线上查询某张表当前在buffer_pool中占用多少个页面，执行时间将近1分钟，这段时间磁盘IO打满
SQL：
select * from sys.innodb_buffer_stats_by_table;
select * from information_schema.innodb_buffer_page;
疑问
1、为什么执行这条语句耗时将近1分钟？
2、为什么这段时间会将磁盘io打满？
3、会不会阻塞DML？
课前知识
1.sys.innodb_buffer_stats_by_table
该系统表是一个视图，汇总了information_schema.innodb_buffer_page表中的信息，所以查询该视图实际上是对information_schema.innodb_buffer_page的查询
2.mysql临时表
用户临时表
系统临时表
内存临时表(heap表)：数据存储在内存中
磁盘临时表(ondisk表)：数据存储在磁盘上
myisam
innodb
共享临时表空间
5.7之前如果开启innodb_file_per_table,则临时表都有各自的表空间，5.7之后临时表共用一个表空间ibtmp1,这个共享表空间mysqld启动时初始化，关闭时删除
3.innodb固有表(innodb磁盘临时表)，与常规的innob表不同，具有以下特征
An intrinsic table is a special kind of temporary table that
is invisible to the end user. It can be created internally by InnoDB, the MySQL
server layer or other modules connected to InnoDB in order to gather and use
data as part of a larger task. Since access to it must be as fast as possible,
it does not need UNDO semantics, system fields DB_TRX_ID & DB_ROLL_PTR,
doublewrite, checksum, insert buffer, use of the shared data dictionary,
locking, or even a transaction. In short, these are not ACID tables at all,
just temporary data stored and manipulated during a larger process.
环境
mysql: oracle mysql 5.7.26
buffer_pool: 512M
问题1分析 (为什么执行这条语句耗时将近1分钟？)
# 打印函数调用堆栈,跟踪函数执行过程，发现可能耗时的地方为i_s_innodb_fill_buffer_pool( )，
这个函数的功能是循环遍历innodb_buffer_pool中的每一个chunk(128M),然后循环遍历每个chunk中的block(16k)，
采集关于innodb_buffer_pool中数据页的信息，并记录在innodb_buffer_page表中。
代码段如下：
/* Go through each chunk of buffer pool. Currently ,each chunk is 128M*/
for (ulint n = 0;
n < ut_min(buf_pool->n_chunks, buf_pool->n_chunks_new); n++) {
const buf_block_t* block;
ulint n_blocks;
buf_page_info_t* info_buffer;
ulint num_page;
ulint mem_size;
ulint chunk_size;
ulint num_to_process = 0;
ulint block_id = 0;
/* Get buffer block of the nth chunk */
block = buf_get_nth_chunk_block(buf_pool, n, &chunk_size);
num_page = 0;
while (chunk_size > 0) {
/* we cache maximum MAX_BUF_INFO_CACHED number of
buffer page info */
num_to_process = ut_min(chunk_size,
MAX_BUF_INFO_CACHED);
mem_size = num_to_process * sizeof(buf_page_info_t);
/* For each chunk, we'll pre-allocate information
structures to cache the page information read from
the buffer pool. Doing so before obtain any mutex */
info_buffer = (buf_page_info_t*) mem_heap_zalloc(
heap, mem_size);
/* Obtain appropriate mutexes. Since this is diagnostic
buffer pool info printout, we are not required to
preserve the overall consistency, so we can
release mutex periodically */
buf_pool_mutex_enter(buf_pool);
/* GO through each block in the chunk ,each block is 16k*/
for (n_blocks = num_to_process; n_blocks--; block++) {
i_s_innodb_buffer_page_get_info(
&block->page, pool_id, block_id,
info_buffer + num_page);
block_id++;
num_page++;
/*添加打印信息 for block*/
}
# 在代码中增加打印信息"for block"
#执行SQL查询
mysql> select * from information_schema.innodb_buffer_page;
#获取日志结果输出信息
cat mysql-error.log|grep 'for block' |wc -l
32764
# 查询innndb_buffer_page信息
select count(*) from information_schema.innodb_buffer_page;
+----------+
| count(*) |
+----------+
| 32764 |
+----------+
# 分析输出结果
从日志输出信息与information_schema.innndb_buffer_page中的行数可知，每循环一次block，采集一个page的信息，在innodb_buffer_page中记录一行数据.
查询语句共循环32764，即扫描了32764个page，每个page的大小是16KB，共计32764*16/1024近似等于512MB，innodb_buffer_pool的大小恰好是512M。
结论
该查询语句会遍历整个innodb_buffer_pool页面，并将采集到的信息填充到information_schema.innndb_buffer_page表，innodb_buffer_pool越大耗时越长。
问题2 （为什么影响磁盘IO，以至于IO打满？）
采集buffpool信息相关代码如下：
# 不会在一开始就创建磁盘临时表，当超出内存临时表的限制大小时才会创建
bool schema_table_store_record(THD *thd, TABLE *table)
{
int error;
if ((error= table->file->ha_write_row(table->record[0])))
{
Temp_table_param *param= table->pos_in_table_list->schema_table_param;
if (create_ondisk_from_heap(thd, table, param->start_recinfo,¶m->recinfo, error, FALSE, NULL)){
return 1;
}
}
return 0;
}
# 转换为磁盘临时表时，会通过系统变量internal_tmp_disk_storage_engine来选择使用myisam引擎还是innodb引擎，默认是innodb
switch (internal_tmp_disk_storage_engine)
{
case TMP_TABLE_MYISAM:
new_table.s->db_plugin= ha_lock_engine(thd, myisam_hton);
break;
case TMP_TABLE_INNODB:
new_table.s->db_plugin= ha_lock_engine(thd, innodb_hton);
break;
default:
DBUG_ASSERT(0);
new_table.s->db_plugin= ha_lock_engine(thd, innodb_hton);
}
# 将heap临时表中已有数据拷贝到ondisk临时表中
/*
copy all old rows from heap table to on-disk table
This is the only code that uses record[1] to read/write but this
is safe as this is a temporary on-disk table without timestamp/
autoincrement or partitioning.
*/
while (!table->file->ha_rnd_next(new_table.record[1]))
{
write_err= new_table.file->ha_write_row(new_table.record[1]);
DBUG_EXECUTE_IF("raise_error", write_err= HA_ERR_FOUND_DUPP_KEY ;);
if (write_err)
goto err;
}
/* copy row that filled HEAP table */
if ((write_err=new_table.file->ha_write_row(table->record[0])))
{
if (!new_table.file->is_ignorable_error(write_err) ||
!ignore_last_dup)
goto err;
if (is_duplicate)
*is_duplicate= TRUE;
}
else
{
if (is_duplicate)
*is_duplicate= FALSE;
}
# 删除heap临时表
/* remove heap table and change to use on-disk table */
(void) table->file->ha_rnd_end();
(void) table->file->ha_close(); // This deletes the table !
delete table->file;
table->file=0;
plugin_unlock(0, table->s->db_plugin);
share.db_plugin= my_plugin_lock(0, &share.db_plugin);
new_table.s= table->s; // Keep old share
*table= new_table;
*table->s= share;
在代码中添加打印信息，并执行SQL
执行SQL期间观察innodb脏页面，发现有大量脏页产生
查看mysql-error.log中的打印输出结果,刚开始写内存表，后来转成innodb磁盘临时表，后需写入都是写入磁盘临时表，
================ha_heap::write_row============
================ha_heap::write_row============
================ha_heap::write_row============
================ha_heap::write_row============
================ha_heap::write_row============
================ha_heap::write_row============
================ha_heap::write_row============
================ha_heap::write_row============
================ha_heap::write_row============
================ha_heap::write_row============
================ha_heap::write_row============
==============create_ondisk_innodb_from_heap=============
==========================copy all old rows from heap table to on-disk table===========
===========ha_innobase::write_row=========
======================ha_innobase::intrinsic_table_write_row==================
======================row_insert_for_mysql_using_cursor==============
===========ha_innobase::write_row=========
======================ha_innobase::intrinsic_table_write_row==================
======================row_insert_for_mysql_using_cursor==============
..............................
..............................
===========ha_innobase::write_row=========
======================ha_innobase::intrinsic_table_write_row==================
======================row_insert_for_mysql_using_cursor==============
==========================remove heap table and change to use on-disk table===========
===========ha_innobase::write_row=========
======================ha_innobase::intrinsic_table_write_row==================
======================row_insert_for_mysql_using_cursor==============
===========ha_innobase::write_row=========
======================ha_innobase::intrinsic_table_write_row==================
======================row_insert_for_mysql_using_cursor==============
.................................
.................................
结论
在遍历innodb_buffer_pool中每一个page上时，将采集到的信息存储在一个内部heap临时表中。当内存表达到上限后，mysql会通过create_ondisk_from_heap( )函数创建磁盘临时表（该磁盘临时表会放在ibtmp1临时共享表空间中），并将内存表中已有数据拷贝到磁盘临时表中，后续对该表的写入也是写入磁盘临时表中。如果用的是innodb磁盘临时表，写入时会产生大量脏页，脏页刷盘时产生磁盘io，innodb_buffer_pool越大产生脏页面越多，对磁盘io影响越大。
问题3 （是否会堵塞DML？）
# 测试
sysbench 压测oltp.lua期间执行select * from information_schema.innodb_buffer_page;
# 结果
不会堵塞DML执行，但是磁盘Io打满会影响sql响应时间
