总的说来事务池的管理主要是PoolManager进行的,然后PoolManager会调用Pool的方法,每一个pool都有一个elem的指针的优先队列
其内存完全在Pool中,而优先队列只是指针,用于分配trx_t.PoolManager可能包含多个pool,每个pool为16个元素,如果在分配的时候没有额外的pool元素,则会新建pool,直到分配完成。
每个事务都在pool存在内存,使用的时候是用的指针指向,也就是其内存在pool的数组中,使用和回收并不会真正的释放内存。只有当系统重启才会析构PoolManager进而析构pool中的内存。
typedef Pool<trx_t, TrxFactory, TrxPoolLock> trx_pool_t;
typedef PoolManager<trx_pool_t, TrxPoolManagerLock > trx_pools_t;
数据结构Pool
template <typename Type, typename Factory, typename LockStrategy>
typedef Type value_type;
struct Element {
Pool* m_pool;
value_type m_type;
};
typedef std::priority_queue<
Element*,
std::vector<Element*, ut_allocator<Element*> >,
std::greater<Element*> > pqueue_t
Element* m_end
Element* m_start
size_t m_size
Element* m_last
pqueue_t m_pqueue
LockStrategy m_lock_strategy
数据结构PoolManager
template <typename Pool, typename LockStrategy>
typedef Pool PoolType;
//Pool<trx_t, TrxFactory, TrxPoolLock> trx_pool_t
typedef typename PoolType::value_type value_type;
//Pool<trx_t, TrxFactory, TrxPoolLock>::value_type trx_pool_t 具体化为 trx_t
typedef std::vector<PoolType*, ut_allocator<PoolType*> > Pools;
size_t m_size;
Pools m_pools;
LockStrategy m_lock_strategy;
pool :内存分配,其中内存是实际trx_t所在的内存和优先队列只用于分配和回收trx_t,其中全是elem的指针
Pool::Pool(size_t size)
->m_lock_strategy.create()
建立锁策略,也就是TrxPoolLock::create
->m_start = reinterpret_cast<Element*>(ut_zalloc_nokey(m_size))
分配内存转换为Element指针,指向头部
->m_last = m_start;
m_last移动指针会用到
->m_end = &m_start[m_size / sizeof(*m_start)]
m_end指向末尾一个元素
->init(ut_min(size_t(16), size_t(m_end - m_start)))
调用Pool::init debug为16,初始化16个元素
->for (size_t i = 0; i < n_elems; ++i, ++m_last)
循环每个元素
->m_last->m_pool = this
每个元素m_pool都指向Pool类
->Factory::init(&m_last->m_type)
调用工厂方法进行创建这里就是,TrxFactory::init,实际上就是在已经有内存的情况下初始化trx_t元素
->先使用placement new语法建立对象
这些对象是需要new出来的
->trx_init(trx)
初始化一些其他对象
->lock_trx_lock_list_init
初始化事务锁链表
->lock_trx_alloc_locks
初始化lock pool
->m_pqueue.push(m_last);
这里m_last是一个指针,放入到std优先队列
PoolManager:由其封装后调用
PoolManager::PoolManager(size_t size)
->m_size(size)
定义大小,这里的大小代表每个pool的内存大小,4M
->m_lock_strategy.create()
创建策略,加锁(注意PoolManager一把锁,每个pool各自都有一把锁)
->PoolManager::add_pool(0) 0为n_pools
初始化的情况下m_pools不可能大于n_pools因此走初始化逻辑
->PoolType* pool
建立指针这里就是trx_pool_t
->pool = UT_NEW_NOKEY(PoolType(m_size))
初始化内存,调用构造函数,Pool::Pool(size_t size)
->m_pools.push_back(pool);
加入vector数组中,这里使用vector数组,难道pool也不止一个?
->解锁
trx_t在Element中,因此返回为&elem->m_type指针
trx_pools_t
| -- m_pools(vector)
| -- trx_pool_t[0] (当前看vector只有一个元素)
/|\ | m_start m_end
| | | |
| | \| \|
| | -- Element[0] Element[1] Element[2] ... Element[15]
|-----(指向)|--*m_pool 同前 同前 同前
|--trx_t(包含了实际的内存)
| -- m_size:pool的内存结构的大小
| -- m_pqueue:Element指针地址的一个优先队列,主要存放和获取trx_t
| -- m_lock_strategy:保护本pool
| -- trx_pool_t[1]同前... 增加与否是要看初始化的16个元素是否使用完,再初始化16个
| -- trx_pool_t[2]同前
...
| -- trx_pool_t[n]同前... n的值为elem是否足够分配了,不会在分配trx_t的时候返回nullptr
| -- m_size:每个pool的内存大小
| -- m_lock_strategy:保护整个trx_pools_t数据结构
p trx_pools->m_pools[0]->m_start[15]->m_type->id
初始化pool,初始化16个Element
trx_pool_init
调用trx_pools = UT_NEW_NOKEY(trx_pools_t(MAX_TRX_BLOCK_SIZE))
获取
innobase_trx_allocate
-->trx_allocate_for_mysql
-->trx_allocate_for_background
-->trx_create_low
->trx_t *trx = trx_pools->get()
从事务池中分配
PoolManager<Pool<trx_t, TrxFactory, TrxPoolLock>, TrxPoolManagerLock>::get
->开启循环
->m_lock_strategy.enter()
事务池加锁
->size_t n_pools = m_pools.size()
获取池的数量,初试情况下为1,但是后面如果不够了会增加
->PoolType *pool = m_pools[index % n_pools]
获取池
->m_lock_strategy.exit()
事务池解锁
->ptr = pool->get()
Pool<trx_t, TrxFactory, TrxPoolLock>::get ------
->m_lock_strategy.enter()
加锁
->if (!m_pqueue.empty())
如果有释放的,必然放入优先队列,使用即可
->elem = m_pqueue.top();
->m_pqueue.pop();
->else if (m_last < m_end)
如果有未初始化的初始化后使用,何时才有未初始化的?
->init(m_end - m_last)
初始化剩余的空间
->elem = m_pqueue.top()
->m_pqueue.pop()
->else
->elem = nullptr
否则为nullptr
->m_lock_strategy.exit()
解锁
->return (elem != nullptr ? &elem->m_type : nullptr)
这里根据elem的指针是否为nullptr,返回elem->m_type的地址
->if (ptr == nullptr && (index / n_pools) > 2)
如果没有获取到trx_t,则开启新的事务池了
-> if (!add_pool(n_pools))
如果增加pool报错则报错
Failed to allocate memory for a pool of size %zu bytes. Will wait for %zu seconds for a thread to free a resource
-> ++index
-> while (ptr == nullptr)
循环的结束条件是tpr必须分配成功,如果第一次失败会增加pool,再次调用Pool::get继续获取
->返回
return (ptr)
#0 Pool<trx_t, TrxFactory, TrxPoolLock>::get (this=0x2fef6f8) at /opt/percona-server-locks-detail-5.7.22/storage/innobase/include/ut0pool.h:103
#1 0x0000000001bb31ee in PoolManager<Pool<trx_t, TrxFactory, TrxPoolLock>, TrxPoolManagerLock>::get (this=0x2fef578) at /opt/percona-server-locks-detail-5.7.22/storage/innobase/include/ut0pool.h:243
#2 0x0000000001baa578 in trx_create_low () at /opt/percona-server-locks-detail-5.7.22/storage/innobase/trx/trx0trx.cc:463
#3 0x0000000001baab2b in trx_allocate_for_background () at /opt/percona-server-locks-detail-5.7.22/storage/innobase/trx/trx0trx.cc:543
#4 0x0000000001baab57 in trx_allocate_for_mysql () at /opt/percona-server-locks-detail-5.7.22/storage/innobase/trx/trx0trx.cc:559
#5 0x000000000196fd45 in innobase_trx_allocate (thd=0x7ffedc017280) at /opt/percona-server-locks-detail-5.7.22/storage/innobase/handler/ha_innodb.cc:2893
#6 0x000000000196fde7 in check_trx_exists (thd=0x7ffedc017280) at /opt/percona-server-locks-detail-5.7.22/storage/innobase/handler/ha_innodb.cc:2918
#7 0x0000000001989af5 in ha_innobase::extra (this=0x7ffedc932030, operation=HA_EXTRA_IS_ATTACHED_CHILDREN) at /opt/percona-server-locks-detail-5.7.22/storage/innobase/handler/ha_innodb.cc:16332
#8 0x00000000014f0414 in Table_cache::get_table (this=0x2dabbc0 <table_cache_manager>, thd=0x7ffedc017280, hash_value=1969222498, key=0x7ffedc9357cd "test9", key_length=11, share=0x7fff59540a88) at /opt/percona-server-locks-detail-5.7.22/sql/table_cache.h:504
#9 0x00000000014dfa3c in open_table (thd=0x7ffedc017280, table_list=0x7ffedc935408, ot_ctx=0x7fff59540c60) at /opt/percona-server-locks-detail-5.7.22/sql/sql_base.cc:3314
#10 0x00000000014e2f54 in open_and_process_table (thd=0x7ffedc017280, lex=0x7ffedc0198b0, tables=0x7ffedc935408, counter=0x7ffedc019970, flags=0, prelocking_strategy=0x7fff59540d60, has_prelocking_list=false, ot_ctx=0x7fff59540c60) at /opt/percona-server-locks-detail-5.7.22/sql/sql_base.cc:5213
#11 0x00000000014e4067 in open_tables (thd=0x7ffedc017280, start=0x7fff59540d20, counter=0x7ffedc019970, flags=0, prelocking_strategy=0x7fff59540d60) at /opt/percona-server-locks-detail-5.7.22/sql/sql_base.cc:5831
#12 0x00000000014e5395 in open_tables_for_query (thd=0x7ffedc017280, tables=0x7ffedc935408, flags=0) at /opt/percona-server-locks-detail-5.7.22/sql/sql_base.cc:6606
#13 0x0000000001782741 in Sql_cmd_delete::mysql_delete (this=0x7ffedc9359a0, thd=0x7ffedc017280, limit=1) at /opt/percona-server-locks-detail-5.7.22/sql/sql_delete.cc:76
#14 0x0000000001786b7e in Sql_cmd_delete::execute (this=0x7ffedc9359a0, thd=0x7ffedc017280) at /opt/percona-server-locks-detail-5.7.22/sql/sql_delete.cc:1386
#15 0x000000000156bce2 in mysql_execute_command (thd=0x7ffedc017280, first_level=true) at /opt/percona-server-locks-detail-5.7.22/sql/sql_parse.cc:3756
#16 0x0000000001571bed in mysql_parse (thd=0x7ffedc017280, parser_state=0x7fff595425b0) at /opt/percona-server-locks-detail-5.7.22/sql/sql_parse.cc:5901
#17 0x000000000156673d in dispatch_command (thd=0x7ffedc017280, com_data=0x7fff59542d90, command=COM_QUERY) at /opt/percona-server-locks-detail-5.7.22/sql/sql_parse.cc:1490
#18 0x00000000015655c5 in do_command (thd=0x7ffedc017280) at /opt/percona-server-locks-detail-5.7.22/sql/sql_parse.cc:1021
#19 0x00000000016a635c in handle_connection (arg=0x65bc5d0) at /opt/percona-server-locks-detail-5.7.22/sql/conn_handler/connection_handler_per_thread.cc:312
#20 0x00000000018ce0f6 in pfs_spawn_thread (arg=0x6576d00) at /opt/percona-server-locks-detail-5.7.22/storage/perfschema/pfs.cc:2190
#21 0x00007ffff7bc6ea5 in start_thread () from /lib64/libpthread.so.0
#22 0x00007ffff66008dd in clone () from /lib64/libc.so.6
加入(释放)
#0 Pool<trx_t, TrxFactory, TrxPoolLock>::put (this=0x7fffdc00a080, elem=0x7fffe0042070) at /newdata/mysql-8.0.23/storage/innobase/include/ut0pool.h:155
#1 0x0000000004c61c58 in Pool<trx_t, TrxFactory, TrxPoolLock>::mem_free (ptr=0x7fffe0042078) at /newdata/mysql-8.0.23/storage/innobase/include/ut0pool.h:131
#2 0x0000000004c60a82 in PoolManager<Pool<trx_t, TrxFactory, TrxPoolLock>, TrxPoolManagerLock>::mem_free (ptr=0x7fffe0042078) at /newdata/mysql-8.0.23/storage/innobase/include/ut0pool.h:253
#3 0x0000000004c55c09 in trx_free (trx=@0x7fffc81a4778: 0x7fffe0042078) at /newdata/mysql-8.0.23/storage/innobase/trx/trx0trx.cc:508
#4 0x0000000004c56134 in trx_free_for_background (trx=0x7fffe0042078) at /newdata/mysql-8.0.23/storage/innobase/trx/trx0trx.cc:589
#5 0x0000000004c56445 in trx_free_for_mysql (trx=0x7fffe0042078) at /newdata/mysql-8.0.23/storage/innobase/trx/trx0trx.cc:670
#6 0x000000000492be2d in innobase_close_connection (hton=0xa3bb890, thd=0x7ffefc0ebd00) at /newdata/mysql-8.0.23/storage/innobase/handler/ha_innodb.cc:5770
#7 0x00000000035ae0ad in closecon_handlerton (thd=0x7ffefc0ebd00, plugin=0x7fffc81a4a98) at /newdata/mysql-8.0.23/sql/handler.cc:917
#8 0x0000000003272d3a in plugin_foreach_with_mask (thd=0x7ffefc0ebd00, funcs=0x7fffc81a4b30, type=1, state_mask=4294967287, arg=0x0) at /newdata/mysql-8.0.23/sql/sql_plugin.cc:2742
#9 0x0000000003272dfa in plugin_foreach_with_mask (thd=0x7ffefc0ebd00, func=0x35ae032 <closecon_handlerton(THD*, plugin_ref, void*)>, type=1, state_mask=8, arg=0x0) at /newdata/mysql-8.0.23/sql/sql_plugin.cc:2755
#10 0x00000000035ae0fb in ha_close_connection (thd=0x7ffefc0ebd00) at /newdata/mysql-8.0.23/sql/handler.cc:929
#11 0x00000000031a6371 in THD::release_resources (this=0x7ffefc0ebd00) at /newdata/mysql-8.0.23/sql/sql_class.cc:1047
#12 0x000000000340dfff in handle_connection (arg=0x8a63490) at /newdata/mysql-8.0.23/sql/conn_handler/connection_handler_per_thread.cc:308
#13 0x000000000500dc16 in pfs_spawn_thread (arg=0xa4c97e0) at /newdata/mysql-8.0.23/storage/perfschema/pfs.cc:2900
#14 0x00007ffff7bc6ea5 in start_thread () from /lib64/libpthread.so.0
#15 0x00007ffff61418dd in clone () from /lib64/libc.so.6
trx_free_for_mysql(此时processlist已经看不到了)
->trx_disconnect_plain
将trx_t从链表中摘下,关闭read view,等操作
->trx_free_for_background
->trx_free
->PoolManager::mem_free(value_type *ptr)
PoolManager<Pool<trx_t, TrxFactory, TrxPoolLock>, TrxPoolManagerLock>::mem_free
->PoolType::mem_free(ptr)
Pool<trx_t, TrxFactory, TrxPoolLock>::mem_free
->byte *p = reinterpret_cast<byte *>(ptr + 1)
->elem = reinterpret_cast<Element *>(p - sizeof(*elem))
移动指针到elem的开头,也就是这个elem内存开始的位置
->elem->m_pool->put(elem)
->Pool::put
->m_lock_strategy.enter()
加锁
->m_pqueue.push(elem)
仅仅是加入到优先队列
->m_lock_strategy.exit()
解锁