innodb:事务池

总的说来事务池的管理主要是PoolManager进行的,然后PoolManager会调用Pool的方法,每一个pool都有一个elem的指针的优先队列
其内存完全在Pool中,而优先队列只是指针,用于分配trx_t.PoolManager可能包含多个pool,每个pool为16个元素,如果在分配的时候没有额外的pool元素,则会新建pool,直到分配完成。
每个事务都在pool存在内存,使用的时候是用的指针指向,也就是其内存在pool的数组中,使用和回收并不会真正的释放内存。只有当系统重启才会析构PoolManager进而析构pool中的内存。

typedef Pool<trx_t, TrxFactory, TrxPoolLock> trx_pool_t;
typedef PoolManager<trx_pool_t, TrxPoolManagerLock > trx_pools_t;

数据结构Pool

template <typename Type, typename Factory, typename LockStrategy> 
  typedef Type value_type;
    struct Element {
        Pool*       m_pool;
        value_type  m_type;
    };
    
    typedef std::priority_queue<
    Element*,
    std::vector<Element*, ut_allocator<Element*> >,  
    std::greater<Element*> >    pqueue_t
    
    Element*        m_end
    Element*        m_start
    size_t          m_size
    Element*        m_last
    pqueue_t        m_pqueue
    LockStrategy        m_lock_strategy

数据结构PoolManager

template <typename Pool, typename LockStrategy>
  typedef Pool PoolType;
  //Pool<trx_t, TrxFactory, TrxPoolLock> trx_pool_t
  typedef typename PoolType::value_type value_type;
  //Pool<trx_t, TrxFactory, TrxPoolLock>::value_type  trx_pool_t 具体化为 trx_t

  typedef std::vector<PoolType*, ut_allocator<PoolType*> >  Pools;
    size_t      m_size;
    Pools       m_pools;
    LockStrategy        m_lock_strategy;

pool :内存分配,其中内存是实际trx_t所在的内存和优先队列只用于分配和回收trx_t,其中全是elem的指针

Pool::Pool(size_t size)
 ->m_lock_strategy.create()
     建立锁策略,也就是TrxPoolLock::create
 ->m_start = reinterpret_cast<Element*>(ut_zalloc_nokey(m_size))
   分配内存转换为Element指针,指向头部
 ->m_last = m_start; 
   m_last移动指针会用到
 ->m_end = &m_start[m_size / sizeof(*m_start)]
   m_end指向末尾一个元素
 ->init(ut_min(size_t(16), size_t(m_end - m_start)))
   调用Pool::init debug为16,初始化16个元素
   ->for (size_t i = 0; i < n_elems; ++i, ++m_last)
     循环每个元素
     ->m_last->m_pool = this
       每个元素m_pool都指向Pool类
     ->Factory::init(&m_last->m_type)
       调用工厂方法进行创建这里就是,TrxFactory::init,实际上就是在已经有内存的情况下初始化trx_t元素
       ->先使用placement new语法建立对象
         这些对象是需要new出来的
       ->trx_init(trx)
         初始化一些其他对象
       ->lock_trx_lock_list_init
         初始化事务锁链表
       ->lock_trx_alloc_locks
         初始化lock pool
 ->m_pqueue.push(m_last); 
   这里m_last是一个指针,放入到std优先队列     

PoolManager:由其封装后调用

PoolManager::PoolManager(size_t size) 
  ->m_size(size)
   定义大小,这里的大小代表每个pool的内存大小,4M
  ->m_lock_strategy.create()
   创建策略,加锁(注意PoolManager一把锁,每个pool各自都有一把锁)
  ->PoolManager::add_pool(0)  0为n_pools
   初始化的情况下m_pools不可能大于n_pools因此走初始化逻辑
  ->PoolType*   pool
    建立指针这里就是trx_pool_t
  ->pool = UT_NEW_NOKEY(PoolType(m_size))
    初始化内存,调用构造函数,Pool::Pool(size_t size)
  ->m_pools.push_back(pool);
    加入vector数组中,这里使用vector数组,难道pool也不止一个?
  ->解锁

trx_t在Element中,因此返回为&elem->m_type指针

trx_pools_t
| -- m_pools(vector)
      | --  trx_pool_t[0] (当前看vector只有一个元素)
            /|\  |    m_start                                           m_end     
             |   |      |                                                 |
             |   |     \|                                                \| 
             |   | -- Element[0]       Element[1]      Element[2]  ... Element[15]
             |-----(指向)|--*m_pool     同前            同前             同前
                         |--trx_t(包含了实际的内存)
                 | -- m_size:pool的内存结构的大小
                 | -- m_pqueue:Element指针地址的一个优先队列,主要存放和获取trx_t
                 | -- m_lock_strategy:保护本pool
      | -- trx_pool_t[1]同前... 增加与否是要看初始化的16个元素是否使用完,再初始化16个
      | -- trx_pool_t[2]同前
      ...
      | -- trx_pool_t[n]同前... n的值为elem是否足够分配了,不会在分配trx_t的时候返回nullptr
| -- m_size:每个pool的内存大小
| -- m_lock_strategy:保护整个trx_pools_t数据结构
  
p trx_pools->m_pools[0]->m_start[15]->m_type->id

初始化pool,初始化16个Element

  trx_pool_init
  调用trx_pools = UT_NEW_NOKEY(trx_pools_t(MAX_TRX_BLOCK_SIZE))

获取
  innobase_trx_allocate
    -->trx_allocate_for_mysql
       -->trx_allocate_for_background
          -->trx_create_low
             ->trx_t *trx = trx_pools->get()
               从事务池中分配
               PoolManager<Pool<trx_t, TrxFactory, TrxPoolLock>, TrxPoolManagerLock>::get
               ->开启循环
                 ->m_lock_strategy.enter()
                   事务池加锁
                 ->size_t n_pools = m_pools.size()
                   获取池的数量,初试情况下为1,但是后面如果不够了会增加
                 ->PoolType *pool = m_pools[index % n_pools]
                   获取池
                 ->m_lock_strategy.exit()
                   事务池解锁
                 ->ptr = pool->get()
                   Pool<trx_t, TrxFactory, TrxPoolLock>::get ------
                   ->m_lock_strategy.enter()
                     加锁
                   ->if (!m_pqueue.empty())
                     如果有释放的,必然放入优先队列,使用即可
                     ->elem = m_pqueue.top();
                     ->m_pqueue.pop();
                   ->else if (m_last < m_end)
                     如果有未初始化的初始化后使用,何时才有未初始化的?
                     ->init(m_end - m_last)
                       初始化剩余的空间
                     ->elem = m_pqueue.top()
                     ->m_pqueue.pop()
                   ->else
                     ->elem = nullptr
                       否则为nullptr
                   ->m_lock_strategy.exit()
                     解锁
                   ->return (elem != nullptr ? &elem->m_type : nullptr)
                     这里根据elem的指针是否为nullptr,返回elem->m_type的地址
               ->if (ptr == nullptr && (index / n_pools) > 2)
                 如果没有获取到trx_t,则开启新的事务池了
                 -> if (!add_pool(n_pools))
                    如果增加pool报错则报错
                    Failed to allocate memory for a pool of size %zu bytes. Will wait for %zu seconds for a thread to free a resource
               -> ++index
               -> while (ptr == nullptr)   
                 循环的结束条件是tpr必须分配成功,如果第一次失败会增加pool,再次调用Pool::get继续获取        
               ->返回
                 return (ptr)   

#0  Pool<trx_t, TrxFactory, TrxPoolLock>::get (this=0x2fef6f8) at /opt/percona-server-locks-detail-5.7.22/storage/innobase/include/ut0pool.h:103
#1  0x0000000001bb31ee in PoolManager<Pool<trx_t, TrxFactory, TrxPoolLock>, TrxPoolManagerLock>::get (this=0x2fef578) at /opt/percona-server-locks-detail-5.7.22/storage/innobase/include/ut0pool.h:243
#2  0x0000000001baa578 in trx_create_low () at /opt/percona-server-locks-detail-5.7.22/storage/innobase/trx/trx0trx.cc:463
#3  0x0000000001baab2b in trx_allocate_for_background () at /opt/percona-server-locks-detail-5.7.22/storage/innobase/trx/trx0trx.cc:543
#4  0x0000000001baab57 in trx_allocate_for_mysql () at /opt/percona-server-locks-detail-5.7.22/storage/innobase/trx/trx0trx.cc:559
#5  0x000000000196fd45 in innobase_trx_allocate (thd=0x7ffedc017280) at /opt/percona-server-locks-detail-5.7.22/storage/innobase/handler/ha_innodb.cc:2893
#6  0x000000000196fde7 in check_trx_exists (thd=0x7ffedc017280) at /opt/percona-server-locks-detail-5.7.22/storage/innobase/handler/ha_innodb.cc:2918
#7  0x0000000001989af5 in ha_innobase::extra (this=0x7ffedc932030, operation=HA_EXTRA_IS_ATTACHED_CHILDREN) at /opt/percona-server-locks-detail-5.7.22/storage/innobase/handler/ha_innodb.cc:16332
#8  0x00000000014f0414 in Table_cache::get_table (this=0x2dabbc0 <table_cache_manager>, thd=0x7ffedc017280, hash_value=1969222498, key=0x7ffedc9357cd "test9", key_length=11, share=0x7fff59540a88) at /opt/percona-server-locks-detail-5.7.22/sql/table_cache.h:504
#9  0x00000000014dfa3c in open_table (thd=0x7ffedc017280, table_list=0x7ffedc935408, ot_ctx=0x7fff59540c60) at /opt/percona-server-locks-detail-5.7.22/sql/sql_base.cc:3314
#10 0x00000000014e2f54 in open_and_process_table (thd=0x7ffedc017280, lex=0x7ffedc0198b0, tables=0x7ffedc935408, counter=0x7ffedc019970, flags=0, prelocking_strategy=0x7fff59540d60, has_prelocking_list=false, ot_ctx=0x7fff59540c60) at /opt/percona-server-locks-detail-5.7.22/sql/sql_base.cc:5213
#11 0x00000000014e4067 in open_tables (thd=0x7ffedc017280, start=0x7fff59540d20, counter=0x7ffedc019970, flags=0, prelocking_strategy=0x7fff59540d60) at /opt/percona-server-locks-detail-5.7.22/sql/sql_base.cc:5831
#12 0x00000000014e5395 in open_tables_for_query (thd=0x7ffedc017280, tables=0x7ffedc935408, flags=0) at /opt/percona-server-locks-detail-5.7.22/sql/sql_base.cc:6606
#13 0x0000000001782741 in Sql_cmd_delete::mysql_delete (this=0x7ffedc9359a0, thd=0x7ffedc017280, limit=1) at /opt/percona-server-locks-detail-5.7.22/sql/sql_delete.cc:76
#14 0x0000000001786b7e in Sql_cmd_delete::execute (this=0x7ffedc9359a0, thd=0x7ffedc017280) at /opt/percona-server-locks-detail-5.7.22/sql/sql_delete.cc:1386
#15 0x000000000156bce2 in mysql_execute_command (thd=0x7ffedc017280, first_level=true) at /opt/percona-server-locks-detail-5.7.22/sql/sql_parse.cc:3756
#16 0x0000000001571bed in mysql_parse (thd=0x7ffedc017280, parser_state=0x7fff595425b0) at /opt/percona-server-locks-detail-5.7.22/sql/sql_parse.cc:5901
#17 0x000000000156673d in dispatch_command (thd=0x7ffedc017280, com_data=0x7fff59542d90, command=COM_QUERY) at /opt/percona-server-locks-detail-5.7.22/sql/sql_parse.cc:1490
#18 0x00000000015655c5 in do_command (thd=0x7ffedc017280) at /opt/percona-server-locks-detail-5.7.22/sql/sql_parse.cc:1021
#19 0x00000000016a635c in handle_connection (arg=0x65bc5d0) at /opt/percona-server-locks-detail-5.7.22/sql/conn_handler/connection_handler_per_thread.cc:312
#20 0x00000000018ce0f6 in pfs_spawn_thread (arg=0x6576d00) at /opt/percona-server-locks-detail-5.7.22/storage/perfschema/pfs.cc:2190
#21 0x00007ffff7bc6ea5 in start_thread () from /lib64/libpthread.so.0
#22 0x00007ffff66008dd in clone () from /lib64/libc.so.6

加入(释放)

#0  Pool<trx_t, TrxFactory, TrxPoolLock>::put (this=0x7fffdc00a080, elem=0x7fffe0042070) at /newdata/mysql-8.0.23/storage/innobase/include/ut0pool.h:155
#1  0x0000000004c61c58 in Pool<trx_t, TrxFactory, TrxPoolLock>::mem_free (ptr=0x7fffe0042078) at /newdata/mysql-8.0.23/storage/innobase/include/ut0pool.h:131
#2  0x0000000004c60a82 in PoolManager<Pool<trx_t, TrxFactory, TrxPoolLock>, TrxPoolManagerLock>::mem_free (ptr=0x7fffe0042078) at /newdata/mysql-8.0.23/storage/innobase/include/ut0pool.h:253
#3  0x0000000004c55c09 in trx_free (trx=@0x7fffc81a4778: 0x7fffe0042078) at /newdata/mysql-8.0.23/storage/innobase/trx/trx0trx.cc:508
#4  0x0000000004c56134 in trx_free_for_background (trx=0x7fffe0042078) at /newdata/mysql-8.0.23/storage/innobase/trx/trx0trx.cc:589
#5  0x0000000004c56445 in trx_free_for_mysql (trx=0x7fffe0042078) at /newdata/mysql-8.0.23/storage/innobase/trx/trx0trx.cc:670
#6  0x000000000492be2d in innobase_close_connection (hton=0xa3bb890, thd=0x7ffefc0ebd00) at /newdata/mysql-8.0.23/storage/innobase/handler/ha_innodb.cc:5770
#7  0x00000000035ae0ad in closecon_handlerton (thd=0x7ffefc0ebd00, plugin=0x7fffc81a4a98) at /newdata/mysql-8.0.23/sql/handler.cc:917
#8  0x0000000003272d3a in plugin_foreach_with_mask (thd=0x7ffefc0ebd00, funcs=0x7fffc81a4b30, type=1, state_mask=4294967287, arg=0x0) at /newdata/mysql-8.0.23/sql/sql_plugin.cc:2742
#9  0x0000000003272dfa in plugin_foreach_with_mask (thd=0x7ffefc0ebd00, func=0x35ae032 <closecon_handlerton(THD*, plugin_ref, void*)>, type=1, state_mask=8, arg=0x0) at /newdata/mysql-8.0.23/sql/sql_plugin.cc:2755
#10 0x00000000035ae0fb in ha_close_connection (thd=0x7ffefc0ebd00) at /newdata/mysql-8.0.23/sql/handler.cc:929
#11 0x00000000031a6371 in THD::release_resources (this=0x7ffefc0ebd00) at /newdata/mysql-8.0.23/sql/sql_class.cc:1047
#12 0x000000000340dfff in handle_connection (arg=0x8a63490) at /newdata/mysql-8.0.23/sql/conn_handler/connection_handler_per_thread.cc:308
#13 0x000000000500dc16 in pfs_spawn_thread (arg=0xa4c97e0) at /newdata/mysql-8.0.23/storage/perfschema/pfs.cc:2900
#14 0x00007ffff7bc6ea5 in start_thread () from /lib64/libpthread.so.0
#15 0x00007ffff61418dd in clone () from /lib64/libc.so.6


trx_free_for_mysql(此时processlist已经看不到了)
->trx_disconnect_plain
  将trx_t从链表中摘下,关闭read view,等操作
->trx_free_for_background
  ->trx_free
    ->PoolManager::mem_free(value_type *ptr)
      PoolManager<Pool<trx_t, TrxFactory, TrxPoolLock>, TrxPoolManagerLock>::mem_free
      ->PoolType::mem_free(ptr)
        Pool<trx_t, TrxFactory, TrxPoolLock>::mem_free
        ->byte *p = reinterpret_cast<byte *>(ptr + 1)
        ->elem = reinterpret_cast<Element *>(p - sizeof(*elem))
          移动指针到elem的开头,也就是这个elem内存开始的位置
        ->elem->m_pool->put(elem)
          ->Pool::put
            ->m_lock_strategy.enter()
              加锁
            ->m_pqueue.push(elem)
              仅仅是加入到优先队列
            ->m_lock_strategy.exit()
              解锁
©著作权归作者所有,转载或内容合作请联系作者
【社区内容提示】社区部分内容疑似由AI辅助生成,浏览时请结合常识与多方信息审慎甄别。
平台声明:文章内容(如有图片或视频亦包括在内)由作者上传并发布,文章内容仅代表作者本人观点,简书系信息发布平台,仅提供信息存储服务。

相关阅读更多精彩内容

  • 事务的定义 事务的基本要素(ACID)原子性:Atomicity,整个数据库事务是不可分割的工作单位一致性:Con...
    晓码君阅读 3,269评论 0 1
  • 最近在研究分布式数据库相关的技术,对于数据库来说,不管是单机数据库还是分布式数据库,事务都是一个绕不去的坎。不光是...
    elvinyang阅读 5,206评论 0 2
  • 事务的定义 事务的基本要素(ACID)原子性:Atomicity,整个数据库事务是不可分割的工作单位一致性:Con...
    binecy阅读 2,394评论 0 0
  • author:sufei 源码版本:8.0.16 一、核心结构体 ​ 在理解分析innodb事务系统过...
    真之棒2016阅读 6,409评论 0 2
  • 第一章 1.9 令人困惑的语法 1.9.1 stl_config.h中的各种组态(configurations) ...
    镜中无我阅读 4,719评论 0 0

友情链接更多精彩内容