大师兄的Python源码学习笔记(五十）: Python的内存管理机制(五）
大师兄的Python源码学习笔记(五十二）: Python的内存管理机制(七）
三、内存池
3. block的释放
- 对block的释放实际上就是将一块block归还给pool。
- 我们知道pool可能有三种状态，在不同状态，他们的位置是不同的。
- 当释放block后，可能会引起pool的状态转变，这种转变可分为两种情况：
- used状态转变为empty状态。
- full状态转变为used状态。
- 最多的情况是pool中尽管释放并收回了一个block，但他仍处于used状态，这是最简单的情况：
Objects\obmalloc.c
/* Free a memory block allocated by pymalloc_alloc().
Return 1 if it was freed.
Return 0 if the block was not allocated by pymalloc_alloc(). */
static int
pymalloc_free(void *ctx, void *p)
{
poolp pool;
block *lastfree;
poolp next, prev;
uint size;
assert(p != NULL);
... ...
pool = POOL_ADDR(p);
if (!address_in_range(p, pool)) {
return 0;
}
/* We allocated this address. */
LOCK();
/* Link p to the start of the pool's freeblock list. Since
* the pool had at least the p block outstanding, the pool
* wasn't empty (so it's already in a usedpools[] list, or
* was full and is in no list -- it's not in the freeblocks
* list in any case).
*/
assert(pool->ref.count > 0); /* else it was empty */
*(block **)p = lastfree = pool->freeblock;
pool->freeblock = (block *)p;
if (!lastfree) {
... ...
}
struct arena_object* ao;
uint nf; /* ao->nfreepools */
/* freeblock wasn't NULL, so the pool wasn't full,
* and the pool is in a usedpools[] list.
*/
if (--pool->ref.count != 0) {
/* pool isn't empty: leave it in usedpools */
goto success;
}
/* Pool is now empty: unlink from usedpools, and
* link to the front of freepools. This ensures that
* previously freed pools will be allocated later
* (being not referenced, they are perhaps paged out).
*/
next = pool->nextpool;
prev = pool->prevpool;
next->prevpool = prev;
prev->nextpool = next;
/* Link the pool to freepools. This is a singly-linked
* list, and pool->prevpool isn't used there.
*/
ao = &arenas[pool->arenaindex];
pool->nextpool = ao->freepools;
ao->freepools = pool;
nf = ++ao->nfreepools;
... ...
}
- 在pool的状态保持used状态的情况下，Python仅仅将block重新放入到自由block链表中，并调整pool的
ref.count引用计数。 - 如果释放block之前，其所属的pool处于full状态，则将pool重新链回usedpools中即可。
- 最复杂的情况发生在pool在收回block前后状态从used状态转为empty的情况:
- 在这种情况下，首先需要将empty状态的pool链入到freepools中。
Objects\obmalloc.c static int pymalloc_free(void *ctx, void *p) { poolp pool; block *lastfree; poolp next, prev; uint size; ... ... pool = POOL_ADDR(p); if (!address_in_range(p, pool)) { return 0; } /* We allocated this address. */ ... ... /* Link p to the start of the pool's freeblock list. Since * the pool had at least the p block outstanding, the pool * wasn't empty (so it's already in a usedpools[] list, or * was full and is in no list -- it's not in the freeblocks * list in any case). */ assert(pool->ref.count > 0); /* else it was empty */ *(block **)p = lastfree = pool->freeblock; pool->freeblock = (block *)p; ... .... }
- 到这里又分成四种情况：
Objects\obmalloc.c static int pymalloc_free(void *ctx, void *p) { poolp pool; block *lastfree; poolp next, prev; uint size; ... ... /* Link the pool to freepools. This is a singly-linked * list, and pool->prevpool isn't used there. */ ao = &arenas[pool->arenaindex]; pool->nextpool = ao->freepools; ao->freepools = pool; nf = ++ao->nfreepools; /* All the rest is arena management. We just freed * a pool, and there are 4 cases for arena mgmt: * 1. If all the pools are free, return the arena to * the system free(). * 2. If this is the only free pool in the arena, * add the arena back to the `usable_arenas` list. * 3. If the "next" arena has a smaller count of free * pools, we have to "slide this arena right" to * restore that usable_arenas is sorted in order of * nfreepools. * 4. Else there's nothing more to do. */ if (nf == ao->ntotalpools) { /* Case 1. First unlink ao from usable_arenas. */ assert(ao->prevarena == NULL || ao->prevarena->address != 0); assert(ao ->nextarena == NULL || ao->nextarena->address != 0); /* Fix the pointer in the prevarena, or the * usable_arenas pointer. */ if (ao->prevarena == NULL) { usable_arenas = ao->nextarena; assert(usable_arenas == NULL || usable_arenas->address != 0); } else { assert(ao->prevarena->nextarena == ao); ao->prevarena->nextarena = ao->nextarena; } /* Fix the pointer in the nextarena. */ if (ao->nextarena != NULL) { assert(ao->nextarena->prevarena == ao); ao->nextarena->prevarena = ao->prevarena; } /* Record that this arena_object slot is * available to be reused. */ ao->nextarena = unused_arena_objects; unused_arena_objects = ao; /* Free the entire arena. */ _PyObject_Arena.free(_PyObject_Arena.ctx, (void *)ao->address, ARENA_SIZE); ao->address = 0; /* mark unassociated */ --narenas_currently_allocated; goto success; } ... ... }
- 如果arena中所有的pool全都是empty的，则释放pool集合占用的内存，并将arena的状态调整为unused。
- 如果之前arena中没有empty的pool，那么usable_arenas链表中就找不到该arena，由于现在arena中有了一个pool，所以需要将这个arena链入到usable_arenas链表的表头。
- 如果arena中的empty的pool个数为n，则从usable_arenas开始寻找arena可以插入的位置，并将arena插入到usable_arenas。（这个操作的原因是usable_arenas实际是一个有序链表，每一个arena中的empty的pool个数，即nfreepools，都不能大于前面的arena，也不能小于前面的arena。保持这种有序性的原因是分配block时，是从usable_arenas的表头开始寻找可用的arena，这样可以保证当一个arena的empty pool数量越多，被使用的机会就越少，因此最终释放其维护的pool集合内存的机会就越大，可以保证多余的内存会被归还给系统。）
- 其他情况，不对arena进行任何处理。
4. 内存池全景
- 对于一个C开发的庞大软件，其内存管理可能是最复杂最繁琐的部分。
- 尽管各种不同的链表变幻无常，但所有的内存都在arena的掌控之中：
