1.Netty内存分配基础数据结构
Netty会提前申请一块连续的内存(用PoolArena类)表示，然后每一个PoolArena包含一系列的chunk，用poolChunk表示；每个PoolSubpage由大小相等的快(Region)组成。
Netty内存存储中数据结构主要涉及的类：
1.PooledByteBufAllocator
2.PoolArena
3.PoolChunk
4.PoolSubpage
5.PoolChunklist
2.源码分析PooledByteBufAllocator
在netty中，将分配内存用ByteBufAllocator进行抽象化，PoolByteBufBufAllocator就是内存池的分配原理。
2.1 PooledByteBufAllocator静态变量，初始化分析。
private static final int DEFAULT_NUM_HEAP_ARENA;
private static final int DEFAULT_NUM_DIRECT_ARENA;
private static final int DEFAULT_PAGE_SIZE;
private static final int DEFAULT_MAX_ORDER;
private static final int DEFAULT_TINY_CACHE_SIZE;
private static final int DEFAULT_SMALL_CACHE_SIZE;
private static final int DEFAULT_NORMAL_CACHE_SIZE;
private static final int DEFAULT_MAX_CACHED_BUFFER_CAPACITY;
private static final int DEFAULT_CACHE_TRIM_INTERVAL;
private static final int MIN_PAGE_SIZE = 4096;
private static final int MAX_CHUNK_SIZE = (int) (((long) Integer.MAX_VALUE + 1) / 2);
1、DEFAULT_NUM_HEAP_ARENA,PoolArena的个数，从下文的初始化中可以看出，取值为 CPU核心逻辑线程数与 （最大运行内存【堆内存或堆外内存】的大小的二分之一 除以每个Chunk-size,并除以3，是确保每个PoolArena包含三个Chunk）
2、DEFAULT_NUM_DIRECT_ARENA，与上述含义是一样，堆外内存。
3、DEFAULT_PAGE_SIZE，默认pageSize,大小， 最小为4K,默认为8K
4、DEFAULT_MAX_ORDER，每个chunk中的page用平衡二叉树映射管理每个PoolSubpage是否被分配，maxOrder为树的深度，深度为maxOrder层的节点数量为 1 << maxOrder。
5、DEFAULT_TINY_CACHE_SIZE ： 512
6、DEFAULT_SMALL_CACHE_SIZE ： 256
7、DEFAULT_NORMAL_CACHE_SIZE 64
8、MAX_CHUNK_SIZE 等于2的30次方，1G。
//静态代码初始化源码分析
static {
int defaultPageSize = SystemPropertyUtil.getInt("io.netty.allocator.pageSize", 8192);
//1所代表的代码段表示初始化DEFAULT_PAGE_SIZE为默认的8K,PageSize不能小于4K并必须是2的幂
// @1 start
Throwable pageSizeFallbackCause = null;
try {
validateAndCalculatePageShifts(defaultPageSize);
} catch (Throwable t) {
pageSizeFallbackCause = t;
defaultPageSize = 8192;
}
DEFAULT_PAGE_SIZE = defaultPageSize;
//@1 end
//2所代表的代码块，表示初始化maxOrder，maxOrder为树的深度，一个chunk的2的maxOrder幂
//@2 start
int defaultMaxOrder = SystemPropertyUtil.getInt("io.netty.allocator.maxOrder", 11);
Throwable maxOrderFallbackCause = null;
try {
validateAndCalculateChunkSize(DEFAULT_PAGE_SIZE, defaultMaxOrder);
} catch (Throwable t) {
maxOrderFallbackCause = t;
defaultMaxOrder = 11;
}
DEFAULT_MAX_ORDER = defaultMaxOrder; //@2 end
// Determine reasonable default for nHeapArena and nDirectArena.
// Assuming each arena has 3 chunks, the pool should not consume more than 50% of max memory.
final Runtime runtime = Runtime.getRuntime();
final int defaultChunkSize = DEFAULT_PAGE_SIZE << DEFAULT_MAX_ORDER; //@3
DEFAULT_NUM_HEAP_ARENA = Math.max(0, //@4 start
SystemPropertyUtil.getInt(
"io.netty.allocator.numHeapArenas",
(int) Math.min(
runtime.availableProcessors(),
Runtime.getRuntime().maxMemory() / defaultChunkSize / 2 / 3)));
DEFAULT_NUM_DIRECT_ARENA = Math.max(0,
SystemPropertyUtil.getInt(
"io.netty.allocator.numDirectArenas",
(int) Math.min(
runtime.availableProcessors(),
PlatformDependent.maxDirectMemory() / defaultChunkSize / 2 / 3))); //@4 end
// cache sizes
//与本地线程分配有关，tiny内存、small内存的数量，
// @5start
DEFAULT_TINY_CACHE_SIZE = SystemPropertyUtil.getInt("io.netty.allocator.tinyCacheSize", 512);
DEFAULT_SMALL_CACHE_SIZE = SystemPropertyUtil.getInt("io.netty.allocator.smallCacheSize", 256);
DEFAULT_NORMAL_CACHE_SIZE = SystemPropertyUtil.getInt("io.netty.allocator.normalCacheSize", 64);
// 32 kb is the default maximum capacity of the cached buffer. Similar to what is explained in
// 'Scalable memory allocation using jemalloc'
DEFAULT_MAX_CACHED_BUFFER_CAPACITY = SystemPropertyUtil.getInt(
"io.netty.allocator.maxCachedBufferCapacity", 32 * 1024);
// the number of threshold of allocations when cached entries will be freed up if not frequently used
DEFAULT_CACHE_TRIM_INTERVAL = SystemPropertyUtil.getInt(
"io.netty.allocator.cacheTrimInterval", 8192); // @5 end
// 日志输出代码省略
}
}
private static int validateAndCalculatePageShifts(int pageSize) {
if (pageSize < MIN_PAGE_SIZE) {
throw new IllegalArgumentException("pageSize: " + pageSize + " (expected: " + MIN_PAGE_SIZE + "+)");
}
if ((pageSize & pageSize - 1) != 0) {
throw new IllegalArgumentException("pageSize: " + pageSize + " (expected: power of 2)");
}
// Logarithm base 2. At this point we know that pageSize is a power of two.
return Integer.SIZE - 1 - Integer.numberOfLeadingZeros(pageSize);
}
private static int validateAndCalculateChunkSize(int pageSize, int maxOrder) {
if (maxOrder > 14) {
throw new IllegalArgumentException("maxOrder: " + maxOrder + " (expected: 0-14)");
}
// Ensure the resulting chunkSize does not overflow.
int chunkSize = pageSize;
for (int i = maxOrder; i > 0; i --) {
if (chunkSize > MAX_CHUNK_SIZE / 2) {
throw new IllegalArgumentException(String.format(
"pageSize (%d) << maxOrder (%d) must not exceed %d", pageSize, maxOrder, MAX_CHUNK_SIZE));
}
chunkSize <<= 1;
}
return chunkSize;
}
2.2PooledByteBufAllocator构造函数解析
/**
* @param preferDirect 是否倾向于使用直接内存
* @param nHeapArena 连续的堆内存数量(PoolArena)的个数
* @param nDirectArena 连续的堆外内存数量(PoolArena)的个数
* @param pageSize 页的内存大小，默认8192 8K
* @param maxOrder chunk的组织是用一颗平衡二叉树来表示的，maxOrder树的深度,一个chunk由2的maxOrder幂组成，maxOrder默认为11，最大为14
* @param tinyCacheSize,默认512,
* @param smallCacheSize默认为256
* @param normalCacheSize 默认为64
*/
public PooledByteBufAllocator(boolean preferDirect, int nHeapArena, int nDirectArena, int pageSize, int maxOrder,
int tinyCacheSize, int smallCacheSize, int normalCacheSize) {
super(preferDirect);
threadCache = new PoolThreadLocalCache(); // @1
this.tinyCacheSize = tinyCacheSize;
this.smallCacheSize = smallCacheSize;
this.normalCacheSize = normalCacheSize;
final int chunkSize = validateAndCalculateChunkSize(pageSize, maxOrder);
if (nHeapArena < 0) {
throw new IllegalArgumentException("nHeapArena: " + nHeapArena + " (expected: >= 0)");
}
if (nDirectArena < 0) {
throw new IllegalArgumentException("nDirectArea: " + nDirectArena + " (expected: >= 0)");
}
int pageShifts = validateAndCalculatePageShifts(pageSize); // @2
if (nHeapArena > 0) { //@3 start
heapArenas = newArenaArray(nHeapArena);
for (int i = 0; i < heapArenas.length; i ++) {
heapArenas[i] = new PoolArena.HeapArena(this, pageSize, maxOrder, pageShifts, chunkSize);
}
} else {
heapArenas = null;
}
if (nDirectArena > 0) {
directArenas = newArenaArray(nDirectArena);
for (int i = 0; i < directArenas.length; i ++) {
directArenas[i] = new PoolArena.DirectArena(this, pageSize, maxOrder, pageShifts, chunkSize);
}
} else {
directArenas = null;
} // @3 end
}
对于上面代码中的讲解：
@1：PoolThreadLocalCache该类在内存分配事将重点讲解，目前先这样理解，在一个线程的上下文，线程从相同的PoolArena中去申请内存。
@2：pageShifts，比如pageSize为8192, 2的13次方法，那pageShifts为Integer.SIZE - 1 - Integer.numberOfLeadingZeros(pageSize);=13，
参考：https://blog.csdn.net/prestigeding/article/details/53977445
