我一直都有一个疑问, 丰巢业务服务的生产环境 jvm 参数设置是禁止 system.gc 的, 也就是开启设置:-XX:+DisableExplicitGC, 但是生产环境却从来没有出现过堆外内存溢出的情况. 说明一下, 丰巢使用了阿里开源的 dubbo, 而 dubbo 底层通信默认情况下使用了 3.2.5.Final 版本的 netty, 而我们对于 netty 的常规认知里, netty 一定是使用了堆外内存, 并且堆外内存在禁止了 system.gc 这个函数调用的话, 在服务没有主动回收分配的堆外内存的情况下, 一定会出现堆外内存的泄露. 带着这个问题, 刚好前天晚上有些时间, 研究了一下 3.2.5 版本的 netty 源码, 又是在科兴科兴园等馒头妈妈时候, 发现了秘密之所在, 我只能说, 科兴科学园真是我的宝地啊.
涉及到的 netty 类: NioWorker,HeapChannelBufferFactory,BigEndianHeapChannelBuffer,SocketReceiveBufferPool
核心的秘密在 SocketReceiveBufferPool 中
- final class SocketReceiveBufferPool {
- private static final int POOL_SIZE = 8;
- @SuppressWarnings("unchecked")
- private final SoftReference<ByteBuffer>[] pool = new SoftReference[POOL_SIZE];
- SocketReceiveBufferPool() {
- super();
- }
- final ByteBuffer acquire(int size) {
- final SoftReference<ByteBuffer>[] pool = this.pool;
- for (int i = 0; i <POOL_SIZE; i ++) {
- SoftReference<ByteBuffer> ref = pool[i];
- if (ref == null) {
- continue;
- }
- ByteBuffer buf = ref.get();
- if (buf == null) {
- pool[i] = null;
- continue;
- }
- if (buf.capacity() <size) {
- continue;
- }
- pool[i] = null;
- buf.clear();
- return buf;
- }
- ByteBuffer buf = ByteBuffer.allocateDirect(normalizeCapacity(size));
- buf.clear();
- return buf;
- }
- final void release(ByteBuffer buffer) {
- final SoftReference<ByteBuffer>[] pool = this.pool;
- for (int i = 0; i <POOL_SIZE; i ++) {
- SoftReference<ByteBuffer> ref = pool[i];
- if (ref == null || ref.get() == null) {
- pool[i] = new SoftReference<ByteBuffer>(buffer);
- return;
- }
- }
- // pool is full - replace one
- final int capacity = buffer.capacity();
- for (int i = 0; i<POOL_SIZE; i ++) {
- SoftReference<ByteBuffer> ref = pool[i];
- ByteBuffer pooled = ref.get();
- if (pooled == null) {
- pool[i] = null;
- continue;
- }
- if (pooled.capacity() <capacity) {
- pool[i] = new SoftReference<ByteBuffer>(buffer);
- return;
- }
- }
- }
- private static final int normalizeCapacity(int capacity) {
- // Normalize to multiple of 1024
- int q = capacity>>> 10;
- int r = capacity & 1023;
- if (r != 0) {
- q ++;
- }
- return q <<10;
- }
- }
SocketReceiveBufferPool 中维护了一个 SoftReference<ByteBuffer > 类型的数组, 关于 java 的 SoftReference, 大家可以自行搜索. 其实就是在此类中维护了一个 directbuffer 的内存池, 此部分的内存是可以重复利用的. 那么问题来了, 如果我们把 netty 用于接收网络信息的 directbuffer 直接传给 dubbo 的业务代码, 那么这个内存池的作用是什么呢, 内存如何被 release 回内存池? 带着这个疑问, 继续分析调用了 SocketReceiveBufferPool 的 NioWorker 代码.
- private boolean read(SelectionKey k) {
- final SocketChannel ch = (SocketChannel) k.channel();
- final NioSocketChannel channel = (NioSocketChannel) k.attachment();
- final ReceiveBufferSizePredictor predictor =
- channel.getConfig().getReceiveBufferSizePredictor();
- final int predictedRecvBufSize = predictor.nextReceiveBufferSize();
- int ret = 0;
- int readBytes = 0;
- boolean failure = true;
- ByteBuffer bb = recvBufferPool.acquire(predictedRecvBufSize);
- 15 try {
- while ((ret = ch.read(bb))> 0) {
- readBytes += ret;
- if (!bb.hasRemaining()) {
- break;
- }
- }
- failure = false;
- } catch (ClosedChannelException e) {
- // Can happen, and does not need a user attention.
- } catch (Throwable t) {
- fireExceptionCaught(channel, t);
- }
- if (readBytes> 0) {
- bb.flip();
- final ChannelBufferFactory bufferFactory =
- channel.getConfig().getBufferFactory();
- final ChannelBuffer buffer = bufferFactory.getBuffer(readBytes);
- buffer.setBytes(0, bb);
- buffer.writerIndex(readBytes);
- //if(buffer instanceof BigEndianHeapChannelBuffer){
- // logger2.info("buffer instanceof BigEndianHeapChannelBuffer.");
- //}
- recvBufferPool.release(bb);
- // Update the predi||\\|||||
- predictor.previousReceiveBufferSize(readBytes);
- // Fire the event.
- fireMessageReceived(channel, buffer);
- } else {
- recvBufferPool.release(bb);
- }
- if (ret < 0 || failure) {
- k.cancel(); // Some JDK implementations run into an infinite loop without this.
- close(channel, succeededFuture(channel));
- return false;
- }
- return true;
- }
在代码里发现了 netty 会再创造一个 chanelbuffer 对象, 然后将 directbuffer 里的内容复制到 chanelbuffer 里面, 而这个 chanelbuffer 对象实际上是一个堆内内存, 然后 netty 会真对这块内存进行解码及返回给上层调用服务等, 也就是说没有直接将 directbuffer 返回给 dubbo 服务, 这样也就解释了, 我们在提供 dubbo 服务的 jvm 里, 禁止掉了 system.gc 的情况下, 没有发生过堆外内存泄漏的原因. 后面我会找时间详细的分析一下 netty4 和 kafka 使用 directbuffer 的情况.
来源: https://www.cnblogs.com/mantu/p/10409637.html