Android Opengl共享context的替代方案(创建多个eglsurface,通过makecurrent进行切换)

显示效果

背景

在应用的开发过程中,我们避免不了要使用Opengl共享context的技术。比如我们需要在预览的同时还要将预览的画面传递给MediaCodec进行编码。有的朋友会说Camera2可以设置多个surface,这样就可以把预览的surface和MediaCodec的surface同时传递给Camera2。这种只是实现比较简单的录制功能。现在多数有录制功能的应用都支持滤镜功能,可以把你拍得更漂亮。由于加入了滤镜的功能,所以我们要通过Opengl技术来实现。

当我们加入opengl的滤镜功能后就有一个需要解决的问题,如何让预览和MediaCo dec同时应用滤镜。这时共享 context技术就派上用场了,通过共享context使创建在不同线程的texture id可以相互共享。通常共享的texture id绑定的是一个frame buffer,滤镜效果都绘制在这个frame buffer上,然后将frame buffer分别绘制在预览和MediaCodec。这里不对共享context的实现进行详细的介绍,这里重点关注的是有没有其他的方案来实现这个功能。

关于Camera2的思考(SurfaceTextureRenderer)

大家都知道Camera2是支持设置多个surface的,那么他是如何把camera数据绘制到这些surface上的呢?实现方式就在SurfaceTextureRenderer中,通过查看这个文件我们了解到他构造了一个opengl环境用于绘制camera数据。在构建opengl环境的过程中创建了EGLDisplay、EGLContext、EGLSurface。这里关于EGLSurface的创建有些特殊,我们可以看到代码中创建了多个EGLSurface。每个EGLSurface都与一个surface进行绑定。

    private void configureEGLOutputSurfaces(Collection surfaces) {
        if (surfaces == null || surfaces.size() == 0) {
            throw new IllegalStateException("No Surfaces were provided to draw to");
        }
        int[] surfaceAttribs = {
                EGL14.EGL_NONE
        };
        for (EGLSurfaceHolder holder : surfaces) {
            holder.eglSurface = EGL14.eglCreateWindowSurface(mEGLDisplay, mConfigs,
                    holder.surface, surfaceAttribs, /*offset*/ 0);
            checkEglError("eglCreateWindowSurface");
        }
    }

下面的代码是绘制camera数据到各个EGLSurface上,具体绘制到哪个EGLSurface上是通过 makeCurrent方法控制的。看到这里大家大概就明白了,通过EGL14.eglMakeCurrent方法就可以实现绘制内容到不同的surface上。当然在切换surface的时候要留意绘制的viewport大小也会发送变化,所以mvp matrix也要进行相应的更新。

public void drawIntoSurfaces(CaptureCollector targetCollector) {
       ...
       
        for (EGLSurfaceHolder holder : mSurfaces) {
            if (LegacyCameraDevice.containsSurfaceId(holder.surface, targetSurfaceIds)) {
                try{
                    LegacyCameraDevice.setSurfaceDimens(holder.surface, holder.width,
                            holder.height);
                    makeCurrent(holder.eglSurface);

                    LegacyCameraDevice.setNextTimestamp(holder.surface, captureHolder.second);
                    drawFrame(mSurfaceTexture, holder.width, holder.height,
                            (mFacing == CameraCharacteristics.LENS_FACING_FRONT) ?
                                    FLIP_TYPE_HORIZONTAL : FLIP_TYPE_NONE);
                    swapBuffers(holder.eglSurface);
                } catch (LegacyExceptionUtils.BufferQueueAbandonedException e) {
                    Log.w(TAG, "Surface abandoned, dropping frame. ", e);
                    request.setOutputAbandoned();
                }
            }
        }
        
        ...
    }
 }   
      private void makeCurrent(EGLSurface surface)
            throws LegacyExceptionUtils.BufferQueueAbandonedException {
        EGL14.eglMakeCurrent(mEGLDisplay, surface, surface, mEGLContext);
        checkEglDrawError("makeCurrent");
    }

通过添加两个SurfaceView验证方案

这里我们创建两个surfaceview用于验证构造多个eglsurface的方案。

 surfaceView1.holder.addCallback(object:SurfaceHolder.Callback{
            override fun surfaceCreated(holder: SurfaceHolder?) {

            }

            override fun surfaceChanged(holder: SurfaceHolder?, format: Int, width: Int, height: Int) {
                if (eglSurfaceHolder1 == null) {
                    eglSurfaceHolder1 = EGLCore.EGLSurfaceHolder(holder!!.surface, width.toFloat(), height.toFloat())
                    renderScope.addSurfaceHolder(eglSurfaceHolder1)
                }
            }

            override fun surfaceDestroyed(holder: SurfaceHolder?) {
                renderScope.removeSurfaceHolder(eglSurfaceHolder1)
                eglSurfaceHolder1 = null
            }
        })

        surfaceView2.holder.addCallback(object:SurfaceHolder.Callback{
            override fun surfaceCreated(holder: SurfaceHolder?) {
            }

            override fun surfaceChanged(holder: SurfaceHolder?, format: Int, width: Int, height: Int) {
                if (eglSurfaceHolder2 == null) {
                    eglSurfaceHolder2 = EGLCore.EGLSurfaceHolder(holder!!.surface, width.toFloat(), height.toFloat())
                    renderScope.addSurfaceHolder(eglSurfaceHolder2)
                }
            }

            override fun surfaceDestroyed(holder: SurfaceHolder?) {
                renderScope.removeSurfaceHolder(eglSurfaceHolder2)
                eglSurfaceHolder2 = null
            }
        })

这里把两个surface都添加到RenderScope中,RenderScope创建opengl线程环境,opengl描画都是发生在这个线程的。

class RenderScope(private val render: Render) : BackgroundScope by HandlerThreadScope() {
    private var eglCore: EGLCore? = null

    init {
        runInBackground {
            eglCore = EGLCore()
            render.onCreate()
        }
    }

    fun addSurfaceHolder(eglSurfaceHolder: EGLCore.EGLSurfaceHolder?) = runInBackground {
        eglCore?.addSurfaceHolder(eglSurfaceHolder)
    }

    fun removeSurfaceHolder(eglSurfaceHolder: EGLCore.EGLSurfaceHolder?) = runInBackground {
        eglCore?.removeSurfaceHolder(eglSurfaceHolder)
    }

    fun removeSurfaceHolder(surface: Surface?) = runInBackground {
        eglCore?.removeSurfaceHolder(surface)
    }

    fun release() = runInBackground {
        render.onDestroy()
        eglCore?.releaseEGLContext()
        eglCore = null
        quit()
    }

    fun requestRender() = runInBackground {
        eglCore?.render { render.onDrawFrame(it) }
    }

    interface Render {
        fun onCreate()
        fun onDestroy()
        fun onDrawFrame(eglSurfaceHolder: EGLCore.EGLSurfaceHolder)
    }
}

EGLCore这个类是用来构造opengl环境的,构造的过程这里不细说了。我们重点关注下addSurfaceHolder这个方法。它是用于添加surface的,通过这个方法添加surface后,渲染过程中就可以把内容绘制到这个surface上。render方法用于渲染使用,看他的实现主要就是通过makecurrent来切换eglsurface,然后调用block 进行描画。block调用的次数与添加的surface数量相同。


class EGLCore {
    private var mEGLDisplay = EGL14.EGL_NO_DISPLAY
    private var mEGLContext = EGL14.EGL_NO_CONTEXT
    private var mConfigs: EGLConfig? = null
    private val mEGLSurfaces = SparseArray()

    init {
        log("initEGLContext")
        releaseEGLContext()
        mEGLDisplay = EGL14.eglGetDisplay(EGL14.EGL_DEFAULT_DISPLAY)
        check(!(mEGLDisplay === EGL14.EGL_NO_DISPLAY)) { "No EGL14 display" }
        val version = IntArray(2)
        check(EGL14.eglInitialize(mEGLDisplay, version,  /*offset*/0, version,  /*offset*/1)) { "Cannot initialize EGL14" }
        val attributeList = intArrayOf(
            EGL14.EGL_RED_SIZE, EGL_COLOR_BIT_LENGTH,
            EGL14.EGL_GREEN_SIZE, EGL_COLOR_BIT_LENGTH,
            EGL14.EGL_BLUE_SIZE, EGL_COLOR_BIT_LENGTH,
            EGL14.EGL_RENDERABLE_TYPE, EGL14.EGL_OPENGL_ES2_BIT,
            EGL_RECORDABLE_ANDROID, 1,
            EGL14.EGL_SURFACE_TYPE, EGL14.EGL_PBUFFER_BIT or EGL14.EGL_WINDOW_BIT,
            EGL14.EGL_NONE
        )
        val configs = arrayOfNulls(1)
        val numConfigs = IntArray(1)
        EGL14.eglChooseConfig(
            mEGLDisplay,
            attributeList,  /*offset*/0,
            configs,  /*offset*/0, configs.size,
            numConfigs,  /*offset*/0
        )
        checkEglError("eglCreateContext RGB888+recordable ES2")
        mConfigs = configs[0]
        val contextAttributeList = intArrayOf(
            EGL14.EGL_CONTEXT_CLIENT_VERSION, GLES_VERSION,
            EGL14.EGL_NONE
        )
        mEGLContext = EGL14.eglCreateContext(
            mEGLDisplay,
            configs[0],
            EGL14.EGL_NO_CONTEXT,
            contextAttributeList,  /*offset*/0
        )
        checkEglError("eglCreateContext")
        check(!(mEGLContext === EGL14.EGL_NO_CONTEXT)) { "No EGLContext could be made" }
        EGL14.eglMakeCurrent(mEGLDisplay, EGL14.EGL_NO_SURFACE, EGL14.EGL_NO_SURFACE, mEGLContext)
    }

    fun addSurfaceHolder(eglSurfaceHolder: EGLSurfaceHolder?) {
        if (mEGLDisplay == EGL14.EGL_NO_DISPLAY) {
            log("addSurfaceHolder mEGLDisplay == EGL14.EGL_NO_DISPLAY")
            return
        }
        eglSurfaceHolder ?: return
        check(eglSurfaceHolder.surface.isValid) { "addSurface surface is not valid!!" }
        val surfaceAttribute = intArrayOf(
            EGL14.EGL_NONE
        )
        eglSurfaceHolder.eglSurface = EGL14.eglCreateWindowSurface(
            mEGLDisplay,
            mConfigs,
            eglSurfaceHolder.surface,
            surfaceAttribute,  /*offset*/0
        )
        checkEglError("eglCreateWindowSurface")
        mEGLSurfaces.put(eglSurfaceHolder.surface.hashCode(), eglSurfaceHolder)
    }

    fun removeSurfaceHolder(eglSurfaceHolder: EGLSurfaceHolder?) {
        if (mEGLDisplay == EGL14.EGL_NO_DISPLAY) {
            log("removeSurfaceHolder mEGLDisplay == EGL14.EGL_NO_DISPLAY")
            return
        }
        eglSurfaceHolder ?: return
        mEGLSurfaces[eglSurfaceHolder.surface.hashCode()]?.let {
            EGL14.eglDestroySurface(mEGLDisplay, it.eglSurface)
            it.eglSurface = null
            mEGLSurfaces.remove(it.surface.hashCode())
        }
    }
    
    fun removeSurfaceHolder(surface: Surface?) {
        if (mEGLDisplay == EGL14.EGL_NO_DISPLAY) {
            log("removeSurfaceHolder mEGLDisplay == EGL14.EGL_NO_DISPLAY")
            return
        }
        surface ?: return
        mEGLSurfaces[surface.hashCode()]?.let {
            EGL14.eglDestroySurface(mEGLDisplay, it.eglSurface)
            it.eglSurface = null
            mEGLSurfaces.remove(surface.hashCode())
        }
    }

    fun render(block: (EGLSurfaceHolder) -> Unit) {
        if (mEGLDisplay == EGL14.EGL_NO_DISPLAY) {
            log("render mEGLDisplay == EGL14.EGL_NO_DISPLAY")
            return
        }
        mEGLSurfaces.forEach { _, holder ->
            if (!holder.available) {
                return@forEach
            }
            EGL14.eglMakeCurrent(mEGLDisplay, holder.eglSurface, holder.eglSurface, mEGLContext)
            checkEglError("makeCurrent")
            block.invoke(holder)
            val result = EGL14.eglSwapBuffers(mEGLDisplay, holder.eglSurface)
            when (val error = EGL14.eglGetError()) {
                EGL14.EGL_SUCCESS -> result
                EGL14.EGL_BAD_NATIVE_WINDOW, EGL14.EGL_BAD_SURFACE -> throw IllegalStateException(
                    "swapBuffers: EGL error: 0x" + Integer.toHexString(error)
                )
                else -> throw IllegalStateException(
                    "swapBuffers: EGL error: 0x" + Integer.toHexString(error)
                )
            }
        }
    }

    fun releaseEGLContext() {
        if (mEGLDisplay !== EGL14.EGL_NO_DISPLAY) {
            log("releaseEGLContext")
            EGL14.eglMakeCurrent(mEGLDisplay, EGL14.EGL_NO_SURFACE, EGL14.EGL_NO_SURFACE, EGL14.EGL_NO_CONTEXT)
            mEGLSurfaces.forEach { _, holder ->
                if (holder.eglSurface != null) {
                    EGL14.eglDestroySurface(mEGLDisplay, holder.eglSurface)
                    holder.eglSurface = null
                }
            }
            mEGLSurfaces.clear()
            EGL14.eglDestroyContext(mEGLDisplay, mEGLContext)
            EGL14.eglReleaseThread()
            EGL14.eglTerminate(mEGLDisplay)
        }
        mConfigs = null
        mEGLDisplay = EGL14.EGL_NO_DISPLAY
        mEGLContext = EGL14.EGL_NO_CONTEXT
    }

    private fun checkEglError(msg: String) {
        val error = EGL14.eglGetError()
        check(error == EGL14.EGL_SUCCESS) { msg + ": EGL error: 0x" + Integer.toHexString(error) }
    }

    private fun log(msg: String) {
        Log.d("EGLCore", msg)
    }

    class EGLSurfaceHolder(
        val surface: Surface,
        val width: Float,
        val height: Float
    ) {
        var eglSurface: EGLSurface? = null
        var available = true
        val mvpMatrix = MVPMatrix().updateViewport(width, height)
        val viewPortRectF = RectF(0f, 0f, width, height)
    }

    companion object {
        private const val GLES_VERSION = 2
        private const val EGL_COLOR_BIT_LENGTH = 8
        const val EGL_RECORDABLE_ANDROID = 0x3142 // from EGL/eglext.h
    }
}

Git

https://github.com/mjlong1231...

你可能感兴趣的