1# NativeImage开发指导 (C/C++)
2
3## 场景介绍
4
5NativeImage是提供**Surface关联OpenGL外部纹理**的模块，表示图形队列的消费者端。开发者可以通过`NativeImage`接口接收和使用`Buffer`，并将`Buffer`关联输出到OpenGL外部纹理。
6针对NativeImage，常见的开发场景如下：
7
8* 通过`NativeImage`提供的Native API接口创建`NativeImage`实例作为消费者端，获取与该实例对应的`NativeWindow`作为生产者端。`NativeWindow`相关接口可用于填充`Buffer`内容并提交，`NativeImage`将`Buffer`内容更新到OpenGL外部纹理上。本模块需要配合NativeWindow、NativeBuffer、EGL、GLES3模块一起使用。
9
10## 接口说明
11
12| 接口名                                                       | 描述                                                         |
13| ------------------------------------------------------------ | ------------------------------------------------------------ |
14| OH_NativeImage_Create (uint32_t textureId, uint32_t textureTarget) | 创建一个OH_NativeImage实例，该实例与OpenGL ES的纹理ID和纹理目标相关联。本接口需要与OH_NativeImage_Destroy接口配合使用，否则会存在内存泄露。 |
15| OH_NativeImage_AcquireNativeWindow (OH_NativeImage \*image)  | 获取与OH_NativeImage相关联的OHNativeWindow指针，该OHNativeWindow在调用OH_NativeImage_Destroy时会将其释放，不需要调用OH_NativeWindow_DestroyNativeWindow释放，否则会出现访问已释放内存错误，可能会导致崩溃。 |
16| OH_NativeImage_AttachContext (OH_NativeImage \*image, uint32_t textureId) | 将OH_NativeImage实例附加到当前OpenGL ES上下文，且该OpenGL ES纹理会绑定到 GL_TEXTURE_EXTERNAL_OES，并通过OH_NativeImage进行更新。 |
17| OH_NativeImage_DetachContext (OH_NativeImage \*image)        | 将OH_NativeImage实例从当前OpenGL ES上下文分离。              |
18| OH_NativeImage_UpdateSurfaceImage (OH_NativeImage \*image)   | 通过OH_NativeImage获取最新帧更新相关联的OpenGL ES纹理。      |
19| OH_NativeImage_GetTimestamp (OH_NativeImage \*image)         | 获取最近调用OH_NativeImage_UpdateSurfaceImage的纹理图像的相关时间戳。 |
20| OH_NativeImage_GetTransformMatrix (OH_NativeImage \*image, float matrix[16]) | 获取最近调用OH_NativeImage_UpdateSurfaceImage的纹理图像的变化矩阵。 |
21| OH_NativeImage_Destroy (OH_NativeImage \*\*image)            | 销毁通过OH_NativeImage_Create创建的OH_NativeImage实例，销毁后该OH_NativeImage指针会被赋值为空。 |
22
23详细的接口说明请参考[native_image](../reference/apis-arkgraphics2d/_o_h___native_image.md)。
24
25## 开发步骤
26
27以下步骤描述了如何使用`NativeImage`提供的Native API接口，创建`OH_NativeImage`实例作为消费者端，将数据内容更新到OpenGL外部纹理上。
28
29**添加动态链接库**
30
31CMakeLists.txt中添加以下lib。
32
33```txt
34libEGL.so
35libGLESv3.so
36libnative_image.so
37libnative_window.so
38libnative_buffer.so
39```
40
41**头文件**
42
43```c++
44#include <iostream>
45#include <string>
46#include <EGL/egl.h>
47#include <EGL/eglext.h>
48#include <GLES3/gl3.h>
49#include <GLES2/gl2ext.h>
50#include <sys/mman.h>
51#include <native_image/native_image.h>
52#include <native_window/external_window.h>
53#include <native_buffer/native_buffer.h>
54```
55
561. **初始化EGL环境**。
57
58   这里提供一份初始化EGL环境的代码示例。XComponent模块的具体使用方法请参考[XComponent开发指导](../ui/napi-xcomponent-guidelines.md)。
59   ```c++
60   using GetPlatformDisplayExt = PFNEGLGETPLATFORMDISPLAYEXTPROC;
61   constexpr const char *EGL_EXT_PLATFORM_WAYLAND = "EGL_EXT_platform_wayland";
62   constexpr const char *EGL_KHR_PLATFORM_WAYLAND = "EGL_KHR_platform_wayland";
63   constexpr int32_t EGL_CONTEXT_CLIENT_VERSION_NUM = 2;
64   constexpr char CHARACTER_WHITESPACE = ' ';
65   constexpr const char *CHARACTER_STRING_WHITESPACE = " ";
66   constexpr const char *EGL_GET_PLATFORM_DISPLAY_EXT = "eglGetPlatformDisplayEXT";
67   EGLContext eglContext_ = EGL_NO_CONTEXT;
68   EGLDisplay eglDisplay_ = EGL_NO_DISPLAY;
69   static inline EGLConfig config_;
70   static inline EGLSurface eglSurface_;
71   // 从XComponent中获取到的OHNativeWindow
72   OHNativeWindow *eglNativeWindow_;
73
74   // 检查egl扩展
75   static bool CheckEglExtension(const char *extensions, const char *extension) {
76       size_t extlen = strlen(extension);
77       const char *end = extensions + strlen(extensions);
78
79       while (extensions < end) {
80           size_t n = 0;
81           if (*extensions == CHARACTER_WHITESPACE) {
82               extensions++;
83               continue;
84           }
85           n = strcspn(extensions, CHARACTER_STRING_WHITESPACE);
86           if (n == extlen && strncmp(extension, extensions, n) == 0) {
87               return true;
88           }
89           extensions += n;
90       }
91       return false;
92   }
93
94   // 获取当前的显示设备
95   static EGLDisplay GetPlatformEglDisplay(EGLenum platform, void *native_display, const EGLint *attrib_list) {
96       static GetPlatformDisplayExt eglGetPlatformDisplayExt = NULL;
97
98       if (!eglGetPlatformDisplayExt) {
99           const char *extensions = eglQueryString(EGL_NO_DISPLAY, EGL_EXTENSIONS);
100           if (extensions && (CheckEglExtension(extensions, EGL_EXT_PLATFORM_WAYLAND) ||
101                              CheckEglExtension(extensions, EGL_KHR_PLATFORM_WAYLAND))) {
102               eglGetPlatformDisplayExt = (GetPlatformDisplayExt)eglGetProcAddress(EGL_GET_PLATFORM_DISPLAY_EXT);
103           }
104       }
105
106       if (eglGetPlatformDisplayExt) {
107           return eglGetPlatformDisplayExt(platform, native_display, attrib_list);
108       }
109
110       return eglGetDisplay((EGLNativeDisplayType)native_display);
111   }
112
113   static void InitEGLEnv() {
114       // 获取当前的显示设备
115       eglDisplay_ = GetPlatformEglDisplay(EGL_PLATFORM_OHOS_KHR, EGL_DEFAULT_DISPLAY, NULL);
116       if (eglDisplay_ == EGL_NO_DISPLAY) {
117           std::cout << "Failed to create EGLDisplay gl errno : " << eglGetError() << std::endl;
118       }
119
120       EGLint major, minor;
121       // 初始化EGLDisplay
122       if (eglInitialize(eglDisplay_, &major, &minor) == EGL_FALSE) {
123           std::cout << "Failed to initialize EGLDisplay" << std::endl;
124       }
125
126       // 绑定图形绘制的API为OpenGLES
127       if (eglBindAPI(EGL_OPENGL_ES_API) == EGL_FALSE) {
128           std::cout << "Failed to bind OpenGL ES API" << std::endl;
129       }
130
131       unsigned int glRet;
132       EGLint count;
133       EGLint config_attribs[] = {EGL_SURFACE_TYPE,
134                                  EGL_WINDOW_BIT,
135                                  EGL_RED_SIZE,
136                                  8,
137                                  EGL_GREEN_SIZE,
138                                  8,
139                                  EGL_BLUE_SIZE,
140                                  8,
141                                  EGL_ALPHA_SIZE,
142                                  8,
143                                  EGL_RENDERABLE_TYPE,
144                                  EGL_OPENGL_ES3_BIT,
145                                  EGL_NONE};
146
147       // 获取一个有效的系统配置信息
148       glRet = eglChooseConfig(eglDisplay_, config_attribs, &config_, 1, &count);
149       if (!(glRet && static_cast<unsigned int>(count) >= 1)) {
150           std::cout << "Failed to eglChooseConfig" << std::endl;
151       }
152
153       static const EGLint context_attribs[] = {EGL_CONTEXT_CLIENT_VERSION, EGL_CONTEXT_CLIENT_VERSION_NUM, EGL_NONE};
154
155       // 创建上下文
156       eglContext_ = eglCreateContext(eglDisplay_, config_, EGL_NO_CONTEXT, context_attribs);
157       if (eglContext_ == EGL_NO_CONTEXT) {
158           std::cout << "Failed to create egl context, error:" << eglGetError() << std::endl;
159       }
160
161       // 创建eglSurface
162       eglSurface_ = eglCreateWindowSurface(eglDisplay_, config_, reinterpret_cast<EGLNativeWindowType>(eglNativeWindow_), context_attribs);
163       if (eglSurface_ == EGL_NO_SURFACE) {
164           std::cout << "Failed to create egl surface, error:" << eglGetError() << std::endl;
165       }
166
167       // 关联上下文
168       eglMakeCurrent(eglDisplay_, eglSurface_, eglSurface_, eglContext_);
169
170       // EGL环境初始化完成
171       std::cout << "Create EGL context successfully, version" << major << "." << minor << std::endl;
172   }
173   ```
174
1752. **创建OH_NativeImage实例**。
176
177   ```c++
178   // 创建 OpenGL 纹理
179   GLuint textureId;
180   glGenTextures(1, &textureId);
181   // 创建 NativeImage 实例，关联 OpenGL 纹理
182   OH_NativeImage* image = OH_NativeImage_Create(textureId, GL_TEXTURE_EXTERNAL_OES);
183   ```
184
1853. **获取对应的数据生产者端NativeWindow**。
186
187   ```c++
188   // 获取生产者NativeWindow
189   OHNativeWindow* nativeWindow = OH_NativeImage_AcquireNativeWindow(image);
190   ```
191
1924. **设置NativeWindow的宽高**。
193
194   ```c++
195   int code = SET_BUFFER_GEOMETRY;
196   int32_t width = 800;
197   int32_t height = 600;
198   int32_t ret = OH_NativeWindow_NativeWindowHandleOpt(nativeWindow, code, width, height);
199   ```
200
2015. **将生产的内容写入OHNativeWindowBuffer**。
202
203   1. 从NativeWindow中获取OHNativeWindowBuffer。
204
205      ```c++
206      OHNativeWindowBuffer *buffer = nullptr;
207      int fenceFd;
208      // 通过 OH_NativeWindow_NativeWindowRequestBuffer 获取 OHNativeWindowBuffer 实例
209      OH_NativeWindow_NativeWindowRequestBuffer(nativeWindow, &buffer, &fenceFd);
210
211      BufferHandle *handle = OH_NativeWindow_GetBufferHandleFromNative(buffer);
212      ```
213
214   2. 将生产的内容写入OHNativeWindowBuffer。
215
216      ```c++
217      // 使用系统mmap接口拿到bufferHandle的内存虚拟地址
218      void *mappedAddr = mmap(handle->virAddr, handle->size, PROT_READ | PROT_WRITE, MAP_SHARED, handle->fd, 0);
219      if (mappedAddr == MAP_FAILED) {
220          // mmap failed
221      }
222      static uint32_t value = 0x00;
223      value++;
224      uint32_t *pixel = static_cast<uint32_t *>(mappedAddr);
225      for (uint32_t x = 0; x < width; x++) {
226          for (uint32_t y = 0; y < height; y++) {
227              *pixel++ = value;
228          }
229      }
230      // 内存使用完记得去掉内存映射
231      int result = munmap(mappedAddr, handle->size);
232      if (result == -1) {
233          // munmap failed
234      }
235      ```
236
237   3. 将OHNativeWindowBuffer提交到NativeWindow。
238
239      ```c++
240      // 设置刷新区域，如果Region中的Rect数组为nullptr,或者rectNumber为0，则认为OHNativeWindowBuffer全部内容有更改。
241      Region region{nullptr, 0};
242      // 通过OH_NativeWindow_NativeWindowFlushBuffer 提交给消费者使用，例如：显示在屏幕上。
243      OH_NativeWindow_NativeWindowFlushBuffer(nativeWindow, buffer, fenceFd, region);
244      ```
245
246   4. 用完需要销毁NativeWindow。
247
248      ```c++
249      OH_NativeWindow_DestroyNativeWindow(nativeWindow);
250      ```
251
2526. **更新内容到OpenGL纹理**。
253
254   ```c++
255   // 更新内容到OpenGL纹理。
256   ret = OH_NativeImage_UpdateSurfaceImage(image);
257   if (ret != 0) {
258       std::cout << "OH_NativeImage_UpdateSurfaceImage failed" << std::endl;
259   }
260   // 获取最近调用OH_NativeImage_UpdateSurfaceImage的纹理图像的时间戳和变化矩阵。
261   int64_t timeStamp = OH_NativeImage_GetTimestamp(image);
262   float matrix[16];
263   ret = OH_NativeImage_GetTransformMatrix(image, matrix);
264   if (ret != 0) {
265       std::cout << "OH_NativeImage_GetTransformMatrix failed" << std::endl;
266   }
267
268   // 对update绑定到对应textureId的纹理做对应的opengl后处理后，将纹理上屏
269   EGLBoolean eglRet = eglSwapBuffers(eglDisplay_, eglSurface_);
270   if (eglRet == EGL_FALSE) {
271       std::cout << "eglSwapBuffers failed" << std::endl;
272   }
273   ```
274
2757. **解绑OpenGL纹理，绑定到新的外部纹理上**。
276
277   ```c++
278   // 将OH_NativeImage实例从当前OpenGL ES上下文分离
279   ret = OH_NativeImage_DetachContext(image);
280   if (ret != 0) {
281       std::cout << "OH_NativeImage_DetachContext failed" << std::endl;
282   }
283   // 将OH_NativeImage实例附加到当前OpenGL ES上下文, 且该OpenGL ES纹理会绑定到 GL_TEXTURE_EXTERNAL_OES, 并通过OH_NativeImage进行更新
284   GLuint textureId2;
285   glGenTextures(1, &textureId2);
286   ret = OH_NativeImage_AttachContext(image, textureId2);
287   ```
288
2898. **OH_NativeImage实例使用完需要销毁掉**。
290
291   ```c++
292   // 销毁OH_NativeImage实例
293   OH_NativeImage_Destroy(&image);
294   ```
295
296## 相关实例
297
298针对NativeImage的开发，有以下相关实例可供参考：
299
300- [Native Window（API11）](https://gitee.com/openharmony/applications_app_samples/tree/master/code/BasicFeature/Native/NdkNativeWindow)