1# Native Image Development (C/C++)
2
3## When to Use
4
5The native image module is used for associating a surface with an OpenGL external texture. It functions as the consumer of a graphics queue. You can use the APIs of this module to obtain and use a buffer, and output the buffer content to an OpenGL external texture.
6
7The following scenario is common for native image development:
8
9Use the native image APIs to create an **OH_NativeImage** instance as the consumer and obtain the corresponding **OHNativeWindow** instance (functioning as the producer). Use the native window APIs to fill in and flush the buffer, and then use the native image APIs to update the buffer content to an OpenGL ES texture.
10
11The native image module must be used together with the native window, native buffer, EGL, and GLES3 modules.
12
13## Available APIs
14
15| API                                                      | Description                                                        |
16| ------------------------------------------------------------ | ------------------------------------------------------------ |
17| OH_NativeImage_Create (uint32_t textureId, uint32_t textureTarget) | Creates an **OH_NativeImage** instance to be associated with the specified OpenGL ES texture ID and target. This function must be used in pair with **OH_NativeImage_Destroy**. Otherwise, memory leak occurs.|
18| OH_NativeImage_AcquireNativeWindow (OH_NativeImage \*image)  | Obtains an **OHNativeWindow** instance associated with an **OH_NativeImage** instance. It is unnecessary to manually release this **OHNativeWindow** with **OH_NativeWindow_DestroyNativeWindow** as it will be automatically freed upon calling **OH_NativeImage_Destroy**. Failing to do so could result in memory access violations after the memory has been freed, which might cause the application to crash.|
19| OH_NativeImage_AttachContext (OH_NativeImage \*image, uint32_t textureId) | Attaches an **OH_NativeImage** instance to the current OpenGL ES context. The OpenGL ES texture will be bound to an **GL_TEXTURE_EXTERNAL_OES** instance and updated through the **OH_NativeImage** instance.|
20| OH_NativeImage_DetachContext (OH_NativeImage \*image)        | Detaches an **OH_NativeImage** instance from the current OpenGL ES context.             |
21| OH_NativeImage_UpdateSurfaceImage (OH_NativeImage \*image)   | Updates the OpenGL ES texture associated with the latest frame through an **OH_NativeImage** instance.     |
22| OH_NativeImage_GetTimestamp (OH_NativeImage \*image)         | Obtains the timestamp of the texture image that recently called the **OH_NativeImage_UpdateSurfaceImage** function.|
23| OH_NativeImage_GetTransformMatrix (OH_NativeImage \*image, float matrix[16]) | Obtains the transformation matrix of the texture image that recently called the **OH_NativeImage_UpdateSurfaceImage** function.|
24| OH_NativeImage_Destroy (OH_NativeImage \*\*image)            | Destroys an **OH_NativeImage** instance created by calling **OH_NativeImage_Create**. After the instance is destroyed, the pointer to it is assigned **NULL**.|
25
26For details about the APIs, see [native_image](../reference/apis-arkgraphics2d/_o_h___native_image.md).
27
28## How to Develop
29
30The following steps describe how to use the native Image APIs to create an **OH_NativeImage** instance as the consumer and update the data to a OpenGL external texture.
31
32**Adding Dynamic Link Libraries**
33
34Add the following libraries to **CMakeLists.txt**:
35
36```txt
37libEGL.so
38libGLESv3.so
39libnative_image.so
40libnative_window.so
41libnative_buffer.so
42```
43
44**Including Header Files**
45
46```c++
47#include <iostream>
48#include <string>
49#include <EGL/egl.h>
50#include <EGL/eglext.h>
51#include <GLES3/gl3.h>
52#include <GLES2/gl2ext.h>
53#include <sys/mman.h>
54#include <native_image/native_image.h>
55#include <native_window/external_window.h>
56#include <native_buffer/native_buffer.h>
57```
58
591. Initialize the EGL environment.
60
61   Refer to the code snippet below. For details about how to use the **XComponent**, see [XComponent Development](../ui/napi-xcomponent-guidelines.md).
62   ```c++
63   using GetPlatformDisplayExt = PFNEGLGETPLATFORMDISPLAYEXTPROC;
64   constexpr const char *EGL_EXT_PLATFORM_WAYLAND = "EGL_EXT_platform_wayland";
65   constexpr const char *EGL_KHR_PLATFORM_WAYLAND = "EGL_KHR_platform_wayland";
66   constexpr int32_t EGL_CONTEXT_CLIENT_VERSION_NUM = 2;
67   constexpr char CHARACTER_WHITESPACE = ' ';
68   constexpr const char *CHARACTER_STRING_WHITESPACE = " ";
69   constexpr const char *EGL_GET_PLATFORM_DISPLAY_EXT = "eglGetPlatformDisplayEXT";
70   EGLContext eglContext_ = EGL_NO_CONTEXT;
71   EGLDisplay eglDisplay_ = EGL_NO_DISPLAY;
72   static inline EGLConfig config_;
73   static inline EGLSurface eglSurface_;
74   // OHNativeWindow obtained from the XComponent.
75   OHNativeWindow *eglNativeWindow_;
76
77   // Check the EGL extension.
78   static bool CheckEglExtension(const char *extensions, const char *extension) {
79       size_t extlen = strlen(extension);
80       const char *end = extensions + strlen(extensions);
81
82       while (extensions < end) {
83           size_t n = 0;
84           if (*extensions == CHARACTER_WHITESPACE) {
85               extensions++;
86               continue;
87           }
88           n = strcspn(extensions, CHARACTER_STRING_WHITESPACE);
89           if (n == extlen && strncmp(extension, extensions, n) == 0) {
90               return true;
91           }
92           extensions += n;
93       }
94       return false;
95   }
96
97   // Obtain the display.
98   static EGLDisplay GetPlatformEglDisplay(EGLenum platform, void *native_display, const EGLint *attrib_list) {
99       static GetPlatformDisplayExt eglGetPlatformDisplayExt = NULL;
100
101       if (!eglGetPlatformDisplayExt) {
102           const char *extensions = eglQueryString(EGL_NO_DISPLAY, EGL_EXTENSIONS);
103           if (extensions && (CheckEglExtension(extensions, EGL_EXT_PLATFORM_WAYLAND) ||
104                              CheckEglExtension(extensions, EGL_KHR_PLATFORM_WAYLAND))) {
105               eglGetPlatformDisplayExt = (GetPlatformDisplayExt)eglGetProcAddress(EGL_GET_PLATFORM_DISPLAY_EXT);
106           }
107       }
108
109       if (eglGetPlatformDisplayExt) {
110           return eglGetPlatformDisplayExt(platform, native_display, attrib_list);
111       }
112
113       return eglGetDisplay((EGLNativeDisplayType)native_display);
114   }
115
116   static void InitEGLEnv() {
117       // Obtain the display.
118       eglDisplay_ = GetPlatformEglDisplay(EGL_PLATFORM_OHOS_KHR, EGL_DEFAULT_DISPLAY, NULL);
119       if (eglDisplay_ == EGL_NO_DISPLAY) {
120           std::cout << "Failed to create EGLDisplay gl errno : " << eglGetError() << std::endl;
121       }
122
123       EGLint major, minor;
124       // Initialize the EGL display.
125       if (eglInitialize(eglDisplay_, &major, &minor) == EGL_FALSE) {
126           std::cout << "Failed to initialize EGLDisplay" << std::endl;
127       }
128
129       // Bind the OpenGL ES API.
130       if (eglBindAPI(EGL_OPENGL_ES_API) == EGL_FALSE) {
131           std::cout << "Failed to bind OpenGL ES API" << std::endl;
132       }
133
134       unsigned int glRet;
135       EGLint count;
136       EGLint config_attribs[] = {EGL_SURFACE_TYPE,
137                                  EGL_WINDOW_BIT,
138                                  EGL_RED_SIZE,
139                                  8,
140                                  EGL_GREEN_SIZE,
141                                  8,
142                                  EGL_BLUE_SIZE,
143                                  8,
144                                  EGL_ALPHA_SIZE,
145                                  8,
146                                  EGL_RENDERABLE_TYPE,
147                                  EGL_OPENGL_ES3_BIT,
148                                  EGL_NONE};
149
150       // Obtain a valid system configuration.
151       glRet = eglChooseConfig(eglDisplay_, config_attribs, &config_, 1, &count);
152       if (!(glRet && static_cast<unsigned int>(count) >= 1)) {
153           std::cout << "Failed to eglChooseConfig" << std::endl;
154       }
155
156       static const EGLint context_attribs[] = {EGL_CONTEXT_CLIENT_VERSION, EGL_CONTEXT_CLIENT_VERSION_NUM, EGL_NONE};
157
158       // Create a context.
159       eglContext_ = eglCreateContext(eglDisplay_, config_, EGL_NO_CONTEXT, context_attribs);
160       if (eglContext_ == EGL_NO_CONTEXT) {
161           std::cout << "Failed to create egl context, error:" << eglGetError() << std::endl;
162       }
163
164       // Create an eglSurface.
165       eglSurface_ = eglCreateWindowSurface(eglDisplay_, config_, reinterpret_cast<EGLNativeWindowType>(eglNativeWindow_), context_attribs);
166       if (eglSurface_ == EGL_NO_SURFACE) {
167           std::cout << "Failed to create egl surface, error:" << eglGetError() << std::endl;
168       }
169
170       // Associate the context.
171       eglMakeCurrent(eglDisplay_, eglSurface_, eglSurface_, eglContext_);
172
173       // The EGL environment initialization is complete.
174       std::cout << "Create EGL context successfully, version" << major << "." << minor << std::endl;
175   }
176   ```
177
1782. Create an **OH_NativeImage** instance.
179
180   ```c++
181   // Create an OpenGL ES texture.
182   GLuint textureId;
183   glGenTextures(1, &textureId);
184   // Create an OH_NativeImage instance, which will be associated with an OpenGL ES texture.
185   OH_NativeImage* image = OH_NativeImage_Create(textureId, GL_TEXTURE_EXTERNAL_OES);
186   ```
187
1883. Obtain the **OHNativeWindow** instance that functions as the producer.
189
190   ```c++
191   // Obtain an OHNativeWindow instance.
192   OHNativeWindow* nativeWindow = OH_NativeImage_AcquireNativeWindow(image);
193   ```
194
1954. Set the width and height of the **OHNativeWindow** instance.
196
197   ```c++
198   int code = SET_BUFFER_GEOMETRY;
199   int32_t width = 800;
200   int32_t height = 600;
201   int32_t ret = OH_NativeWindow_NativeWindowHandleOpt(nativeWindow, code, width, height);
202   ```
203
2045. Write the produced content to the **OHNativeWindowBuffer**.
205
206   1. Obtain an **OHNativeWindowBuffer** instance from the **NativeWindow** instance.
207
208      ```c++
209      OHNativeWindowBuffer *buffer = nullptr;
210      int fenceFd;
211      // Obtain an OHNativeWindowBuffer instance by calling OH_NativeWindow_NativeWindowRequestBuffer.
212      OH_NativeWindow_NativeWindowRequestBuffer(nativeWindow, &buffer, &fenceFd);
213
214      BufferHandle *handle = OH_NativeWindow_GetBufferHandleFromNative(buffer);
215      ```
216
217   2. Write the produced content to the **OHNativeWindowBuffer**.
218
219      ```c++
220      // Use mmap() to obtain the memory virtual address of buffer handle.
221      void *mappedAddr = mmap(handle->virAddr, handle->size, PROT_READ | PROT_WRITE, MAP_SHARED, handle->fd, 0);
222      if (mappedAddr == MAP_FAILED) {
223          // mmap failed
224      }
225      static uint32_t value = 0x00;
226      value++;
227      uint32_t *pixel = static_cast<uint32_t *>(mappedAddr);
228      for (uint32_t x = 0; x < width; x++) {
229          for (uint32_t y = 0; y < height; y++) {
230              *pixel++ = value;
231          }
232      }
233      // Unmap the memory when the memory is no longer required.
234      int result = munmap(mappedAddr, handle->size);
235      if (result == -1) {
236          // munmap failed
237      }
238      ```
239
240   3. Flush the **OHNativeWindowBuffer** to the **NativeWindow**.
241
242      ```c++
243      // Set the refresh region. If the Rect array in Region is a null pointer or rectNumber is 0, all contents in the OHNativeWindowBuffer are changed.
244      Region region{nullptr, 0};
245      // Flush the buffer to the consumer through OH_NativeWindow_NativeWindowFlushBuffer, for example, by displaying it on the screen.
246      OH_NativeWindow_NativeWindowFlushBuffer(nativeWindow, buffer, fenceFd, region);
247      ```
248
249   4. Destroy the **OHNativeWindow** instance when it is no longer needed.
250
251      ```c++
252      OH_NativeWindow_DestroyNativeWindow(nativeWindow);
253      ```
254
2556. Update the content to the OpenGL texture.
256
257   ```c++
258   // Update the content to the OpenGL texture.
259   ret = OH_NativeImage_UpdateSurfaceImage(image);
260   if (ret != 0) {
261       std::cout << "OH_NativeImage_UpdateSurfaceImage failed" << std::endl;
262   }
263   // Obtain the timestamp and transformation matrix of the texture image that recently called the **OH_NativeImage_UpdateSurfaceImage** function.
264   int64_t timeStamp = OH_NativeImage_GetTimestamp(image);
265   float matrix[16];
266   ret = OH_NativeImage_GetTransformMatrix(image, matrix);
267   if (ret != 0) {
268       std::cout << "OH_NativeImage_GetTransformMatrix failed" << std::endl;
269   }
270
271   // Perform OpenGL post-processing on the texture, and then display the texture on the screen.
272   EGLBoolean eglRet = eglSwapBuffers(eglDisplay_, eglSurface_);
273   if (eglRet == EGL_FALSE) {
274       std::cout << "eglSwapBuffers failed" << std::endl;
275   }
276   ```
277
2787. Detach the **OH_NativeImage** from the current OpenGL texture and attach it to a new external texture.
279
280   ```c++
281   // Detach an OH_NativeImage instance from the current OpenGL ES context.
282   ret = OH_NativeImage_DetachContext(image);
283   if (ret != 0) {
284       std::cout << "OH_NativeImage_DetachContext failed" << std::endl;
285   }
286   // Attach an OH_NativeImage instance to the current OpenGL ES context. The OpenGL ES texture will be bound to an GL_TEXTURE_EXTERNAL_OES instance and updated through the OH_NativeImage instance.
287   GLuint textureId2;
288   glGenTextures(1, &textureId2);
289   ret = OH_NativeImage_AttachContext(image, textureId2);
290   ```
291
2928. Destroy the **OH_NativeImage** instance when it is no longer needed.
293
294   ```c++
295   // Destroy the OH_NativeImage instance.
296   OH_NativeImage_Destroy(&image);
297   ```
298
299