1# Using ImageEffect to Edit Images
2
3## When to Use
4
5The **ImageEffect** class provides a series of APIs for editing an image. You can use the APIs to implement various filter effects on images of different input types (pixel map, native window, native buffer, or URI).
6With the native APIs provided by **ImageEffect**, you can:
7
8- Add a filter or filter chain, set an input image, and make the filter effective on the image.
9- Customize a filter.
10- Use a system defined filter.
11
12## **Available APIs**
13
14For details about the APIs, see [ImageEffect](../../reference/apis-image-kit/_image_effect.md).
15
16## How to Develop
17
18**Adding Dynamic Link Libraries**
19
20Add the following libraries to **CMakeLists.txt**.
21
22```txt
23target_link_libraries(entry PUBLIC
24    libace_ndk.z.so
25    libimage_effect.so
26    libpixelmap.so
27    libnative_window.so
28    libnative_buffer.so
29)
30```
31
32Add the following dynamic link libraries based on the image type: **libpixelmap.so** for the pixel map type, **libnative_window.so** for the native window type, and **libnative_buffer.so** for the native buffer type.
33
34**Adding the Header Files**
35
36```c++
37#include <multimedia/image_effect/image_effect.h>
38#include <multimedia/image_effect/image_effect_filter.h>
39#include <multimedia/image_effect/image_effect_errors.h>
40```
41
42### Making a Filter Effective
43
441. Create an **ImageEffect** instance.
45
46    ```c++
47    // Create an ImageEffect instance, with the alias set to ImageEdit.
48    OH_ImageEffect *imageEffect = OH_ImageEffect_Create("ImageEdit");
49    ```
50
512. Add a filter.
52
53    ```c++
54    // Add a filter to obtain an OH_EffectFilter instance. You can call this API multiple times to add multiple filters to form a filter chain.
55    OH_EffectFilter *filter = OH_ImageEffect_AddFilter(imageEffect, OH_EFFECT_BRIGHTNESS_FILTER);
56    CHECK_AND_RETURN_LOG(filter != nullptr, "OH_ImageEffect_AddFilter fail!");
57
58    // Set a filter parameter. For example, set the intensity to 50.
59    ImageEffect_Any value = { .dataType = ImageEffect_DataType::EFFECT_DATA_TYPE_FLOAT, .dataValue.floatValue = 50.f };
60    ImageEffect_ErrorCode errorCode = OH_EffectFilter_SetValue(filter, OH_EFFECT_FILTER_INTENSITY_KEY, &value);
61    ```
62
633. Set the data to be processed.
64
65    **Scenario 1: Set the OH_PixelmapNative input type.**
66
67    For details about how to use OH_PixelmapNative, see [PixelMap Data Processing (C/C++)](image-pixelmap-operation-native.md).
68
69    ```c++
70    // Set an input pixel map.
71    errorCode = OH_ImageEffect_SetInputPixelmap(imageEffect, inputPixelmap);
72    CHECK_AND_RETURN_LOG(errorCode == ImageEffect_ErrorCode::EFFECT_SUCCESS, "OH_ImageEffect_SetInputPixelmap fail!");
73
74    // (Optional) Set an output pixel map. If this API is not called, the filter effect directly takes effect on the input pixel map.
75    errorCode = OH_ImageEffect_SetOutputPixelmap(imageEffect, outputPixelmap);
76    CHECK_AND_RETURN_LOG(errorCode == ImageEffect_ErrorCode::EFFECT_SUCCESS, "OH_ImageEffect_SetOutputPixelmap fail!");
77    ```
78
79    **Scenario 2: Set the OH_NativeBuffer input type.**
80
81    For details about how to use OH_NativeBuffer, see [Native Buffer Development (C/C++)](../../graphics/native-buffer-guidelines.md).
82
83    ```c++
84    // Set an input native buffer.
85    errorCode = OH_ImageEffect_SetInputNativeBuffer(imageEffect, inputNativeBuffer);
86    CHECK_AND_RETURN_LOG(errorCode == ImageEffect_ErrorCode::EFFECT_SUCCESS, "OH_ImageEffect_SetInputNativeBuffer fail!");
87
88    // (Optional) Set an output native buffer. If this API is not called, the filter effect directly takes effect on the input native buffer.
89    errorCode = OH_ImageEffect_SetOutputNativeBuffer(imageEffect, outputNativeBuffer);
90    CHECK_AND_RETURN_LOG(errorCode == ImageEffect_ErrorCode::EFFECT_SUCCESS, "OH_ImageEffect_SetOutputNativeBuffer fail!");
91    ```
92
93    **Scenario 3: Set the URI input type.**
94
95    ```c++
96    // Set an input URI.
97    errorCode = OH_ImageEffect_SetInputUri(imageEffect, inputUri);
98    CHECK_AND_RETURN_LOG(errorCode == ImageEffect_ErrorCode::EFFECT_SUCCESS, "OH_ImageEffect_SetInputUri fail!");
99
100    // (Optional) Set an output URI. If this API is not called, the filter effect directly takes effect on the input URI.
101    errorCode = OH_ImageEffect_SetOutputUri(imageEffect, outputUri);
102    CHECK_AND_RETURN_LOG(errorCode == ImageEffect_ErrorCode::EFFECT_SUCCESS, "OH_ImageEffect_SetOutputUri fail!");
103    ```
104
105    **Scenario 4: Set the OHNativeWindow input type.**
106
107    The following uses camera preview as an example to describe the scenario. The surface ID provided by the **XComponent** for camera preview streams can be converted into an **OHNativeWindow** instance at the native C++ layer.
108    For details about how to use the **XComponent**, see [XComponent](../../reference/apis-arkui/arkui-ts/ts-basic-components-xcomponent.md).
109    For details about how to use the native window module, see [OHNativeWindow](../../reference/apis-arkgraphics2d/_native_window.md).
110    For details about how to use the camera, see [Camera Preview (C/C++)](../camera/native-camera-preview.md).
111
112    (1) Add an **XComponent** to the .ets file.
113
114     ```ts
115     XComponent({
116         id: 'xcomponentId',
117         type: 'surface',
118         controller: this.mXComponentController,
119         libraryname: 'entry'
120     })
121     .onLoad(() => {
122         // Obtain the surface ID of the <XComponent>.
123         this.mSurfaceId = this.mXComponentController.getXComponentSurfaceId()
124
125         // Obtain the input surface ID.
126         this.mSurfaceId = imageEffect.getSurfaceId(this.mSurfaceId)
127
128         // Call the camera API to start preview and transfer the input surface ID to the camera framework.
129         // ...
130     })
131     .width('100%')
132     .height('100%')
133     ```
134
135    (2) Implement imageEffect.getSurfaceId at the native C++ layer.
136
137     ```c++
138     // Create a NativeWindow instance based on the surface ID. Note that the instance must be released by calling OH_NativeWindow_DestoryNativeWindow when it is no longer needed.
139     uint64_t outputSurfaceId;
140     std::istrstream iss(outputSurfaceIdStr);
141     issue >> outputSurfaceId;
142     OHNativeWindow *outputNativeWindow = nullptr;
143     int32_t res = OH_NativeWindow_CreateNativeWindowFromSurfaceId(outputSurfaceId, &outputNativeWindow);
144     CHECK_AND_RETURN_LOG(res == 0, "OH_NativeWindow_CreateNativeWindowFromSurfaceId fail!");
145
146     // Set an output surface.
147     ImageEffect_ErrorCode errorCode = OH_ImageEffect_SetOutputSurface(imageEffect, outputNativeWindow);
148     CHECK_AND_RETURN_LOG(errorCode == ImageEffect_ErrorCode::EFFECT_SUCCESS, "OH_ImageEffect_SetOutputSurface fail!");
149
150     // Obtain the input surface. Note that the obtained inputNativeWindow instance must be released by calling OH_NativeWindow_DestoryNativeWindow when it is no longer needed.
151     OHNativeWindow *inputNativeWindow = nullptr;
152     errorCode = OH_ImageEffect_GetInputSurface(imageEffect, &inputNativeWindow);
153     CHECK_AND_RETURN_LOG(errorCode == ImageEffect_ErrorCode::EFFECT_SUCCESS, "OH_ImageEffect_GetInputSurface fail!");
154
155     // Obtain the surface ID from the inputNativeWindow instance.
156     uint64_t inputSurfaceId = 0;
157     res = OH_NativeWindow_GetSurfaceId(inputNativeWindow, &inputSurfaceId);
158     CHECK_AND_RETURN_LOG(res == 0, "OH_NativeWindow_GetSurfaceId fail!");
159
160     // Convert the surface ID to a string and return the string.
161     std::string inputSurfaceIdStr = std::to_string(inputSurfaceId);
162     ```
163
1644. Start the image effector.
165
166    ```c++
167    // Start the image effector.
168    errorCode = OH_ImageEffect_Start(imageEffect);
169    CHECK_AND_RETURN_LOG(errorCode == ImageEffect_ErrorCode::EFFECT_SUCCESS, "OH_ImageEffect_Start fail!");
170    ```
171
1725. (Optional) Stop the image effector. This operation is required only in the input surface scenario.
173
174    ```c++
175    // Stop the image effector.
176    errorCode = OH_ImageEffect_Stop(imageEffect);
177    CHECK_AND_RETURN_LOG(errorCode == ImageEffect_ErrorCode::EFFECT_SUCCESS, "OH_ImageEffect_Stop fail!");
178    ```
179
1806. (Optional) Serialize the image effector.
181
182    ```c++
183    char *info = nullptr;
184    errorCode = OH_ImageEffect_Save(imageEffect, &info);
185    CHECK_AND_RETURN_LOG(errorCode == ImageEffect_ErrorCode::EFFECT_SUCCESS, "OH_ImageEffect_Save fail!");
186    ```
187
1887. Destroy the **ImageEffect** instance.
189
190    ```c++
191    // Release the ImageEffect instance.
192    errorCode = OH_ImageEffect_Release(imageEffect);
193    CHECK_AND_RETURN_LOG(errorCode == ImageEffect_ErrorCode::EFFECT_SUCCESS, result, "OH_ImageEffect_Release fail!");
194    ```
195
196### Customizing a Filter
197
198To implement and register a custom filter, perform the following steps:
199
2001. Define **ImageEffect_FilterDelegate**.
201
202    ```c++
203    // Image information struct.
204    struct EffectBufferInfo {
205        void *addr = nullptr;
206        int32_t width = 0;
207        int32_t height = 0;
208        int32_t rowSize = 0;
209        ImageEffect_Format format = ImageEffect_Format::EFFECT_PIXEL_FORMAT_UNKNOWN;
210    };
211
212    // Implement a custom filter.
213    ImageEffect_FilterDelegate filterDelegate = {
214        .setValue = [](OH_EffectFilter *filter, const char *key, const ImageEffect_Any *value) {
215            // Verify the parameters. If the verification is successful, true is returned. Otherwise, false is returned.
216            // ...
217            return true;
218        },
219        .render = [](OH_EffectFilter *filter, OH_EffectBufferInfo *info, OH_EffectFilterDelegate_PushData pushData) {
220            return Render(filter, info, pushData);
221        },
222        .save = [](OH_EffectFilter *filter, char **info) {
223            // Obtain the custom filter parameter. In this example, Brightness is the key of the custom filter.
224            ImageEffect_Any value;
225            ImageEffect_ErrorCode errorCode = OH_EffectFilter_GetValue(filter, "Brightness", &value);
226            CHECK_AND_RETURN_LOG(errorCode == ImageEffect_ErrorCode::EFFECT_SUCCESS, false, "OH_EffectFilter_GetValue fail!");
227
228            // Generate a key-value pair.
229            json values;
230            values["Brightness"] = value.dataValue.floatValue;
231            json root;
232            root["name"] = "CustomBrightness";
233            root["values"] = values;
234
235           // Convert the JSON object into a string.
236            // ...
237
238            // Assign a serialized string address to *info.
239            *info = infoStr;
240            return true;
241        },
242        .restore = [](const char *info) {
243            // Create an OH_EffectFilter instance, in which CustomBrightness is the name of the custom filter.
244            OH_EffectFilter *filter = OH_EffectFilter_Create("CustomBrightness");
245            // Parse the JSON string info to obtain the key and value.
246            // ...
247
248            // Set a filter parameter. value is the parameter parsed based on the JSON string in info.
249            ImageEffect_ErrorCode errorCode = OH_EffectFilter_SetValue(filter, "Brightness", &value);
250
251            // ...
252            return filter;
253        }
254    };
255    ```
256
257    You can implement the **Render** API in two scenarios.
258
259    **Scenario 1: The custom algorithm can directly modify the pixel data in info (for example, the brightness filter).**
260
261    ```c++
262    bool Render(OH_EffectFilter *filter, OH_EffectBufferInfo *info, OH_EffectFilterDelegate_PushData pushData)
263    {
264        // Obtain the image information.
265        EffectBufferInfo inputBufferInfo;
266        OH_EffectBufferInfo_GetAddr(info, &inputBufferInfo.addr);
267        OH_EffectBufferInfo_GetWidth(info, &inputBufferInfo.width);
268        OH_EffectBufferInfo_GetHeight(info, &inputBufferInfo.height);
269        OH_EffectBufferInfo_GetRowSize(info, &inputBufferInfo.rowSize);
270        OH_EffectBufferInfo_GetEffectFormat(info, &inputBufferInfo.format);
271
272        // Invoke the custom filter algorithm.
273        ApplyCustomAlgo(inputBufferInfo);
274
275        // After the editing is complete, call pushData to directly transfer the original image.
276        pushData(filter, info);
277        return true;
278    }
279    ```
280
281    **Scenario 2: The custom algorithm cannot directly modify the pixel data in info (for example, the crop filter).**
282
283    ```c++
284    bool Render(OH_EffectFilter *filter, OH_EffectBufferInfo *info, OH_EffectFilterDelegate_PushData pushData)
285    {
286        // Obtain the image information.
287        EffectBufferInfo inputBufferInfo;
288        OH_EffectBufferInfo_GetAddr(info, &inputBufferInfo.addr);
289        OH_EffectBufferInfo_GetWidth(info, &inputBufferInfo.width);
290        OH_EffectBufferInfo_GetHeight(info, &inputBufferInfo.height);
291        OH_EffectBufferInfo_GetRowSize(info, &inputBufferInfo.rowSize);
292        OH_EffectBufferInfo_GetEffectFormat(info, &inputBufferInfo.format);
293
294        // Create output pixel information.
295        EffectBufferInfo outputBufferInfo = CreateOutputBufferInfo(inputBufferInfo);
296
297        // Invoke the custom filter algorithm.
298        ApplyCustomAlgo(inputBufferInfo, outputBufferInfo);
299
300        // Generate outputOhInfo.
301        OH_EffectBufferInfo *outputOhInfo = OH_EffectBufferInfo_Create();
302        OH_EffectBufferInfo_SetAddr(outputOhInfo, outputBufferInfo.addr);
303        OH_EffectBufferInfo_SetWidth(outputOhInfo, outputBufferInfo.width);
304        OH_EffectBufferInfo_SetHeight(outputOhInfo, outputBufferInfo.height);
305        OH_EffectBufferInfo_SetRowSize(outputOhInfo, outputBufferInfo.rowSize);
306        OH_EffectBufferInfo_SetEffectFormat(outputOhInfo, outputBufferInfo.format);
307
308        // Call pushData to transfer outputOhInfo after editing.
309        pushData(filter, outputOhInfo);
310
311        // Release resources.
312        OH_EffectBufferInfo_Release(outputOhInfo);
313        ReleaseOutputBuffer(outputBufferInfo.addr);
314
315        return true;
316    }
317    ```
318
3192. Generate custom filter information.
320
321    ```c++
322    // Create an OH_EffectFilterInfo instance.
323    OH_EffectFilterInfo *customFilterInfo = OH_EffectFilterInfo_Create();
324    CHECK_AND_RETURN_LOG(customFilterInfo != nullptr, "OH_EffectFilter_GetValue fail!");
325
326    // Set the name of the custom filter.
327    OH_EffectFilterInfo_SetFilterName(customFilterInfo, "CustomBrightness");
328
329    // Set the buffer types supported by the custom filter.
330    ImageEffect_BufferType bufferTypeArray[] = { ImageEffect_BufferType::EFFECT_BUFFER_TYPE_PIXEL };
331    OH_EffectFilterInfo_SetSupportedBufferTypes(customFilterInfo, sizeof(bufferTypeArray) / sizeof(ImageEffect_BufferType), bufferTypeArray);
332
333    // Set the pixel formats supported by the custom filter.
334    ImageEffect_Format formatArray[] = { ImageEffect_Format::EFFECT_PIXEL_FORMAT_RGBA8888 };
335    OH_EffectFilterInfo_SetSupportedFormats(customFilterInfo, sizeof(formatArray) / sizeof(ImageEffect_Format), formatArray);
336    ```
337
3383. Register **ImageEffect_FilterDelegate** with the image effector.
339
340    ```c++
341    // Register the custom filter.
342    ImageEffect_ErrorCode errorCode = OH_EffectFilter_Register(customFilterInfo, &filterDelegate);
343    CHECK_AND_RETURN_LOG(errorCode == ImageEffect_ErrorCode::EFFECT_SUCCESS, "OH_EffectFilter_Register fail!");
344    ```
345
346### Using a System Defined Filter
347
3481. Create a filter.
349
350    ```c++
351    // Create a filter, for example, a contrast filter.
352    OH_EffectFilter *filter = OH_EffectFilter_Create(OH_EFFECT_CONTRAST_FILTER);
353    ```
354
3552. Set a filter parameter.
356
357    ```c++
358    // Set a filter parameter. For example, set the intensity to 50.
359    ImageEffect_Any value = {.dataType = ImageEffect_DataType::EFFECT_DATA_TYPE_FLOAT, .dataValue.floatValue = 50.f};
360    ImageEffect_ErrorCode errorCode = OH_EffectFilter_SetValue(filter, OH_EFFECT_FILTER_INTENSITY_KEY, &value);
361    CHECK_AND_RETURN_LOG(errorCode == ImageEffect_ErrorCode::EFFECT_SUCCESS, "OH_EffectFilter_SetValue fail!");
362    ```
363
3643. Make the filter take effect.
365
366    ```c++
367    // Start rendering with the filter.
368    errorCode = OH_EffectFilter_Render(filter, inputPixelmap, outputPixelmap);
369    ```
370
3714. Destroy the filter instance.
372
373    ```c++
374    // Destroy the filter instance.
375    errorCode = OH_EffectFilter_Release(filter);
376    ```
377
378### Query Capabilities
379
380- Query filter information based on the filter name.
381
382    ```c++
383    // Create an OH_EffectFilterInfo instance.
384    OH_EffectFilterInfo *filterInfo = OH_EffectFilterInfo_Create();
385    CHECK_AND_RETURN_LOG(filterInfo != nullptr, "OH_EffectFilterInfo_Create fail!");
386
387    // Query the filter information based on the filter name.
388    ImageEffect_ErrorCode errorCode = OH_EffectFilter_LookupFilterInfo(OH_EFFECT_BRIGHTNESS_FILTER, filterInfo);
389    CHECK_AND_RETURN_LOG(errorCode == ImageEffect_ErrorCode::EFFECT_SUCCESS, "OH_EffectFilter_LookupFilterInfo fail!");
390
391    // Obtain the filter name from the filter information.
392    char *name = nullptr;
393    OH_EffectFilterInfo_GetFilterName(filterInfo, &name);
394
395    // Obtain the supported buffer types.
396    uint32_t supportedBufferTypesCnt = 0;
397    ImageEffect_BufferType *bufferTypeArray = nullptr;
398    OH_EffectFilterInfo_GetSupportedBufferTypes(filterInfo, &supportedBufferTypesCnt, &bufferTypeArray);
399
400    // Obtain the supported pixel formats.
401    uint32_t supportedFormatsCnt = 0;
402    ImageEffect_Format *formatArray = nullptr;
403    OH_EffectFilterInfo_GetSupportedFormats(filterInfo, supportedFormatsCnt, &formatArray);
404
405    // Destroy the OH_EffectFilterInfo instance.
406    OH_EffectFilterInfo_Release(filterInfo);
407    ```
408
409- Query filters that meet given conditions.
410
411    ```c++
412    // Query all filters. Resources need to be released.
413    ImageEffect_FilterNames *filterNames = OH_EffectFilter_LookupFilters("Default");
414
415    // ...
416
417    // Release virtual memory resources by filter names.
418    OH_EffectFilter_ReleaseFilterNames();
419    ```
420
421