1# Using ImageEffect to Edit Images 2 3## When to Use 4 5The **ImageEffect** class provides a series of APIs for editing an image. You can use the APIs to implement various filter effects on images of different input types (pixel map, native window, native buffer, or URI). 6With the native APIs provided by **ImageEffect**, you can: 7 8- Add a filter or filter chain, set an input image, and make the filter effective on the image. 9- Customize a filter. 10- Use a system defined filter. 11 12## **Available APIs** 13 14For details about the APIs, see [ImageEffect](../../reference/apis-image-kit/_image_effect.md). 15 16## How to Develop 17 18**Adding Dynamic Link Libraries** 19 20Add the following libraries to **CMakeLists.txt**. 21 22```txt 23target_link_libraries(entry PUBLIC 24 libace_ndk.z.so 25 libimage_effect.so 26 libpixelmap.so 27 libnative_window.so 28 libnative_buffer.so 29) 30``` 31 32Add the following dynamic link libraries based on the image type: **libpixelmap.so** for the pixel map type, **libnative_window.so** for the native window type, and **libnative_buffer.so** for the native buffer type. 33 34**Adding the Header Files** 35 36```c++ 37#include <multimedia/image_effect/image_effect.h> 38#include <multimedia/image_effect/image_effect_filter.h> 39#include <multimedia/image_effect/image_effect_errors.h> 40``` 41 42### Making a Filter Effective 43 441. Create an **ImageEffect** instance. 45 46 ```c++ 47 // Create an ImageEffect instance, with the alias set to ImageEdit. 48 OH_ImageEffect *imageEffect = OH_ImageEffect_Create("ImageEdit"); 49 ``` 50 512. Add a filter. 52 53 ```c++ 54 // Add a filter to obtain an OH_EffectFilter instance. You can call this API multiple times to add multiple filters to form a filter chain. 55 OH_EffectFilter *filter = OH_ImageEffect_AddFilter(imageEffect, OH_EFFECT_BRIGHTNESS_FILTER); 56 CHECK_AND_RETURN_LOG(filter != nullptr, "OH_ImageEffect_AddFilter fail!"); 57 58 // Set a filter parameter. For example, set the intensity to 50. 59 ImageEffect_Any value = { .dataType = ImageEffect_DataType::EFFECT_DATA_TYPE_FLOAT, .dataValue.floatValue = 50.f }; 60 ImageEffect_ErrorCode errorCode = OH_EffectFilter_SetValue(filter, OH_EFFECT_FILTER_INTENSITY_KEY, &value); 61 ``` 62 633. Set the data to be processed. 64 65 **Scenario 1: Set the OH_PixelmapNative input type.** 66 67 For details about how to use OH_PixelmapNative, see [PixelMap Data Processing (C/C++)](image-pixelmap-operation-native.md). 68 69 ```c++ 70 // Set an input pixel map. 71 errorCode = OH_ImageEffect_SetInputPixelmap(imageEffect, inputPixelmap); 72 CHECK_AND_RETURN_LOG(errorCode == ImageEffect_ErrorCode::EFFECT_SUCCESS, "OH_ImageEffect_SetInputPixelmap fail!"); 73 74 // (Optional) Set an output pixel map. If this API is not called, the filter effect directly takes effect on the input pixel map. 75 errorCode = OH_ImageEffect_SetOutputPixelmap(imageEffect, outputPixelmap); 76 CHECK_AND_RETURN_LOG(errorCode == ImageEffect_ErrorCode::EFFECT_SUCCESS, "OH_ImageEffect_SetOutputPixelmap fail!"); 77 ``` 78 79 **Scenario 2: Set the OH_NativeBuffer input type.** 80 81 For details about how to use OH_NativeBuffer, see [Native Buffer Development (C/C++)](../../graphics/native-buffer-guidelines.md). 82 83 ```c++ 84 // Set an input native buffer. 85 errorCode = OH_ImageEffect_SetInputNativeBuffer(imageEffect, inputNativeBuffer); 86 CHECK_AND_RETURN_LOG(errorCode == ImageEffect_ErrorCode::EFFECT_SUCCESS, "OH_ImageEffect_SetInputNativeBuffer fail!"); 87 88 // (Optional) Set an output native buffer. If this API is not called, the filter effect directly takes effect on the input native buffer. 89 errorCode = OH_ImageEffect_SetOutputNativeBuffer(imageEffect, outputNativeBuffer); 90 CHECK_AND_RETURN_LOG(errorCode == ImageEffect_ErrorCode::EFFECT_SUCCESS, "OH_ImageEffect_SetOutputNativeBuffer fail!"); 91 ``` 92 93 **Scenario 3: Set the URI input type.** 94 95 ```c++ 96 // Set an input URI. 97 errorCode = OH_ImageEffect_SetInputUri(imageEffect, inputUri); 98 CHECK_AND_RETURN_LOG(errorCode == ImageEffect_ErrorCode::EFFECT_SUCCESS, "OH_ImageEffect_SetInputUri fail!"); 99 100 // (Optional) Set an output URI. If this API is not called, the filter effect directly takes effect on the input URI. 101 errorCode = OH_ImageEffect_SetOutputUri(imageEffect, outputUri); 102 CHECK_AND_RETURN_LOG(errorCode == ImageEffect_ErrorCode::EFFECT_SUCCESS, "OH_ImageEffect_SetOutputUri fail!"); 103 ``` 104 105 **Scenario 4: Set the OHNativeWindow input type.** 106 107 The following uses camera preview as an example to describe the scenario. The surface ID provided by the **XComponent** for camera preview streams can be converted into an **OHNativeWindow** instance at the native C++ layer. 108 For details about how to use the **XComponent**, see [XComponent](../../reference/apis-arkui/arkui-ts/ts-basic-components-xcomponent.md). 109 For details about how to use the native window module, see [OHNativeWindow](../../reference/apis-arkgraphics2d/_native_window.md). 110 For details about how to use the camera, see [Camera Preview (C/C++)](../camera/native-camera-preview.md). 111 112 (1) Add an **XComponent** to the .ets file. 113 114 ```ts 115 XComponent({ 116 id: 'xcomponentId', 117 type: 'surface', 118 controller: this.mXComponentController, 119 libraryname: 'entry' 120 }) 121 .onLoad(() => { 122 // Obtain the surface ID of the <XComponent>. 123 this.mSurfaceId = this.mXComponentController.getXComponentSurfaceId() 124 125 // Obtain the input surface ID. 126 this.mSurfaceId = imageEffect.getSurfaceId(this.mSurfaceId) 127 128 // Call the camera API to start preview and transfer the input surface ID to the camera framework. 129 // ... 130 }) 131 .width('100%') 132 .height('100%') 133 ``` 134 135 (2) Implement imageEffect.getSurfaceId at the native C++ layer. 136 137 ```c++ 138 // Create a NativeWindow instance based on the surface ID. Note that the instance must be released by calling OH_NativeWindow_DestoryNativeWindow when it is no longer needed. 139 uint64_t outputSurfaceId; 140 std::istrstream iss(outputSurfaceIdStr); 141 issue >> outputSurfaceId; 142 OHNativeWindow *outputNativeWindow = nullptr; 143 int32_t res = OH_NativeWindow_CreateNativeWindowFromSurfaceId(outputSurfaceId, &outputNativeWindow); 144 CHECK_AND_RETURN_LOG(res == 0, "OH_NativeWindow_CreateNativeWindowFromSurfaceId fail!"); 145 146 // Set an output surface. 147 ImageEffect_ErrorCode errorCode = OH_ImageEffect_SetOutputSurface(imageEffect, outputNativeWindow); 148 CHECK_AND_RETURN_LOG(errorCode == ImageEffect_ErrorCode::EFFECT_SUCCESS, "OH_ImageEffect_SetOutputSurface fail!"); 149 150 // Obtain the input surface. Note that the obtained inputNativeWindow instance must be released by calling OH_NativeWindow_DestoryNativeWindow when it is no longer needed. 151 OHNativeWindow *inputNativeWindow = nullptr; 152 errorCode = OH_ImageEffect_GetInputSurface(imageEffect, &inputNativeWindow); 153 CHECK_AND_RETURN_LOG(errorCode == ImageEffect_ErrorCode::EFFECT_SUCCESS, "OH_ImageEffect_GetInputSurface fail!"); 154 155 // Obtain the surface ID from the inputNativeWindow instance. 156 uint64_t inputSurfaceId = 0; 157 res = OH_NativeWindow_GetSurfaceId(inputNativeWindow, &inputSurfaceId); 158 CHECK_AND_RETURN_LOG(res == 0, "OH_NativeWindow_GetSurfaceId fail!"); 159 160 // Convert the surface ID to a string and return the string. 161 std::string inputSurfaceIdStr = std::to_string(inputSurfaceId); 162 ``` 163 1644. Start the image effector. 165 166 ```c++ 167 // Start the image effector. 168 errorCode = OH_ImageEffect_Start(imageEffect); 169 CHECK_AND_RETURN_LOG(errorCode == ImageEffect_ErrorCode::EFFECT_SUCCESS, "OH_ImageEffect_Start fail!"); 170 ``` 171 1725. (Optional) Stop the image effector. This operation is required only in the input surface scenario. 173 174 ```c++ 175 // Stop the image effector. 176 errorCode = OH_ImageEffect_Stop(imageEffect); 177 CHECK_AND_RETURN_LOG(errorCode == ImageEffect_ErrorCode::EFFECT_SUCCESS, "OH_ImageEffect_Stop fail!"); 178 ``` 179 1806. (Optional) Serialize the image effector. 181 182 ```c++ 183 char *info = nullptr; 184 errorCode = OH_ImageEffect_Save(imageEffect, &info); 185 CHECK_AND_RETURN_LOG(errorCode == ImageEffect_ErrorCode::EFFECT_SUCCESS, "OH_ImageEffect_Save fail!"); 186 ``` 187 1887. Destroy the **ImageEffect** instance. 189 190 ```c++ 191 // Release the ImageEffect instance. 192 errorCode = OH_ImageEffect_Release(imageEffect); 193 CHECK_AND_RETURN_LOG(errorCode == ImageEffect_ErrorCode::EFFECT_SUCCESS, result, "OH_ImageEffect_Release fail!"); 194 ``` 195 196### Customizing a Filter 197 198To implement and register a custom filter, perform the following steps: 199 2001. Define **ImageEffect_FilterDelegate**. 201 202 ```c++ 203 // Image information struct. 204 struct EffectBufferInfo { 205 void *addr = nullptr; 206 int32_t width = 0; 207 int32_t height = 0; 208 int32_t rowSize = 0; 209 ImageEffect_Format format = ImageEffect_Format::EFFECT_PIXEL_FORMAT_UNKNOWN; 210 }; 211 212 // Implement a custom filter. 213 ImageEffect_FilterDelegate filterDelegate = { 214 .setValue = [](OH_EffectFilter *filter, const char *key, const ImageEffect_Any *value) { 215 // Verify the parameters. If the verification is successful, true is returned. Otherwise, false is returned. 216 // ... 217 return true; 218 }, 219 .render = [](OH_EffectFilter *filter, OH_EffectBufferInfo *info, OH_EffectFilterDelegate_PushData pushData) { 220 return Render(filter, info, pushData); 221 }, 222 .save = [](OH_EffectFilter *filter, char **info) { 223 // Obtain the custom filter parameter. In this example, Brightness is the key of the custom filter. 224 ImageEffect_Any value; 225 ImageEffect_ErrorCode errorCode = OH_EffectFilter_GetValue(filter, "Brightness", &value); 226 CHECK_AND_RETURN_LOG(errorCode == ImageEffect_ErrorCode::EFFECT_SUCCESS, false, "OH_EffectFilter_GetValue fail!"); 227 228 // Generate a key-value pair. 229 json values; 230 values["Brightness"] = value.dataValue.floatValue; 231 json root; 232 root["name"] = "CustomBrightness"; 233 root["values"] = values; 234 235 // Convert the JSON object into a string. 236 // ... 237 238 // Assign a serialized string address to *info. 239 *info = infoStr; 240 return true; 241 }, 242 .restore = [](const char *info) { 243 // Create an OH_EffectFilter instance, in which CustomBrightness is the name of the custom filter. 244 OH_EffectFilter *filter = OH_EffectFilter_Create("CustomBrightness"); 245 // Parse the JSON string info to obtain the key and value. 246 // ... 247 248 // Set a filter parameter. value is the parameter parsed based on the JSON string in info. 249 ImageEffect_ErrorCode errorCode = OH_EffectFilter_SetValue(filter, "Brightness", &value); 250 251 // ... 252 return filter; 253 } 254 }; 255 ``` 256 257 You can implement the **Render** API in two scenarios. 258 259 **Scenario 1: The custom algorithm can directly modify the pixel data in info (for example, the brightness filter).** 260 261 ```c++ 262 bool Render(OH_EffectFilter *filter, OH_EffectBufferInfo *info, OH_EffectFilterDelegate_PushData pushData) 263 { 264 // Obtain the image information. 265 EffectBufferInfo inputBufferInfo; 266 OH_EffectBufferInfo_GetAddr(info, &inputBufferInfo.addr); 267 OH_EffectBufferInfo_GetWidth(info, &inputBufferInfo.width); 268 OH_EffectBufferInfo_GetHeight(info, &inputBufferInfo.height); 269 OH_EffectBufferInfo_GetRowSize(info, &inputBufferInfo.rowSize); 270 OH_EffectBufferInfo_GetEffectFormat(info, &inputBufferInfo.format); 271 272 // Invoke the custom filter algorithm. 273 ApplyCustomAlgo(inputBufferInfo); 274 275 // After the editing is complete, call pushData to directly transfer the original image. 276 pushData(filter, info); 277 return true; 278 } 279 ``` 280 281 **Scenario 2: The custom algorithm cannot directly modify the pixel data in info (for example, the crop filter).** 282 283 ```c++ 284 bool Render(OH_EffectFilter *filter, OH_EffectBufferInfo *info, OH_EffectFilterDelegate_PushData pushData) 285 { 286 // Obtain the image information. 287 EffectBufferInfo inputBufferInfo; 288 OH_EffectBufferInfo_GetAddr(info, &inputBufferInfo.addr); 289 OH_EffectBufferInfo_GetWidth(info, &inputBufferInfo.width); 290 OH_EffectBufferInfo_GetHeight(info, &inputBufferInfo.height); 291 OH_EffectBufferInfo_GetRowSize(info, &inputBufferInfo.rowSize); 292 OH_EffectBufferInfo_GetEffectFormat(info, &inputBufferInfo.format); 293 294 // Create output pixel information. 295 EffectBufferInfo outputBufferInfo = CreateOutputBufferInfo(inputBufferInfo); 296 297 // Invoke the custom filter algorithm. 298 ApplyCustomAlgo(inputBufferInfo, outputBufferInfo); 299 300 // Generate outputOhInfo. 301 OH_EffectBufferInfo *outputOhInfo = OH_EffectBufferInfo_Create(); 302 OH_EffectBufferInfo_SetAddr(outputOhInfo, outputBufferInfo.addr); 303 OH_EffectBufferInfo_SetWidth(outputOhInfo, outputBufferInfo.width); 304 OH_EffectBufferInfo_SetHeight(outputOhInfo, outputBufferInfo.height); 305 OH_EffectBufferInfo_SetRowSize(outputOhInfo, outputBufferInfo.rowSize); 306 OH_EffectBufferInfo_SetEffectFormat(outputOhInfo, outputBufferInfo.format); 307 308 // Call pushData to transfer outputOhInfo after editing. 309 pushData(filter, outputOhInfo); 310 311 // Release resources. 312 OH_EffectBufferInfo_Release(outputOhInfo); 313 ReleaseOutputBuffer(outputBufferInfo.addr); 314 315 return true; 316 } 317 ``` 318 3192. Generate custom filter information. 320 321 ```c++ 322 // Create an OH_EffectFilterInfo instance. 323 OH_EffectFilterInfo *customFilterInfo = OH_EffectFilterInfo_Create(); 324 CHECK_AND_RETURN_LOG(customFilterInfo != nullptr, "OH_EffectFilter_GetValue fail!"); 325 326 // Set the name of the custom filter. 327 OH_EffectFilterInfo_SetFilterName(customFilterInfo, "CustomBrightness"); 328 329 // Set the buffer types supported by the custom filter. 330 ImageEffect_BufferType bufferTypeArray[] = { ImageEffect_BufferType::EFFECT_BUFFER_TYPE_PIXEL }; 331 OH_EffectFilterInfo_SetSupportedBufferTypes(customFilterInfo, sizeof(bufferTypeArray) / sizeof(ImageEffect_BufferType), bufferTypeArray); 332 333 // Set the pixel formats supported by the custom filter. 334 ImageEffect_Format formatArray[] = { ImageEffect_Format::EFFECT_PIXEL_FORMAT_RGBA8888 }; 335 OH_EffectFilterInfo_SetSupportedFormats(customFilterInfo, sizeof(formatArray) / sizeof(ImageEffect_Format), formatArray); 336 ``` 337 3383. Register **ImageEffect_FilterDelegate** with the image effector. 339 340 ```c++ 341 // Register the custom filter. 342 ImageEffect_ErrorCode errorCode = OH_EffectFilter_Register(customFilterInfo, &filterDelegate); 343 CHECK_AND_RETURN_LOG(errorCode == ImageEffect_ErrorCode::EFFECT_SUCCESS, "OH_EffectFilter_Register fail!"); 344 ``` 345 346### Using a System Defined Filter 347 3481. Create a filter. 349 350 ```c++ 351 // Create a filter, for example, a contrast filter. 352 OH_EffectFilter *filter = OH_EffectFilter_Create(OH_EFFECT_CONTRAST_FILTER); 353 ``` 354 3552. Set a filter parameter. 356 357 ```c++ 358 // Set a filter parameter. For example, set the intensity to 50. 359 ImageEffect_Any value = {.dataType = ImageEffect_DataType::EFFECT_DATA_TYPE_FLOAT, .dataValue.floatValue = 50.f}; 360 ImageEffect_ErrorCode errorCode = OH_EffectFilter_SetValue(filter, OH_EFFECT_FILTER_INTENSITY_KEY, &value); 361 CHECK_AND_RETURN_LOG(errorCode == ImageEffect_ErrorCode::EFFECT_SUCCESS, "OH_EffectFilter_SetValue fail!"); 362 ``` 363 3643. Make the filter take effect. 365 366 ```c++ 367 // Start rendering with the filter. 368 errorCode = OH_EffectFilter_Render(filter, inputPixelmap, outputPixelmap); 369 ``` 370 3714. Destroy the filter instance. 372 373 ```c++ 374 // Destroy the filter instance. 375 errorCode = OH_EffectFilter_Release(filter); 376 ``` 377 378### Query Capabilities 379 380- Query filter information based on the filter name. 381 382 ```c++ 383 // Create an OH_EffectFilterInfo instance. 384 OH_EffectFilterInfo *filterInfo = OH_EffectFilterInfo_Create(); 385 CHECK_AND_RETURN_LOG(filterInfo != nullptr, "OH_EffectFilterInfo_Create fail!"); 386 387 // Query the filter information based on the filter name. 388 ImageEffect_ErrorCode errorCode = OH_EffectFilter_LookupFilterInfo(OH_EFFECT_BRIGHTNESS_FILTER, filterInfo); 389 CHECK_AND_RETURN_LOG(errorCode == ImageEffect_ErrorCode::EFFECT_SUCCESS, "OH_EffectFilter_LookupFilterInfo fail!"); 390 391 // Obtain the filter name from the filter information. 392 char *name = nullptr; 393 OH_EffectFilterInfo_GetFilterName(filterInfo, &name); 394 395 // Obtain the supported buffer types. 396 uint32_t supportedBufferTypesCnt = 0; 397 ImageEffect_BufferType *bufferTypeArray = nullptr; 398 OH_EffectFilterInfo_GetSupportedBufferTypes(filterInfo, &supportedBufferTypesCnt, &bufferTypeArray); 399 400 // Obtain the supported pixel formats. 401 uint32_t supportedFormatsCnt = 0; 402 ImageEffect_Format *formatArray = nullptr; 403 OH_EffectFilterInfo_GetSupportedFormats(filterInfo, supportedFormatsCnt, &formatArray); 404 405 // Destroy the OH_EffectFilterInfo instance. 406 OH_EffectFilterInfo_Release(filterInfo); 407 ``` 408 409- Query filters that meet given conditions. 410 411 ```c++ 412 // Query all filters. Resources need to be released. 413 ImageEffect_FilterNames *filterNames = OH_EffectFilter_LookupFilters("Default"); 414 415 // ... 416 417 // Release virtual memory resources by filter names. 418 OH_EffectFilter_ReleaseFilterNames(); 419 ``` 420 421