1# Audio Encoding 2 3You can call the native APIs provided by the AudioCodec module to encode audio, that is, to compress audio PCM data into a desired format. 4 5PCM data can be from any source. For example, you can use a microphone to record audio data or import edited PCM data. After audio encoding, you can output streams in the desired format and encapsulate the streams into a target file. 6 7For details about the supported encoding capabilities, see [AVCodec Supported Formats](avcodec-support-formats.md#audio-encoding). 8 9**Usage Scenario** 10 11- Audio recording 12 13 Record and pass in PCM data, and encode the data into streams in the desired format. 14- Audio editing 15 16 Export edited PCM data, and encode the data into streams in the desired format. 17> **NOTE** 18> 19> AAC encoders adopt the VBR mode by default, which may differ in the configured parameters. 20 21## How to Develop 22 23Read [AudioCodec](../../reference/apis-avcodec-kit/_audio_codec.md) for the API reference. 24 25Refer to the code snippet below to complete the entire audio encoding process, including creating an encoder, setting encoding parameters (such as the sampling rate, bit rate, and number of audio channels), and starting, refreshing, resetting, and destroying the encoder. 26 27During application development, you must call the APIs in the defined sequence. Otherwise, an exception or undefined behavior may occur. 28 29The figure below shows the call relationship of audio encoding. 30 31- The dotted line indicates an optional operation. 32 33- The solid line indicates a mandatory operation. 34 35 36 37### Linking the Dynamic Libraries in the CMake Script 38 39```cmake 40target_link_libraries(sample PUBLIC libnative_media_codecbase.so) 41target_link_libraries(sample PUBLIC libnative_media_core.so) 42target_link_libraries(sample PUBLIC libnative_media_acodec.so) 43``` 44 45### How to Develop 46 471. Add the header files. 48 49 ```cpp 50 #include <multimedia/player_framework/native_avcodec_audiocodec.h> 51 #include <multimedia/native_audio_channel_layout.h> 52 #include <multimedia/player_framework/native_avcapability.h> 53 #include <multimedia/player_framework/native_avcodec_base.h> 54 #include <multimedia/player_framework/native_avformat.h> 55 #include <multimedia/player_framework/native_avbuffer.h> 56 ``` 57 582. Create an encoder instance. In the code snippet below, **OH_AVCodec *** is the pointer to the encoder instance created. 59 60 You can create an encoder by name or MIME type. 61 62 ```cpp 63 // Namespace of the C++ standard library. 64 using namespace std; 65 // Create an encoder by name. 66 OH_AVCapability *capability = OH_AVCodec_GetCapability(OH_AVCODEC_MIMETYPE_AUDIO_AAC, true); 67 const char *name = OH_AVCapability_GetName(capability); 68 OH_AVCodec *audioEnc_ = OH_AudioCodec_CreateByName(name); 69 ``` 70 71 ```cpp 72 // Specify whether encoding is used. The value true means encoding. 73 bool isEncoder = true; 74 // Create an encoder by MIME type. 75 OH_AVCodec *audioEnc_ = OH_AudioCodec_CreateByMime(OH_AVCODEC_MIMETYPE_AUDIO_AAC, isEncoder); 76 ``` 77 78 ```cpp 79 // Initialize the queues. 80 class AEncBufferSignal { 81 public: 82 std::mutex inMutex_; 83 std::mutex outMutex_; 84 std::mutex startMutex_; 85 std::condition_variable inCond_; 86 std::condition_variable outCond_; 87 std::condition_variable startCond_; 88 std::queue<uint32_t> inQueue_; 89 std::queue<uint32_t> outQueue_; 90 std::queue<OH_AVBuffer *> inBufferQueue_; 91 std::queue<OH_AVBuffer *> outBufferQueue_; 92 }; 93 AEncBufferSignal *signal_; 94 ``` 95 963. Call **OH_AudioCodec_RegisterCallback()** to register callback functions. 97 98 Register the **OH_AVCodecCallback** struct that defines the following callback function pointers: 99 100 - **OH_AVCodecOnError**, a callback used to report a codec operation error 101 - **OH_AVCodecOnStreamChanged**, a callback used to report a codec stream change, for example, audio channel change 102 - **OH_AVCodecOnNeedInputBuffer**, a callback used to report input data required, which means that the encoder is ready for receiving PCM data 103 - **OH_AVCodecOnNewOutputBuffer**, a callback used to report output data generated, which means that encoding is complete 104 105 You need to process the callback functions to ensure that the encoder runs properly. 106 107 ```cpp 108 // Implement the OH_AVCodecOnError callback function. 109 static void OnError(OH_AVCodec *codec, int32_t errorCode, void *userData) 110 { 111 (void)codec; 112 (void)errorCode; 113 (void)userData; 114 } 115 // Implement the OH_AVCodecOnStreamChanged callback function. 116 static void OnOutputFormatChanged(OH_AVCodec *codec, OH_AVFormat *format, void *userData) 117 { 118 (void)codec; 119 (void)format; 120 (void)userData; 121 } 122 // Implement the OH_AVCodecOnNeedInputBuffer callback function. 123 static void OnInputBufferAvailable(OH_AVCodec *codec, uint32_t index, OH_AVBuffer *data, void *userData) 124 { 125 (void)codec; 126 // The input stream is sent to the InputBuffer queue. 127 AEncBufferSignal *signal = static_cast<AEncBufferSignal *>(userData); 128 unique_lock<mutex> lock(signal->inMutex_); 129 signal->inQueue_.push(index); 130 signal->inBufferQueue_.push(data); 131 signal->inCond_.notify_all(); 132 } 133 // Implement the OH_AVCodecOnNewOutputBuffer callback function. 134 static void OnOutputBufferAvailable(OH_AVCodec *codec, uint32_t index, OH_AVBuffer *data, void *userData) 135 { 136 (void)codec; 137 // The index of the output buffer is sent to OutputQueue_. 138 // The encoded data is sent to the outBuffer queue. 139 AEncBufferSignal *signal = static_cast<AEncBufferSignal *>(userData); 140 unique_lock<mutex> lock(signal->outMutex_); 141 signal->outQueue_.push(index); 142 signal->outBufferQueue_.push(data); 143 } 144 signal_ = new AEncBufferSignal(); 145 OH_AVCodecCallback cb_ = {&OnError, &OnOutputFormatChanged, &OnInputBufferAvailable, &OnOutputBufferAvailable}; 146 // Set the asynchronous callbacks. 147 int32_t ret = OH_AudioCodec_RegisterCallback(audioEnc_, cb_, signal_); 148 if (ret != AV_ERR_OK) { 149 // Exception handling. 150 } 151 ``` 152 1534. Call **OH_AudioCodec_Configure** to configure the encoder. 154 155 The following options are mandatory: sampling rate, bit rate, number of audio channels, audio channel type, and bit depth. 156 157 The maximum input length is optional. 158 159 For FLAC encoding, the compliance level and sampling precision are also mandatory. 160 161 The sample below lists the value range of each audio encoding type. 162 | Audio Encoding Type| Sampling Rate (Hz) | Audio Channel Count | 163 | ----------- | ------------------------------------------------------------------------------- | :----------------: | 164 | AAC | 8000, 11025, 12000, 16000, 22050, 24000, 32000, 44100, 48000, 64000, 88200, 96000| 1, 2, 3, 4, 5, 6, and 8| 165 | FLAC | 8000, 11025, 12000, 16000, 22050, 24000, 32000, 44100, 48000, 64000, 88200, 96000| 1–8 | 166 | MP3 | 8000, 11025, 12000, 16000, 22050, 24000, 32000, 44100, 48000 | 1–2 | 167 | G711mu | 8000 | 1 | 168 <!--RP3--><!--RP3End--> 169 170 The code snippet below shows the API call process, where AAC encoding at the bit rate of 32000 bit/s is carried out on the PCM audio with the 44100 Hz sampling rate, 2-channel stereo, and SAMPLE_S16LE sampling format. 171 <!--RP4--> 172 ```cpp 173 int32_t ret; 174 // (Mandatory) Configure the audio sampling rate. 175 constexpr uint32_t DEFAULT_SAMPLERATE = 44100; 176 // (Mandatory) Configure the audio bit rate. 177 constexpr uint64_t DEFAULT_BITRATE = 32000; 178 // (Mandatory) Configure the number of audio channels. 179 constexpr uint32_t DEFAULT_CHANNEL_COUNT = 2; 180 // (Mandatory) Configure the audio channel type. 181 constexpr OH_AudioChannelLayout CHANNEL_LAYOUT = OH_AudioChannelLayout::CH_LAYOUT_STEREO; 182 // (Mandatory) Configure the audio bit depth. 183 constexpr OH_BitsPerSample SAMPLE_FORMAT = OH_BitsPerSample::SAMPLE_S16LE; 184 // A frame of audio data takes 20 ms. 185 constexpr float TIME_PER_FRAME = 0.02; 186 // (Optional) Configure the maximum input length and the size of each frame of audio data. 187 constexpr uint32_t DEFAULT_MAX_INPUT_SIZE = DEFAULT_SAMPLERATE * TIME_PER_FRAME * DEFAULT_CHANNEL_COUNT * sizeof(short); // aac 188 OH_AVFormat *format = OH_AVFormat_Create(); 189 // Set the format. 190 OH_AVFormat_SetIntValue(format,OH_MD_KEY_AUD_CHANNEL_COUNT, DEFAULT_CHANNEL_COUNT); 191 OH_AVFormat_SetIntValue(format,OH_MD_KEY_AUD_SAMPLE_RATE, DEFAULT_SAMPLERATE); 192 OH_AVFormat_SetLongValue(format,OH_MD_KEY_BITRATE, DEFAULT_BITRATE); 193 OH_AVFormat_SetIntValue(format, OH_MD_KEY_AUDIO_SAMPLE_FORMAT, SAMPLE_FORMAT); 194 OH_AVFormat_SetLongValue(format,OH_MD_KEY_CHANNEL_LAYOUT, CHANNEL_LAYOUT); 195 OH_AVFormat_SetIntValue(format,OH_MD_KEY_MAX_INPUT_SIZE, DEFAULT_MAX_INPUT_SIZE); 196 // Configure the encoder. 197 ret = OH_AudioCodec_Configure(audioEnc_, format); 198 if (ret != AV_ERR_OK) { 199 // Exception handling. 200 } 201 ``` 202 <!--RP4End--> 203 The following shows the API call process in the case of FLAC encoding. 204 205 ```cpp 206 int32_t ret; 207 // (Mandatory) Configure the audio sampling rate. 208 constexpr uint32_t DEFAULT_SAMPLERATE = 44100; 209 // (Mandatory) Configure the audio bit rate. 210 constexpr uint64_t DEFAULT_BITRATE = 261000; 211 // (Mandatory) Configure the number of audio channels. 212 constexpr uint32_t DEFAULT_CHANNEL_COUNT = 2; 213 // (Mandatory) Configure the audio channel type. 214 constexpr OH_AudioChannelLayout CHANNEL_LAYOUT = OH_AudioChannelLayout::CH_LAYOUT_STEREO; 215 // (Mandatory) Configure the audio bit depth. Only SAMPLE_S16LE and SAMPLE_S32LE are available for FLAC encoding. 216 constexpr OH_BitsPerSample SAMPLE_FORMAT = OH_BitsPerSample::SAMPLE_S32LE; 217 // Configure the audio compliance level. The default value is 0, and the value ranges from -2 to 2. 218 constexpr int32_t COMPLIANCE_LEVEL = 0; 219 // (Mandatory) Configure the audio sampling precision. SAMPLE_S16LE, SAMPLE_S24LE, and SAMPLE_S32LE are available. 220 constexpr OH_BitsPerSample BITS_PER_CODED_SAMPLE = OH_BitsPerSample::SAMPLE_S24LE; 221 OH_AVFormat *format = OH_AVFormat_Create(); 222 // Set the format. 223 OH_AVFormat_SetIntValue(format, OH_MD_KEY_AUD_CHANNEL_COUNT, DEFAULT_CHANNEL_COUNT); 224 OH_AVFormat_SetIntValue(format, OH_MD_KEY_AUD_SAMPLE_RATE, DEFAULT_SAMPLERATE); 225 OH_AVFormat_SetLongValue(format, OH_MD_KEY_BITRATE, DEFAULT_BITRATE); 226 OH_AVFormat_SetIntValue(format, OH_MD_KEY_BITS_PER_CODED_SAMPLE, BITS_PER_CODED_SAMPLE); 227 OH_AVFormat_SetIntValue(format, OH_MD_KEY_AUDIO_SAMPLE_FORMAT, SAMPLE_FORMAT); 228 OH_AVFormat_SetLongValue(format, OH_MD_KEY_CHANNEL_LAYOUT, CHANNEL_LAYOUT); 229 OH_AVFormat_SetLongValue(format, OH_MD_KEY_COMPLIANCE_LEVEL, COMPLIANCE_LEVEL); 230 // Configure the encoder. 231 ret = OH_AudioCodec_Configure(audioEnc_, format); 232 if (ret != AV_ERR_OK) { 233 // Exception handling. 234 } 235 ``` 236 237 <!--RP2--><!--RP2End--> 238 2395. Call **OH_AudioCodec_Prepare()** to prepare internal resources for the encoder. 240 241 ```cpp 242 ret = OH_AudioCodec_Prepare(audioEnc_); 243 if (ret != AV_ERR_OK) { 244 // Exception handling. 245 } 246 ``` 247 2486. Call **OH_AudioCodec_Start()** to start the encoder. 249 250 ```c++ 251 unique_ptr<ifstream> inputFile_ = make_unique<ifstream>(); 252 unique_ptr<ofstream> outFile_ = make_unique<ofstream>(); 253 // Open the path of the binary file to be encoded. (A PCM file is used as an example.) 254 inputFile_->open(inputFilePath.data(), ios::in | ios::binary); 255 // Configure the path of the output file. (An encoded stream file is used as an example.) 256 outFile_->open(outputFilePath.data(), ios::out | ios::binary); 257 // Start encoding. 258 ret = OH_AudioCodec_Start(audioEnc_); 259 if (ret != AV_ERR_OK) { 260 // Exception handling. 261 } 262 ``` 263 2647. Call **OH_AudioCodec_PushInputBuffer()** to write the data to encode. 265 266 For AAC encoding, set **SAMPLES_PER_FRAME** to the number of PCM samples every 20 ms, that is, sampling rate x 0.02. 267 268 For FLAC encoding, set **SAMPLES_PER_FRAME** based on the table below. 269 270 | Sampling Rate| Sample Count| 271 | :----: | :----: | 272 | 8000 | 576 | 273 | 16000 | 1152 | 274 | 22050 | 2304 | 275 | 24000 | 2304 | 276 | 32000 | 2304 | 277 | 44100 | 4608 | 278 | 48000 | 4608 | 279 | 88200 | 8192 | 280 | 96000 | 8192 | 281 282 > **NOTE** 283 > 284 > It is recommended that **SAMPLES_PER_FRAME** in AAC encoding be the number of PCM samples every 20 ms, that is, sampling rate x 0.02. In the case of FLAC encoding, if the number of samples is greater than the corresponding value provided in the table, an error code is returned. If the number is less than the corresponding value provided in the table, the encoded file may be damaged. 285 286 ```c++ 287 // Number of samples per frame. 288 constexpr int32_t SAMPLES_PER_FRAME = DEFAULT_SAMPLERATE * TIME_PER_FRAME; 289 // Number of audio channels. For AMR encoding, only mono audio input is supported. 290 constexpr int32_t DEFAULT_CHANNEL_COUNT = 2; 291 // Length of the input data of each frame, that is, number of audio channels x number of samples per frame x number of bytes per sample (SAMPLE_S16LE used as an example). 292 constexpr int32_t INPUT_FRAME_BYTES = DEFAULT_CHANNEL_COUNT * SAMPLES_PER_FRAME * sizeof(short); 293 uint32_t index = signal_->inQueue_.front(); 294 auto buffer = signal_->inBufferQueue_.front(); 295 OH_AVCodecBufferAttr attr = {0}; 296 if (!inputFile_->eof()) { 297 inputFile_->read((char *)OH_AVBuffer_GetAddr(buffer), INPUT_FRAME_BYTES); 298 attr.size = INPUT_FRAME_BYTES; 299 attr.flags = AVCODEC_BUFFER_FLAGS_NONE; 300 } else { 301 attr.size = 0; 302 attr.flags = AVCODEC_BUFFER_FLAGS_EOS; 303 } 304 OH_AVBuffer_SetBufferAttr(buffer, &attr); 305 // Send the data to the input queue for encoding. The index is the subscript of the queue. 306 ret = OH_AudioCodec_PushInputBuffer(audioEnc_, index); 307 if (ret != AV_ERR_OK) { 308 // Exception handling. 309 } 310 ``` 311 In the preceding example, **attr.flags** indicates the type of the buffer flag. 312 313 To indicate the End of Stream (EOS), pass in the **AVCODEC_BUFFER_FLAGS_EOS** flag. 314 | Value| Description| 315 | -------- | -------- | 316 | AVCODEC_BUFFER_FLAGS_NONE | Common frame.| 317 | AVCODEC_BUFFER_FLAGS_EOS | The buffer is an end-of-stream frame.| 318 | AVCODEC_BUFFER_FLAGS_CODEC_DATA | The buffer contains codec-specific data.| 319 3208. Call **OH_AudioCodec_FreeOutputBuffer()** to output the encoded stream. 321 322 ```c++ 323 uint32_t index = signal_->outQueue_.front(); 324 OH_AVBuffer *avBuffer = signal_->outBufferQueue_.front(); 325 // Obtain the buffer attributes. 326 OH_AVCodecBufferAttr attr = {0}; 327 ret = OH_AVBuffer_GetBufferAttr(avBuffer, &attr); 328 if (ret != AV_ERR_OK) { 329 // Exception handling. 330 } 331 // Write the encoded data (specified by data) to the output file. 332 outputFile_->write(reinterpret_cast<char *>(OH_AVBuffer_GetAddr(avBuffer)), attr.size); 333 // Release the output buffer. 334 ret = OH_AudioCodec_FreeOutputBuffer(audioEnc_, index); 335 if (ret != AV_ERR_OK) { 336 // Exception handling. 337 } 338 if (attr.flags == AVCODEC_BUFFER_FLAGS_EOS) { 339 // End 340 } 341 ``` 342 3439. (Optional) Call **OH_AudioCodec_Flush()** to refresh the encoder. 344 345 After **OH_AudioCodec_Flush()** is called, the current encoding queue is cleared. 346 347 To continue encoding, you must call **OH_AudioCodec_Start()** again. 348 349 You need to call **OH_AudioCodec_Flush()** in the following cases: 350 351 * The EOS of the file is reached. 352 * An error with **OH_AudioCodec_IsValid** set to **true** (indicating that the execution can continue) occurs. 353 354 ```c++ 355 // Refresh the encoder. 356 ret = OH_AudioCodec_Flush(audioEnc_); 357 if (ret != AV_ERR_OK) { 358 // Exception handling. 359 } 360 // Start encoding again. 361 ret = OH_AudioCodec_Start(audioEnc_); 362 if (ret != AV_ERR_OK) { 363 // Exception handling. 364 } 365 ``` 366 36710. (Optional) Call **OH_AudioCodec_Reset()** to reset the encoder. 368 369 After **OH_AudioCodec_Reset()** is called, the encoder returns to the initialized state. To continue encoding, you must call **OH_AudioCodec_Configure()** and then **OH_AudioCodec_Start()**. 370 371 ```c++ 372 // Reset the encoder. 373 ret = OH_AudioCodec_Reset(audioEnc_); 374 if (ret != AV_ERR_OK) { 375 // Exception handling. 376 } 377 // Reconfigure the encoder. 378 ret = OH_AudioCodec_Configure(audioEnc_, format); 379 if (ret != AV_ERR_OK) { 380 // Exception handling. 381 } 382 ``` 383 38411. Call **OH_AudioCodec_Stop()** to stop the encoder. 385 386 ```c++ 387 // Stop the encoder. 388 ret = OH_AudioCodec_Stop(audioEnc_); 389 if (ret != AV_ERR_OK) { 390 // Exception handling. 391 } 392 ``` 393 39412. Call **OH_AudioCodec_Destroy()** to destroy the encoder instance and release resources. 395 396 > **NOTE** 397 > 398 > You only need to call the API once. 399 400 ```c++ 401 // Call OH_AudioCodec_Destroy to destroy the encoder. 402 ret = OH_AudioCodec_Destroy(audioEnc_); 403 if (ret != AV_ERR_OK) { 404 // Exception handling. 405 } else { 406 audioEnc_ = NULL; // The encoder cannot be destroyed repeatedly. 407 } 408 ``` 409
