1# Taking Over the Media Playback on Web Pages
2
3The **Web** component provides the capability for applications to take over media playback on web pages, which improves media playback qualities on the web page.
4
5## When to Use
6
7On web pages, media players are simple and provide few functions, with lower video quality and some videos cannot be played.
8
9In this case, you can use your own or a third-party player to take over web page media playback to improve media playback experience.
10
11## Implementation Principle
12
13### Framework of Using the ArkWeb Kernel to Play Media
14
15When web media playback takeover is disabled, the playback architecture of the ArkWeb kernel is as follows:
16
17  ![arkweb media pipeline](figures/arkweb_media_pipeline.png)
18
19  > **NOTE**
20  >
21  > - In the preceding figure, step 1 indicates that the ArkWeb kernel creates a **WebMediaPlayer** to play media resources on web pages.
22  > - Step 2 indicates that the **WebMediaPlayer** uses the system decoder to render media data.
23
24When web media playback takeover is enabled, the playback architecture of the ArkWeb kernel is as follows:
25
26  ![arkweb native media player](figures/arkweb_native_media_player.png)
27
28  > **NOTE**
29  >
30  > - In the preceding figure, step 1 indicates that the ArkWeb kernel creates a **WebMediaPlayer** to play media resources on web pages.
31  > - Step 2 indicates that **WebMediaPlayer** uses **NativeMediaPlayer** provided by the application to render media data.
32
33
34### Interactions Between the ArkWeb Kernel and Application
35
36  ![interactions between arkweb and native media player](figures/interactions_between_arkweb_and_native_media_player.png)
37
38  > **NOTE**
39  >
40  > - For details about step 1 in the preceding figure, see [Enabling Web Media Playback Takeover](#enabling-web-media-playback-takeover).
41  > - For details about step 2, see [Creating a Native Media Player](#creating-a-native-media-player).
42  > - For details about step 3, see [Drawing Native Media Player Components](#drawing-native-media-player-components).
43  > - For details about step 4, see [Executing Playback Control Commands Sent by ArkWeb Kernel to the Native Media Player](#executing-playback-control-commands-sent-by-arkweb-kernel-to-the-native-media-player).
44  > - For details about step 5, see [Notifying the Status Information of Native Media Player to the ArkWeb Kernel](#notifying-the-status-information-of-native-media-player-to-the-arkweb-kernel).
45
46## How to Develop
47
48### Enabling Web Media Playback Takeover
49
50You need to use the [enableNativeMediaPlayer](../reference/apis-arkweb/ts-basic-components-web.md#enablenativemediaplayer12) API to enable the function of taking over web page media playback.
51
52  ```ts
53  // xxx.ets
54  import { webview } from '@kit.ArkWeb';
55
56  @Entry
57  @Component
58  struct WebComponent {
59    controller: webview.WebviewController = new webview.WebviewController();
60
61    build() {
62      Column() {
63        Web({ src: 'www.example.com', controller: this.controller })
64          .enableNativeMediaPlayer({ enable: true, shouldOverlay: false })
65      }
66    }
67  }
68  ```
69
70### Creating a Native Media Player
71
72Once web media playback takeover is enabled, the ArkWeb kernel triggers the callback registered by [onCreateNativeMediaPlayer](../reference/apis-arkweb/js-apis-webview.md#oncreatenativemediaplayer12) each time a media file needs to be played on the web page.
73
74You need to register a callback for creating a native media player by invoking **onCreateNativeMediaPlayer**.
75
76The callback function determines whether to create a native media player to take over the web page media resources based on the media information.
77
78  * If the application does not take over the web page media resource, **null** is returned in the callback function.
79  * If the application takes over the web page media resource, a native media player instance is returned in the callback function.
80
81The native media player needs to implement the [NativeMediaPlayerBridge](../reference/apis-arkweb/js-apis-webview.md#nativemediaplayerbridge12) API so that the ArkWeb kernel can control the playback on the native media player.
82
83  ```ts
84  // xxx.ets
85  import { webview } from '@kit.ArkWeb';
86
87  // Implement the webview.NativeMediaPlayerBridge API.
88  // The ArkWeb kernel calls the webview.NativeMediaPlayerBridge methods to control playback on NativeMediaPlayer.
89  class NativeMediaPlayerImpl implements webview.NativeMediaPlayerBridge {
90    // ...Implement the APIs in NativeMediaPlayerBridge...
91    constructor(handler: webview.NativeMediaPlayerHandler, mediaInfo: webview.MediaInfo) {}
92    updateRect(x: number, y: number, width: number, height: number) {}
93    play() {}
94    pause() {}
95    seek(targetTime: number) {}
96    release() {}
97    setVolume(volume: number) {}
98    setMuted(muted: boolean) {}
99    setPlaybackRate(playbackRate: number) {}
100    enterFullscreen() {}
101    exitFullscreen() {}
102  }
103
104  @Entry
105  @Component
106  struct WebComponent {
107    controller: webview.WebviewController = new webview.WebviewController();
108
109    build() {
110      Column() {
111        Web({ src: 'www.example.com', controller: this.controller })
112          .enableNativeMediaPlayer({ enable: true, shouldOverlay: false })
113          .onPageBegin((event) => {
114            this.controller.onCreateNativeMediaPlayer((handler: webview.NativeMediaPlayerHandler, mediaInfo: webview.MediaInfo) => {
115              // Determine whether to take over the media.
116              if (!shouldHandle(mediaInfo)) {
117                // The native media player does not take over the media.
118                // Return null. The ArkWeb kernel will play the media with the web media player.
119                return null;
120              }
121              // Take over the web media.
122              // Return a native media player instance to the ArkWeb kernel.
123              let nativePlayer: webview.NativeMediaPlayerBridge = new NativeMediaPlayerImpl(handler, mediaInfo);
124              return nativePlayer;
125            });
126          })
127      }
128    }
129  }
130
131  // stub
132  function shouldHandle(mediaInfo: webview.MediaInfo) {
133    return true;
134  }
135  ```
136
137### Drawing Native Media Player Components
138
139When an application takes over the media playback on web pages, you need to draw the native media player component and video images on the surface provided by the ArkWeb kernel. Then the ArkWeb kernel combines the surface with the web page and displays it on the screen.
140
141This process is the same as that of [Rendering and Drawing XComponent+AVPlayer and Button Components at the Same Layer](web-same-layer.md#)
142
1431. In the application startup phase, **UIContext** must be saved so that it can be used in subsequent rendering and drawing processes at the same layer.
144
145   ```ts
146   // xxxAbility.ets
147
148   import { UIAbility } from '@kit.AbilityKit';
149   import { window } from '@kit.ArkUI';
150
151   export default class EntryAbility extends UIAbility {
152     onWindowStageCreate(windowStage: window.WindowStage): void {
153       windowStage.loadContent('pages/Index', (err, data) => {
154         if (err.code) {
155           return;
156         }
157         // Save UIContext, which will be used in subsequent rendering and drawing at the same layer.
158         AppStorage.setOrCreate<UIContext>("UIContext", windowStage.getMainWindowSync().getUIContext());
159       });
160     }
161
162     // ...Other APIs that need to be overridden...
163   }
164   ```
165
1662. Use the surface created by the ArkWeb kernel for rendering and drawing at the same layer.
167
168   ```ts
169   // xxx.ets
170   import { webview } from '@kit.ArkWeb';
171   import { BuilderNode, FrameNode, NodeController, NodeRenderType } from '@kit.ArkUI';
172
173   interface ComponentParams {}
174
175   class MyNodeController extends NodeController {
176     private rootNode: BuilderNode<[ComponentParams]> | undefined;
177
178     constructor(surfaceId: string, renderType: NodeRenderType) {
179       super();
180
181       // Obtain the saved UIContext.
182       let uiContext = AppStorage.get<UIContext>("UIContext");
183       this.rootNode = new BuilderNode(uiContext as UIContext, { surfaceId: surfaceId, type: renderType });
184     }
185
186     makeNode(uiContext: UIContext): FrameNode | null {
187       if (this.rootNode) {
188         return this.rootNode.getFrameNode() as FrameNode;
189       }
190       return null;
191     }
192
193     build() {
194       // Construct the native media player component.
195     }
196   }
197
198   @Entry
199   @Component
200   struct WebComponent {
201     node_controller?: MyNodeController;
202     controller: webview.WebviewController = new webview.WebviewController();
203     @State show_native_media_player: boolean = false;
204
205     build() {
206       Column() {
207         Stack({ alignContent: Alignment.TopStart }) {
208           if (this.show_native_media_player) {
209             NodeContainer(this.node_controller)
210               .width(300)
211               .height(150)
212               .backgroundColor(Color.Transparent)
213               .border({ width: 2, color: Color.Orange })
214           }
215           Web({ src: 'www.example.com', controller: this.controller })
216             .enableNativeMediaPlayer({ enable: true, shouldOverlay: false })
217             .onPageBegin((event) => {
218               this.controller.onCreateNativeMediaPlayer((handler: webview.NativeMediaPlayerHandler, mediaInfo:    webview.MediaInfo) => {
219                 // Take over the web media.
220
221                 // Use the surface provided by the rendering at the same layer to construct a native media player component.
222                 this.node_controller = new MyNodeController(mediaInfo.surfaceInfo.id, NodeRenderType.RENDER_TYPE_TEXTURE);
223                 this.node_controller.build();
224
225                 // Show the native media player component.
226                 this.show_native_media_player = true;
227
228                 // Return a native media player instance to the ArkWeb kernel.
229                 return null;
230               });
231             })
232         }
233       }
234     }
235   }
236   ```
237
238For details about how to dynamically create components and draw them on the surface, see [Rendering and Drawing XComponent+AVPlayer and Button Components at the Same Layer](web-same-layer.md#).
239
240### Executing Playback Control Commands Sent by ArkWeb Kernel to the Native Media Player
241
242To facilitate the control over native media player by the ArkWeb kernel, you need to implement the [NativeMediaPlayerBridge](../reference/apis-arkweb/js-apis-webview.md#nativemediaplayerbridge12) API on the native media player and operate the native media player based on the function of each API.
243
244  ```ts
245  // xxx.ets
246  import { webview } from '@kit.ArkWeb';
247
248  class ActualNativeMediaPlayerListener {
249    constructor(handler: webview.NativeMediaPlayerHandler) {}
250  }
251
252  class NativeMediaPlayerImpl implements webview.NativeMediaPlayerBridge {
253    constructor(handler: webview.NativeMediaPlayerHandler, mediaInfo: webview.MediaInfo) {
254      // 1. Create a listener for the native media player.
255      let listener: ActualNativeMediaPlayerListener = new ActualNativeMediaPlayerListener(handler);
256      // 2. Create a native media player.
257      // 3. Listen for the native media player.
258      // ...
259    }
260
261    updateRect(x: number, y: number, width: number, height: number) {
262      // The position and size of the <video> tag are changed.
263      // Make changes based on the information change.
264    }
265
266    play() {
267      // Starts the native media player for playback.
268    }
269
270    pause() {
271      // Pause the playback.
272    }
273
274    seek(targetTime: number) {
275      // The native media player seeks to the target playback time.
276    }
277
278    release() {
279      // Destroy the native media player.
280    }
281
282    setVolume(volume: number) {
283      // The ArkWeb kernel adjusts the volume of the native media player.
284      // Set the volume of the native media player.
285    }
286
287    setMuted(muted: boolean) {
288      // Mute or unmute the native media player.
289    }
290
291    setPlaybackRate(playbackRate: number) {
292      // Set the playback rate of the native media player.
293    }
294
295    enterFullscreen() {
296      // Enter the full-screen mode.
297    }
298
299    exitFullscreen() {
300      // Exit the full-screen mode.
301    }
302  }
303  ```
304
305### Notifying the Status Information of Native Media Player to the ArkWeb Kernel
306
307The ArkWeb kernel need to update the status information (such as the video width and height, playback time, and cache time) of the native player to the web page. Therefore, you need to notify the ArkWeb kernel of the status information of the native player.
308
309Through the [onCreateNativeMediaPlayer](../reference/apis-arkweb/js-apis-webview.md#oncreatenativemediaplayer12) API, the Ark Web kernel passes a [NativeMediaPlayerHandler](../reference/apis-arkweb/js-apis-webview.md#nativemediaplayerhandler12) object to the application. You need to use this object to notify the ArkWeb kernel of the latest status information of the native media player.
310
311  ```ts
312  // xxx.ets
313  import { webview } from '@kit.ArkWeb';
314
315  class ActualNativeMediaPlayerListener {
316    handler: webview.NativeMediaPlayerHandler;
317
318    constructor(handler: webview.NativeMediaPlayerHandler) {
319      this.handler = handler;
320    }
321
322    onPlaying() {
323      // The native media player starts playback.
324      this.handler.handleStatusChanged(webview.PlaybackStatus.PLAYING);
325    }
326    onPaused() {
327      // The native media player pauses the playback.
328      this.handler.handleStatusChanged(webview.PlaybackStatus.PAUSED);
329    }
330    onSeeking() {
331      // The native media player starts to seek the target time point.
332      this.handler.handleSeeking();
333    }
334    onSeekDone() {
335      // The target time point is sought.
336      this.handler.handleSeekFinished();
337    }
338    onEnded() {
339      // The playback on the native media player is ended.
340      this.handler.handleEnded();
341    }
342    onVolumeChanged() {
343      // Obtain the volume of the native media player.
344      let volume: number = getVolume();
345      this.handler.handleVolumeChanged(volume);
346    }
347    onCurrentPlayingTimeUpdate() {
348      // Update the playback time.
349      let currentTime: number = getCurrentPlayingTime();
350      // Convert the time unit to second.
351      let currentTimeInSeconds = convertToSeconds(currentTime);
352      this.handler.handleTimeUpdate(currentTimeInSeconds);
353    }
354    onBufferedChanged() {
355      // The buffer is changed.
356      // Obtain the cache duration of the native media player.
357      let bufferedEndTime: number = getCurrentBufferedTime();
358      // Convert the time unit to second.
359      let bufferedEndTimeInSeconds = convertToSeconds(bufferedEndTime);
360      this.handler.handleBufferedEndTimeChanged(bufferedEndTimeInSeconds);
361
362      // Check the cache state.
363      // If the cache state changes, notify the ArkWeb engine of the cache state.
364      let lastReadyState: webview.ReadyState = getLastReadyState();
365      let currentReadyState: webview.ReadyState = getCurrentReadyState();
366      if (lastReadyState != currentReadyState) {
367        this.handler.handleReadyStateChanged(currentReadyState);
368      }
369    }
370    onEnterFullscreen() {
371      // The native media player enters the full-screen mode.
372      let isFullscreen: boolean = true;
373      this.handler.handleFullscreenChanged(isFullscreen);
374    }
375    onExitFullscreen() {
376      // The native media player exits the full-screen mode.
377      let isFullscreen: boolean = false;
378      this.handler.handleFullscreenChanged(isFullscreen);
379    }
380    onUpdateVideoSize(width: number, height: number) {
381      // Notify the ArkWeb kernel of the video width and height parsed by the native player.
382      this.handler.handleVideoSizeChanged(width, height);
383    }
384    onDurationChanged(duration: number) {
385      // Notify the ArkWeb kernel of the new media duration parsed by the native player.
386      this.handler.handleDurationChanged(duration);
387    }
388    onError(error: webview.MediaError, errorMessage: string) {
389      // Notify the ArkWeb kernel that an error occurs in the native player.
390      this.handler.handleError(error, errorMessage);
391    }
392    onNetworkStateChanged(state: webview.NetworkState) {
393      // Notify the ArkWeb kernel that the network state of the native player changes.
394      this.handler.handleNetworkStateChanged(state);
395    }
396    onPlaybackRateChanged(playbackRate: number) {
397      // Notify the ArkWeb kernel that the playback rate of the native player changes.
398      this.handler.handlePlaybackRateChanged(playbackRate);
399    }
400    onMutedChanged(muted: boolean) {
401      // Notify the ArkWeb kernel that the native player is muted.
402      this.handler.handleMutedChanged(muted);
403    }
404
405    // ...Listen for other status of the native player.
406  }
407  @Entry
408  @Component
409  struct WebComponent {
410    controller: webview.WebviewController = new webview.WebviewController();
411    @State show_native_media_player: boolean = false;
412
413    build() {
414      Column() {
415        Web({ src: 'www.example.com', controller: this.controller })
416          .enableNativeMediaPlayer({enable: true, shouldOverlay: false})
417          .onPageBegin((event) => {
418            this.controller.onCreateNativeMediaPlayer((handler: webview.NativeMediaPlayerHandler, mediaInfo: webview.MediaInfo) => {
419              // Take over the web media.
420
421              // Create a native media player instance.
422              // let nativePlayer: NativeMediaPlayerImpl = new NativeMediaPlayerImpl(handler, mediaInfo);
423
424              // Create a listener for the native media player state.
425              let nativeMediaPlayerListener: ActualNativeMediaPlayerListener = new ActualNativeMediaPlayerListener(handler);
426              // Listen for the native media player state.
427              // nativePlayer.setListener(nativeMediaPlayerListener);
428
429              // Return the native media player instance to the ArkWeb kernel.
430              return null;
431            });
432          })
433      }
434    }
435  }
436
437  // stub
438  function getVolume() {
439    return 1;
440  }
441  function getCurrentPlayingTime() {
442    return 1;
443  }
444  function getCurrentBufferedTime() {
445    return 1;
446  }
447  function convertToSeconds(input: number) {
448    return input;
449  }
450  function getLastReadyState() {
451    return webview.ReadyState.HAVE_NOTHING;
452  }
453  function getCurrentReadyState() {
454    return webview.ReadyState.HAVE_NOTHING;
455  }
456  ```
457
458
459## Sample Code
460
461- Add the following permissions to **module.json5** before using it:
462
463  ```ts
464  "ohos.permission.INTERNET"
465  ```
466
467- Example of saving **UIContext** during application startup on the application side:
468
469  ```ts
470  // xxxAbility.ets
471
472  import { UIAbility } from '@kit.AbilityKit';
473  import { window } from '@kit.ArkUI';
474
475  export default class EntryAbility extends UIAbility {
476    onWindowStageCreate(windowStage: window.WindowStage): void {
477      windowStage.loadContent('pages/Index', (err, data) => {
478        if (err.code) {
479          return;
480        }
481        // Save UIContext, which will be used in subsequent rendering and drawing at the same layer.
482        AppStorage.setOrCreate<UIContext>("UIContext", windowStage.getMainWindowSync().getUIContext());
483      });
484    }
485
486    // ...Other APIs that need to be overridden...
487  }
488  ```
489
490- Example of web media playback takeover:
491
492  ```ts
493  // Index.ets
494
495  import { webview } from '@kit.ArkWeb';
496  import { BuilderNode, FrameNode, NodeController, NodeRenderType, UIContext } from '@kit.ArkUI';
497  import { AVPlayerDemo, AVPlayerListener } from './PlayerDemo';
498
499  // Implement the webview.NativeMediaPlayerBridge API.
500  // The ArkWeb kernel calls the APIs to control playback on NativeMediaPlayer.
501  class NativeMediaPlayerImpl implements webview.NativeMediaPlayerBridge {
502    private surfaceId: string;
503    mediaSource: string;
504    private mediaHandler: webview.NativeMediaPlayerHandler;
505    nativePlayerInfo: NativePlayerInfo;
506    nativePlayer: AVPlayerDemo;
507    httpHeaders: Record<string, string>;
508
509    constructor(nativePlayerInfo: NativePlayerInfo, handler: webview.NativeMediaPlayerHandler, mediaInfo: webview.MediaInfo) {
510      console.log(`NativeMediaPlayerImpl.constructor, surface_id[${mediaInfo.surfaceInfo.id}]`);
511      this.nativePlayerInfo = nativePlayerInfo;
512      this.mediaHandler = handler;
513      this.surfaceId = mediaInfo.surfaceInfo.id;
514      this.mediaSource = mediaInfo.mediaSrcList.find((item)=>{item.source.indexOf('.mp4') > 0})?.source
515        || mediaInfo.mediaSrcList[0].source;
516      this.httpHeaders = mediaInfo.headers;
517      this.nativePlayer = new AVPlayerDemo();
518
519      // Use the rendering function at the same layer to draw the video and its playback control components to the web page.
520      this.nativePlayerInfo.node_controller = new MyNodeController(
521        this.nativePlayerInfo, this.surfaceId, this.mediaHandler, this, NodeRenderType.RENDER_TYPE_TEXTURE);
522      this.nativePlayerInfo.node_controller.build();
523      this.nativePlayerInfo.show_native_media_player = true;
524
525      console.log(`NativeMediaPlayerImpl.mediaSource: ${this.mediaSource}, headers: ${JSON.stringify(this.httpHeaders)}`);
526    }
527
528    updateRect(x: number, y: number, width: number, height: number): void {
529      let width_in_vp = px2vp(width);
530      let height_in_vp = px2vp(height);
531      console.log(`updateRect(${x}, ${y}, ${width}, ${height}), vp:{${width_in_vp}, ${height_in_vp}}`);
532
533      this.nativePlayerInfo.updateNativePlayerRect(x, y, width, height);
534    }
535
536    play() {
537      console.log('NativeMediaPlayerImpl.play');
538      this.nativePlayer.play();
539    }
540    pause() {
541      console.log('NativeMediaPlayerImpl.pause');
542      this.nativePlayer.pause();
543    }
544    seek(targetTime: number) {
545      console.log(`NativeMediaPlayerImpl.seek(${targetTime})`);
546      this.nativePlayer.seek(targetTime);
547    }
548    setVolume(volume: number) {
549      console.log(`NativeMediaPlayerImpl.setVolume(${volume})`);
550      this.nativePlayer.setVolume(volume);
551    }
552    setMuted(muted: boolean) {
553      console.log(`NativeMediaPlayerImpl.setMuted(${muted})`);
554    }
555    setPlaybackRate(playbackRate: number) {
556      console.log(`NativeMediaPlayerImpl.setPlaybackRate(${playbackRate})`);
557      this.nativePlayer.setPlaybackRate(playbackRate);
558    }
559    release() {
560      console.log('NativeMediaPlayerImpl.release');
561      this.nativePlayer?.release();
562      this.nativePlayerInfo.show_native_media_player = false;
563      this.nativePlayerInfo.node_width = 300;
564      this.nativePlayerInfo.node_height = 150;
565      this.nativePlayerInfo.destroyed();
566    }
567    enterFullscreen() {
568      console.log('NativeMediaPlayerImpl.enterFullscreen');
569    }
570    exitFullscreen() {
571      console.log('NativeMediaPlayerImpl.exitFullscreen');
572    }
573  }
574
575  // Listen for the NativeMediaPlayer status and report the status to the ArkWeb kernel using webview.NativeMediaPlayerHandler.
576  class AVPlayerListenerImpl implements AVPlayerListener {
577    handler: webview.NativeMediaPlayerHandler;
578    component: NativePlayerComponent;
579
580    constructor(handler: webview.NativeMediaPlayerHandler, component: NativePlayerComponent) {
581      this.handler = handler;
582      this.component = component;
583    }
584    onPlaying() {
585      console.log('AVPlayerListenerImpl.onPlaying');
586      this.handler.handleStatusChanged(webview.PlaybackStatus.PLAYING);
587    }
588    onPaused() {
589      console.log('AVPlayerListenerImpl.onPaused');
590      this.handler.handleStatusChanged(webview.PlaybackStatus.PAUSED);
591    }
592    onDurationChanged(duration: number) {
593      console.log(`AVPlayerListenerImpl.onDurationChanged(${duration})`);
594      this.handler.handleDurationChanged(duration);
595    }
596    onBufferedTimeChanged(buffered: number) {
597      console.log(`AVPlayerListenerImpl.onBufferedTimeChanged(${buffered})`);
598      this.handler.handleBufferedEndTimeChanged(buffered);
599    }
600    onTimeUpdate(time: number) {
601      this.handler.handleTimeUpdate(time);
602    }
603    onEnded() {
604      console.log('AVPlayerListenerImpl.onEnded');
605      this.handler.handleEnded();
606    }
607    onError() {
608      console.log('AVPlayerListenerImpl.onError');
609      this.component.has_error = true;
610      setTimeout(()=>{
611        this.handler.handleError(1, "Oops!");
612      }, 200);
613    }
614    onVideoSizeChanged(width: number, height: number) {
615      console.log(`AVPlayerListenerImpl.onVideoSizeChanged(${width}, ${height})`);
616      this.handler.handleVideoSizeChanged(width, height);
617      this.component.onSizeChanged(width, height);
618    }
619    onDestroyed(): void {
620      console.log('AVPlayerListenerImpl.onDestroyed');
621    }
622  }
623
624  interface ComponentParams {
625    text: string;
626    text2: string;
627    playerInfo: NativePlayerInfo;
628    handler: webview.NativeMediaPlayerHandler;
629    player: NativeMediaPlayerImpl;
630  }
631
632  // Define the player components.
633  @Component
634  struct NativePlayerComponent {
635    params?: ComponentParams;
636    @State bgColor: Color = Color.Red;
637    mXComponentController: XComponentController = new XComponentController();
638
639    videoController: VideoController = new VideoController();
640    offset_x: number = 0;
641    offset_y: number = 0;
642    @State video_width_percent: number = 100;
643    @State video_height_percent: number = 100;
644    view_width: number = 0;
645    view_height: number = 0;
646    video_width: number = 0;
647    video_height: number = 0;
648
649    fullscreen: boolean = false;
650    @State has_error: boolean = false;
651
652    onSizeChanged(width: number, height: number) {
653      this.video_width = width;
654      this.video_height = height;
655      let scale: number = this.view_width / width;
656      let scaled_video_height: number = scale * height;
657      this.video_height_percent = scaled_video_height / this.view_height * 100;
658      console.log(`NativePlayerComponent.onSizeChanged(${width},${height}), video_height_percent[${this.video_height_percent }]`);
659    }
660
661    build() {
662      Column() {
663        Stack() {
664          XComponent({ id: 'video_player_id', type: XComponentType.SURFACE, controller: this.mXComponentController })
665            .width(this.video_width_percent + '%')
666            .height(this.video_height_percent + '%')
667            .onLoad(()=>{
668              if (!this.params) {
669                console.log('this.params is null');
670                return;
671              }
672              console.log('NativePlayerComponent.onLoad, params[' + this.params
673                + '], text[' + this.params.text + '], text2[' + this.params.text2
674                + '], web_tab[' + this.params.playerInfo + '], handler[' + this.params.handler + ']');
675              this.params.player.nativePlayer.setSurfaceID(this.mXComponentController.getXComponentSurfaceId());
676
677              this.params.player.nativePlayer.avPlayerLiveDemo({
678                url: this.params.player.mediaSource,
679                listener: new AVPlayerListenerImpl(this.params.handler, this),
680                httpHeaders: this.params.player.httpHeaders,
681              });
682            })
683          Column() {
684            Row() {
685              Button(this.params?.text)
686                .height(50)
687                .border({ width: 2, color: Color.Red })
688                .backgroundColor(this.bgColor)
689                .onClick(()=>{
690                  console.log(`NativePlayerComponent.Button[${this.params?.text}] is clicked`);
691                  this.params?.player.nativePlayer?.play();
692                })
693                .onTouch((event: TouchEvent) => {
694                  event.stopPropagation();
695                })
696              Button(this.params?.text2)
697                .height(50)
698                .border({ width: 2, color: Color.Red })
699                .onClick(()=>{
700                  console.log(`NativePlayerComponent.Button[${this.params?.text2}] is clicked`);
701                  this.params?.player.nativePlayer?.pause();
702                })
703                .onTouch((event: TouchEvent) => {
704                  event.stopPropagation();
705                })
706            }
707            .width('100%')
708            .justifyContent(FlexAlign.SpaceEvenly)
709          }
710          if (this.has_error) {
711            Column() {
712              Text('Error')
713                .fontSize(30)
714            }
715            .backgroundColor('#eb5555')
716            .width('100%')
717            .height('100%')
718            .justifyContent(FlexAlign.Center)
719          }
720        }
721      }
722      .width('100%')
723      .height('100%')
724      .onAreaChange((oldValue: Area, newValue: Area) => {
725        console.log(`NativePlayerComponent.onAreaChange(${JSON.stringify(oldValue)}, ${JSON.stringify(newValue)})`);
726        this.view_width = new Number(newValue.width).valueOf();
727        this.view_height = new Number(newValue.height).valueOf();
728        this.onSizeChanged(this.video_width, this.video_height);
729      })
730    }
731  }
732
733  @Builder
734  function NativePlayerComponentBuilder(params: ComponentParams) {
735    NativePlayerComponent({ params: params })
736      .backgroundColor(Color.Green)
737      .border({ width: 1, color: Color.Brown })
738      .width('100%')
739      .height('100%')
740  }
741
742  // Use NodeController to dynamically create a player component and draw the component content on the surface specified by surfaceId.
743  class MyNodeController extends NodeController {
744    private rootNode: BuilderNode<[ComponentParams]> | undefined;
745    playerInfo: NativePlayerInfo;
746    listener: webview.NativeMediaPlayerHandler;
747    player: NativeMediaPlayerImpl;
748
749    constructor(playerInfo: NativePlayerInfo,
750                surfaceId: string,
751                listener: webview.NativeMediaPlayerHandler,
752                player: NativeMediaPlayerImpl,
753                renderType: NodeRenderType) {
754      super();
755      this.playerInfo = playerInfo;
756      this.listener = listener;
757      this.player = player;
758      let uiContext = AppStorage.get<UIContext>("UIContext");
759      this.rootNode = new BuilderNode(uiContext as UIContext, { surfaceId: surfaceId, type: renderType });
760      console.log(`MyNodeController, rootNode[${this.rootNode}], playerInfo[${playerInfo}], listener[${listener}], surfaceId[${surfaceId}]`);
761    }
762
763    makeNode(): FrameNode | null {
764      if (this.rootNode) {
765        return this.rootNode.getFrameNode() as FrameNode;
766      }
767      return null;
768    }
769
770    build() {
771      let params: ComponentParams = {
772        "text": "play",
773        "text2": "pause",
774        playerInfo: this.playerInfo,
775        handler: this.listener,
776        player: this.player
777      };
778      if (this.rootNode) {
779        this.rootNode.build(wrapBuilder(NativePlayerComponentBuilder), params);
780      }
781    }
782
783    postTouchEvent(event: TouchEvent) {
784      return this.rootNode?.postTouchEvent(event);
785    }
786  }
787
788  class Rect {
789    x: number = 0;
790    y: number = 0;
791    width: number = 0;
792    height: number = 0;
793
794    static toNodeRect(rectInPx: webview.RectEvent) : Rect {
795      let rect = new Rect();
796      rect.x = px2vp(rectInPx.x);
797      rect.y = px2vp(rectInPx.x);
798      rect.width = px2vp(rectInPx.width);
799      rect.height = px2vp(rectInPx.height);
800      return rect;
801    }
802  }
803
804  @Observed
805  class NativePlayerInfo {
806    public web: WebComponent;
807    public embed_id: string;
808    public player: webview.NativeMediaPlayerBridge;
809    public node_controller?: MyNodeController;
810    public show_native_media_player: boolean = false;
811    public node_pos_x: number;
812    public node_pos_y: number;
813    public node_width: number;
814    public node_height: number;
815
816    playerComponent?: NativeMediaPlayerComponent;
817
818    constructor(web: WebComponent, handler: webview.NativeMediaPlayerHandler, videoInfo: webview.MediaInfo) {
819      this.web = web;
820      this.embed_id = videoInfo.embedID;
821
822      let node_rect = Rect.toNodeRect(videoInfo.surfaceInfo.rect);
823      this.node_pos_x = node_rect.x;
824      this.node_pos_y = node_rect.y;
825      this.node_width = node_rect.width;
826      this.node_height = node_rect.height;
827
828      this.player = new NativeMediaPlayerImpl(this, handler, videoInfo);
829    }
830
831    updateNativePlayerRect(x: number, y: number, width: number, height: number) {
832      this.playerComponent?.updateNativePlayerRect(x, y, width, height);
833    }
834
835    destroyed() {
836      let info_list = this.web.native_player_info_list;
837      console.log(`NativePlayerInfo[${this.embed_id}] destroyed, list.size[${info_list.length}]`);
838      this.web.native_player_info_list = info_list.filter((item) => item.embed_id != this.embed_id);
839      console.log(`NativePlayerInfo after destroyed, new_list.size[${this.web.native_player_info_list.length}]`);
840    }
841  }
842
843  @Component
844  struct NativeMediaPlayerComponent {
845    @ObjectLink playerInfo: NativePlayerInfo;
846
847    aboutToAppear() {
848      this.playerInfo.playerComponent = this;
849    }
850
851    build() {
852      NodeContainer(this.playerInfo.node_controller)
853        .width(this.playerInfo.node_width)
854        .height(this.playerInfo.node_height)
855        .offset({x: this.playerInfo.node_pos_x, y: this.playerInfo.node_pos_y})
856        .backgroundColor(Color.Transparent)
857        .border({ width: 2, color: Color.Orange })
858        .onAreaChange((oldValue, newValue) => {
859          console.log(`NodeContainer[${this.playerInfo.embed_id}].onAreaChange([${oldValue.width} x ${oldValue.height}]->[${newValue.width} x ${newValue.height}]`);
860        })
861    }
862
863    updateNativePlayerRect(x: number, y: number, width: number, height: number) {
864      let node_rect = Rect.toNodeRect({x, y, width, height});
865      this.playerInfo.node_pos_x = node_rect.x;
866      this.playerInfo.node_pos_y = node_rect.y;
867      this.playerInfo.node_width = node_rect.width;
868      this.playerInfo.node_height = node_rect.height;
869    }
870  }
871
872  @Entry
873  @Component
874  struct WebComponent {
875    controller: WebviewController = new webview.WebviewController();
876    page_url: Resource = $rawfile('main.html');
877
878    @State native_player_info_list: NativePlayerInfo[] = [];
879
880    area?: Area;
881
882    build() {
883      Column() {
884        Stack({alignContent: Alignment.TopStart}) {
885          ForEach(this.native_player_info_list, (item: NativePlayerInfo) => {
886            if (item.show_native_media_player) {
887              NativeMediaPlayerComponent({ playerInfo: item })
888            }
889          }, (item: NativePlayerInfo) => {
890            return item.embed_id;
891          })
892          Web({ src: this.page_url, controller: this.controller })
893            .enableNativeMediaPlayer({ enable: true, shouldOverlay: true })
894            .onPageBegin(() => {
895              this.controller.onCreateNativeMediaPlayer((handler: webview.NativeMediaPlayerHandler, mediaInfo: webview.MediaInfo) => {
896                console.log('onCreateNativeMediaPlayer(' + JSON.stringify(mediaInfo) + ')');
897                let nativePlayerInfo = new NativePlayerInfo(this, handler, mediaInfo);
898                this.native_player_info_list.push(nativePlayerInfo);
899                return nativePlayerInfo.player;
900              });
901            })
902            .onNativeEmbedGestureEvent((event)=>{
903              if (!event.touchEvent || !event.embedId) {
904                event.result?.setGestureEventResult(false);
905                return;
906              }
907              console.log(`WebComponent.onNativeEmbedGestureEvent, embedId[${event.embedId}]`);
908              let native_player_info = this.getNativePlayerInfoByEmbedId(event.embedId);
909              if (!native_player_info) {
910                console.log(`WebComponent.onNativeEmbedGestureEvent, embedId[${event.embedId}], no native_player_info`);
911                event.result?.setGestureEventResult(false);
912                return;
913              }
914              if (!native_player_info.node_controller) {
915                console.log(`WebComponent.onNativeEmbedGestureEvent, embedId[${event.embedId}], no node_controller`);
916                event.result?.setGestureEventResult(false);
917                return;
918              }
919              let ret = native_player_info.node_controller.postTouchEvent(event.touchEvent);
920              console.log(`WebComponent.postTouchEvent, ret[${ret}], touchEvent[${JSON.stringify(event.touchEvent)}]`);
921              event.result?.setGestureEventResult(ret);
922            })
923            .width('100%')
924            .height('100%')
925            .onAreaChange((oldValue: Area, newValue: Area) => {
926              oldValue;
927              this.area = newValue;
928            })
929        }
930      }
931    }
932
933    getNativePlayerInfoByEmbedId(embedId: string) : NativePlayerInfo | undefined {
934      return this.native_player_info_list.find((item)=> item.embed_id == embedId);
935    }
936  }
937  ```
938
939- Example of using video playback on the application side:
940
941  ```ts
942  import { media } from '@kit.MediaKit';
943  import { BusinessError } from '@kit.BasicServicesKit';
944
945  export interface AVPlayerListener {
946    onPlaying() : void;
947    onPaused() : void;
948    onDurationChanged(duration: number) : void;
949    onBufferedTimeChanged(buffered: number) : void;
950    onTimeUpdate(time: number) : void;
951    onEnded() : void;
952    onError() : void;
953    onVideoSizeChanged(width: number, height: number): void;
954    onDestroyed(): void;
955  }
956
957  interface PlayerParam {
958    url: string;
959    listener?: AVPlayerListener;
960    httpHeaders?: Record<string, string>;
961  }
962
963  interface PlayCommand {
964    func: Function;
965    name?: string;
966  }
967
968  interface CheckPlayCommandResult {
969    ignore: boolean;
970    index_to_remove: number;
971  }
972
973  export class AVPlayerDemo {
974    private surfaceID: string = ''; // The surfaceID parameter specifies the window used to display the video. Its value is obtained through the XComponent.
975
976    avPlayer?: media.AVPlayer;
977    prepared: boolean = false;
978
979    commands: PlayCommand[] = [];
980
981    setSurfaceID(surface_id: string) {
982      console.log(`AVPlayerDemo.setSurfaceID : ${surface_id}`);
983      this.surfaceID = surface_id;
984    }
985    // Set AVPlayer callback functions.
986    setAVPlayerCallback(avPlayer: media.AVPlayer, listener?: AVPlayerListener) {
987      // Callback function for the seek operation.
988      avPlayer.on('seekDone', (seekDoneTime: number) => {
989        console.info(`AVPlayer seek succeeded, seek time is ${seekDoneTime}`);
990      });
991      // Callback function for errors. If an error occurs during the operation on the AVPlayer, reset() is called to reset the AVPlayer.
992      avPlayer.on('error', (err: BusinessError) => {
993        console.error(`Invoke avPlayer failed, code is ${err.code}, message is ${err.message}`);
994        listener?.onError();
995        avPlayer.reset(); // Call reset() to reset the AVPlayer, which enters the idle state.
996      });
997      // Callback for state changes.
998      avPlayer.on('stateChange', async (state: string, reason: media.StateChangeReason) => {
999        switch (state) {
1000          case 'idle': // This state is reported upon a successful callback of reset().
1001            console.info('AVPlayer state idle called.');
1002            avPlayer.release(); // Call release() to release the instance.
1003            break;
1004          case 'initialized': // This state is reported when the AVPlayer sets the playback source.
1005            console.info('AVPlayer state initialized called.');
1006            avPlayer.surfaceId = this.surfaceID; // Set the window to display the video. This setting is not required when a pure audio asset is to be played.
1007            avPlayer.prepare();
1008            break;
1009          case 'prepared': // This state is reported upon a successful callback of prepare().
1010            console.info('AVPlayer state prepared called.');
1011            this.prepared = true;
1012            this.schedule();
1013            break;
1014          case 'playing': // This state is reported upon a successful callback of play().
1015            console.info('AVPlayer state playing called.');
1016            listener?.onPlaying();
1017            break;
1018          case 'paused': // This state is reported upon a successful callback of pause().
1019            console.info('AVPlayer state paused called.');
1020            listener?.onPaused();
1021            break;
1022          case 'completed': // This state is reported upon the completion of the playback.
1023            console.info('AVPlayer state completed called.');
1024            avPlayer.stop(); // Call stop() to stop the playback.
1025            break;
1026          case 'stopped': // This state is reported upon a successful callback of stop().
1027            console.info('AVPlayer state stopped called.');
1028            listener?.onEnded();
1029            break;
1030          case 'released':
1031            this.prepared = false;
1032            listener?.onDestroyed();
1033            console.info('AVPlayer state released called.');
1034            break;
1035          default:
1036            console.info('AVPlayer state unknown called.');
1037            break;
1038        }
1039      });
1040      avPlayer.on('durationUpdate', (duration: number) => {
1041        console.info(`AVPlayer state durationUpdate success,new duration is :${duration}`);
1042        listener?.onDurationChanged(duration/1000);
1043      });
1044      avPlayer.on('timeUpdate', (time:number) => {
1045        listener?.onTimeUpdate(time/1000);
1046      });
1047      avPlayer.on('bufferingUpdate', (infoType: media.BufferingInfoType, value: number) => {
1048        console.info(`AVPlayer state bufferingUpdate success,and infoType value is:${infoType}, value is : ${value}`);
1049        if (infoType == media.BufferingInfoType.BUFFERING_PERCENT) {
1050        }
1051        listener?.onBufferedTimeChanged(value);
1052      })
1053      avPlayer.on('videoSizeChange', (width: number, height: number) => {
1054        console.info(`AVPlayer state videoSizeChange success,and width is:${width}, height is : ${height}`);
1055        listener?.onVideoSizeChanged(width, height);
1056      })
1057    }
1058
1059    // The following demo shows how to play live streams by setting the network address through the URL.
1060    async avPlayerLiveDemo(playerParam: PlayerParam) {
1061      // Create an AVPlayer instance.
1062      this.avPlayer = await media.createAVPlayer();
1063      // Create a callback for state changes.
1064      this.setAVPlayerCallback(this.avPlayer, playerParam.listener);
1065
1066      let mediaSource: media.MediaSource = media.createMediaSourceWithUrl(playerParam.url, playerParam.httpHeaders);
1067      let strategy: media.PlaybackStrategy = {
1068        preferredWidth: 100,
1069        preferredHeight: 100,
1070        preferredBufferDuration: 100,
1071        preferredHdr: false
1072      };
1073      this.avPlayer.setMediaSource(mediaSource, strategy);
1074      console.log(`AVPlayer url:[${playerParam.url}]`);
1075    }
1076
1077    schedule() {
1078      if (!this.avPlayer) {
1079        return;
1080      }
1081      if (!this.prepared) {
1082        return;
1083      }
1084      if (this.commands.length > 0) {
1085        let command = this.commands.shift();
1086        if (command) {
1087          command.func();
1088        }
1089        if (this.commands.length > 0) {
1090          setTimeout(() => {
1091            this.schedule();
1092          });
1093        }
1094      }
1095    }
1096
1097    private checkCommand(selfName: string, oppositeName: string) {
1098      let index_to_remove = -1;
1099      let ignore_this_action = false;
1100      let index = this.commands.length - 1;
1101      while (index >= 0) {
1102        if (this.commands[index].name == selfName) {
1103          ignore_this_action = true;
1104          break;
1105        }
1106        if (this.commands[index].name == oppositeName) {
1107          index_to_remove = index;
1108          break;
1109        }
1110        index--;
1111      }
1112
1113      let result : CheckPlayCommandResult = {
1114        ignore: ignore_this_action,
1115        index_to_remove: index_to_remove,
1116      };
1117      return result;
1118    }
1119
1120    play() {
1121      let commandName = 'play';
1122      let checkResult = this.checkCommand(commandName, 'pause');
1123      if (checkResult.ignore) {
1124        console.log(`AVPlayer ${commandName} ignored.`);
1125        this.schedule();
1126        return;
1127      }
1128      if (checkResult.index_to_remove >= 0) {
1129        let removedCommand = this.commands.splice(checkResult.index_to_remove, 1);
1130        console.log(`AVPlayer ${JSON.stringify(removedCommand)} removed.`);
1131        return;
1132      }
1133      this.commands.push({ func: ()=>{
1134        console.info('AVPlayer.play()');
1135        this.avPlayer?.play();
1136      }, name: commandName});
1137      this.schedule();
1138    }
1139    pause() {
1140      let commandName = 'pause';
1141      let checkResult = this.checkCommand(commandName, 'play');
1142      console.log(`checkResult:${JSON.stringify(checkResult)}`);
1143      if (checkResult.ignore) {
1144        console.log(`AVPlayer ${commandName} ignored.`);
1145        this.schedule();
1146        return;
1147      }
1148      if (checkResult.index_to_remove >= 0) {
1149        let removedCommand = this.commands.splice(checkResult.index_to_remove, 1);
1150        console.log(`AVPlayer ${JSON.stringify(removedCommand)} removed.`);
1151        return;
1152      }
1153      this.commands.push({ func: ()=>{
1154        console.info('AVPlayer.pause()');
1155        this.avPlayer?.pause();
1156      }, name: commandName});
1157      this.schedule();
1158    }
1159    release() {
1160      this.commands.push({ func: ()=>{
1161        console.info('AVPlayer.release()');
1162        this.avPlayer?.release();
1163      }});
1164      this.schedule();
1165    }
1166    seek(time: number) {
1167      this.commands.push({ func: ()=>{
1168        console.info(`AVPlayer.seek(${time})`);
1169        this.avPlayer?.seek(time * 1000);
1170      }});
1171      this.schedule();
1172    }
1173    setVolume(volume: number) {
1174      this.commands.push({ func: ()=>{
1175        console.info(`AVPlayer.setVolume(${volume})`);
1176        this.avPlayer?.setVolume(volume);
1177      }});
1178      this.schedule();
1179    }
1180    setPlaybackRate(playbackRate: number) {
1181      let speed = media.PlaybackSpeed.SPEED_FORWARD_1_00_X;
1182      let delta = 0.05;
1183      playbackRate += delta;
1184      if (playbackRate < 1) {
1185        speed = media.PlaybackSpeed.SPEED_FORWARD_0_75_X;
1186      } else if (playbackRate < 1.25) {
1187        speed = media.PlaybackSpeed.SPEED_FORWARD_1_00_X;
1188      } else if (playbackRate < 1.5) {
1189        speed = media.PlaybackSpeed.SPEED_FORWARD_1_25_X;
1190      } else if (playbackRate < 2) {
1191        speed = media.PlaybackSpeed.SPEED_FORWARD_1_75_X;
1192      } else {
1193        speed = media.PlaybackSpeed.SPEED_FORWARD_2_00_X;
1194      }
1195      this.commands.push({ func: ()=>{
1196        console.info(`AVPlayer.setSpeed(${speed})`);
1197        this.avPlayer?.setSpeed(speed);
1198      }});
1199      this.schedule();
1200    }
1201  }
1202  ```
1203
1204- Example of a frontend page:
1205
1206  ```html
1207  <html>
1208  <head>
1209      <title>Video Hosting Test html</title>
1210      <meta name="viewport" content="width=device-width">
1211  </head>
1212  <body>
1213  <div>
1214      <!--Replace the URL with the actual URL of the video source.-->
1215      <video src='https://xxx.xxx/demo.mp4' style='width: 100%'></video>
1216  </div>
1217  </body>
1218  </html>
1219  ```
1220