Remote Player

How to stream video directly to the cloud connector

The Senza platform has two modes:

  1. It can use a web browser in the cloud to run HTML5 apps, which can play video using a web player. The complete web user interface, including the video, is sent as a single video stream to the cloud connector.
  2. Your app can tell the cloud connector to stream video directly from the source. In this case the web browser can switch to background mode, saving compute resources.

From your app's point of view, the "local player" is the HTML5 video element that your app uses to play video, which may involve a third-party library such as the Shaka player. The "remote player" is the software in the cloud connector device that can play video streams directly.

You can use the remotePlayer class described on this page to control the playback of video on the device. The most commonly used functions are very similar to what you're used to when playing HTML5 video:

  • Call remotePlayer.load(url) to set the source URL of the video.
  • By default the video will start playing from the beginning. To synchronize the timecode from the local player to the remote player, call remotePlayer.currentTime = localPlayer.currentTime.
  • Call remotePlayer.play() to start playing the video. This will switch to background mode.

When the viewer presses a button on the remote, you'll want switch back to foreground mode by calling lifecycle.moveToForeground. See the App Lifecycle page for more information on switching between foreground and background modes.

Examples

The Playing Video tutorial walks you through the process of writing an app where you can switch between playing video in the browser (with user interface elements layered on top) and playing video in the remote player.

In this tutorial, all of the calls to the remote player are gathered together into a VideoManager class, which manages both the local and remote players with a unified interface. If you drop this script directly into your app, you only need to add a few additional lines of code to integrate with the remote player.

The Reference Integration provides a complete example of the calls you would need to make to fully integrate with the remote player. It summarizes the code from the tutorial above, including the videoManager class.

Modules

remotePlayerRemotePlayer

Classes

RemotePlayerError

Error object to be thrown on remotePlayer api failures. See error list

RemotePlayer

RemotePlayer a singleton class to communicate with remote player

Functions

writeLicenseResponse(statusCode, response)

Typedefs

Config : Object
LoadMode : Object
TextTrack : Object
AudioTrack : Object

remotePlayer ⇒ RemotePlayer

Returns: RemotePlayer - pointer to the RemotePlayer singleton
Example

import { init, remotePlayer } from "senza-sdk";
await init();
try {
    await remotePlayer.load("http://playable.url/file.mp4");
    remotePlayer.play();
} catch(err) {
    console.log("remotePlayer api failed with error code", err.code, err.msg);
}

RemotePlayerError

Error object to be thrown on remotePlayer api failures.
See error list

Kind: global class

RemotePlayer

RemotePlayer a singleton class to communicate with remote player

Kind: global class
Emits: event:timeupdate, event:tracksupdate, event:ended, event:error, event:onloadmodechange, seeking (Not implemented yet), seeked (Not implemented yet), loadedmetadata (Not implemented yet)

remotePlayer.currentTime

Getter/Setter for currentTime

Kind: instance property of RemotePlayer

remotePlayer.duration

For VOD asset, the duration of the asset

Kind: instance property of RemotePlayer
Read only: true

remotePlayer.textTrackVisibility

Whether the subtitles are visible or not.

Kind: instance property of RemotePlayer
Read only: true

remotePlayer.getConfiguration() ⇒ Config

get the player configuration.

Kind: instance method of RemotePlayer

remotePlayer.configure(props)

setting values for properties in the player configuration using an object.
If the config does not support a property this is a no-op.

Kind: instance method of RemotePlayer

ParamTypeDescription
propsObjectthe object with all the different properties to change

Example

remotePlayer.configure({ preferredAudioLanguage: 'en-US' })

remotePlayer.registerVideoElement(video)

Deprecated

In order to support a seamless switch between the video in the UI and ABR, the web application must
register the video element being used for the currently played video before calling the load and play apis.

Kind: instance method of RemotePlayer

ParamTypeDescription
videoobjectThe video element currently playing video in the web application

remotePlayer.attach(video)

In order to support a seamless switch between the video in the UI and ABR, the web application must
attach the video element being used for the currently played video before calling the load and play apis.

Kind: instance method of RemotePlayer

ParamTypeDescription
videoobjectThe video element currently playing video in the web application

remotePlayer.load(url, [position]) ⇒ Promise

Tell the remote player to load the given URL.

Kind: instance method of RemotePlayer
Throws:

ParamTypeDescription
urlstringurl to load
[position]numberstart position in seconds (if not provided, start from beginning (VOD) or current time (LTV))

remotePlayer.unload()

Deprecated

Kind: instance method of RemotePlayer

remotePlayer.play([moveToBackground])

Play loaded URL. Assuming load was called before.

Kind: instance method of RemotePlayer
Throws:

ParamTypeDefaultDescription
[moveToBackground]booleantrueif true, the UI will automatically move to background state if false, the UI will remain in foreground state, need to call lifecycle.moveToBackground() to move to background This is an experimental feature and may not work as expected. This is deprectaed now see Config.autoBackground

Example

remotePlayer.load("https://example.com/video.mp4", 0);
remotePlayer.configure({autoBackground:false})
remotePlayer.play();
lifecycle.moveToBackground();

// if Config.autoBackground=true
remotePlayer.load("https://example.com/video.mp4", 0);
remotePlayer.play();

remotePlayer.pause()

pauses the currently playing audio or video

Kind: instance method of RemotePlayer
Throws:

remotePlayer.getAssetUri() ⇒ string

Get the currently loaded uri, or empty string is player is not loaded.

Kind: instance method of RemotePlayer
Returns: string - loaded uri
Throws:

remotePlayer.getLoadMode() ⇒ LoadMode

Get the current load mode (either NOT_LOADED or LOADING or LOADED).

Kind: instance method of RemotePlayer
Returns: LoadMode - current load mode
Throws:

Example

import { remotePlayer } from "senza-sdk";
const loadMode = remotePlayer.getLoadMode();
if (loadMode === remotePlayer.LoadMode.LOADED) {
    console.log("remote player is currently loaded");
}

remotePlayer.getTextTracks() ⇒ Array.<TextTrack>

Get all the text (subtitles) tracks.
Text tracks are available only after playing content and returning to the UI.
When text tracks are updated, the @event tracksupdate if fired.
Auto Translation languages are returned only for VOD assets where there is at least one text track in the manifest to be used for translation.
The list of tracks is sorted ascending to two parts by the lang key in the following order:

  1. The actual text tracks which is available in the asset manifest and the selected track
  2. Auto-Translation tracks

Kind: instance method of RemotePlayer
Returns: Array.<TextTrack> - an array of TextTracks if available, otherwise - empty array

remotePlayer.getAudioTracks() ⇒ Array.<AudioTrack>

Get all the audio tracks.
Text tracks are available only after playing content and returning to the UI.
When audio tracks are updated, the @event tracksupdate is fired.

Kind: instance method of RemotePlayer
Returns: Array.<AudioTrack> - an array of objects if available, otherwise - empty array

remotePlayer.selectAudioTrack(audioTrackId)

Select a specific audio track.
Track id should come from a call to getAudioTracks.
Tf no tracks exist - this is a no-op.
Tf the id does not match the id of any of the existing tracks - this is a no-op.

Kind: instance method of RemotePlayer

ParamTypeDescription
audioTrackIdstringthe selected audio track id

remotePlayer.selectTextTrack(textTrackId)

Select a specific text (subtitle) track.
Track id should come from a call to getTextTracks.
If no tracks exist - this is a no-op.
If the id does not match the id of any of the existing tracks - this is a no-op.

Kind: instance method of RemotePlayer

ParamTypeDescription
textTrackIdstringthe selected text track id

remotePlayer.setTextTrackVisibility(visible)

Enable or disable the subtitles.
If the player is in an unloaded state, the request will be applied next time content is played.

Kind: instance method of RemotePlayer
Throws:

  • TypeError if visible is not a boolean variable
ParamTypeDescription
visiblebooleanwhether the subtitles are visible or not

remotePlayer.getPresentationStartTimeAsDate() ⇒ Date

Get the presentation start time as a date. This should only be called when the player has loaded a live stream.
If the player has not loaded a live stream, this will return null.

Kind: instance method of RemotePlayer
Returns: Date - presentation start time

"canPlay"

canPlay event will be rised when the remote player can start play the event

Kind: event emitted by RemotePlayer
Example

remotePlayer.addEventListener("canPlay", () => {
  console.info("remotePlayer canPlay");
  remotePlayer.play();
});

"timeupdate"

Kind: event emitted by RemotePlayer
Example

remotePlayer.addEventListener("timeupdate", () => {
  console.info("remotePlayer timeupdate", remotePlayer.currentTime);
  localPlayer.getMediaElement().currentTime = remotePlayer.currentTime || 0;
});

"tracksupdate"

Kind: event emitted by RemotePlayer
Example

remotePlayer.addEventListener("tracksupdate", () => {
  console.info("remotePlayer tracksupdate", remotePlayer.getAudioTracks(), remotePlayer.getTextTracks());
});

"ended"

Kind: event emitted by RemotePlayer
Example

remotePlayer.addEventListener("ended", () => {
  console.info("remotePlayer ended");
});

"error"

Kind: event emitted by RemotePlayer
See: Possible error codes:

CodeDomainDescription
23Playerload() failed due to remote player initialization error
98Playerload() failed due to remote player failure to send message to the client
99Playerload() failed due to remote player reporting invalid message
1000Encrypted contentFailed to create or initialise the CDM
1001Encrypted contentFailed to create a CDM session
1002Encrypted contentCDM failed to generate a license request
1003Encrypted contentThe CDM rejected the license server response
1004Encrypted contentThe CDM rejected the license server certificate
1005Encrypted contentAll keys in the license have expired
1006Encrypted contentOutput device is incompatible with the license requirements (HDCP)
1007Encrypted contentThe device has been revoked
1008Encrypted contentThe device secrets aren't available
1009Encrypted contentKeys are loaded but the KID requested by playback isn't found. The app has likely issued a license for the wrong content or there is a mismatch between the KIDs in the license and the data plane
1010Encrypted contentThe CDM failed to provision, therefore it is not possible to play encrypted content
1100Encrypted contentThe CDM session has already received a license response. The app has likely issued 2, or more, license responses for the same request. The subsequent licenses will be ignored so this error is informational only
1101Encrypted contentThe license has been rejected since an error was received by the CDM. The app has likely sent a non-200 code to WriteLicenseResponse
1102Encrypted contentA license response wasn't received from the app within a pre-defined timeout
1103Encrypted contentThe CDM session associated with this license response is in an invalid state. This is an internal Senza platform error
1104Encrypted contentThe CDM failed to send a license request to the app. This is an internal Senza platform error
1999Encrypted contentAn unknown encrypted content error
2000PlayerContent makes reference to no or unsupported key system
3000PlayerUnexpected problem with playback, only used if no more specific code in 3xxx range applies
3001PlayerProblem accessing content manifest, only used if no more specific code in 8xxx range applies
3002PlayerUnexpectedly stopped playback
3100PlayerProblem parsing MP4 content
3200PlayerProblem with decoder
3300PlayerDRM keys unavailable, player waited for keys but none arrived
3400PlayerProblem accessing segments, only used if no more specific code in 34xx range applies
3401PlayerProblem accessing segments, connection issue or timeout
3402PlayerProblem accessing segments, server returned HTTP error code
3403PlayerProblem accessing segments, server authentication issue
3404PlayerProblem accessing segments, server returned not found
3900-3999PlayerInternal player error
6000PlayerThe call to load() has reached the configurable timeout with no response from the remote player
6001Playerplay() was called while the remote player is not loaded
6002Playerload() was called while the application was in state 'background' or 'inTransitionToBackground'
6500PlayerremotePlayer api was called before initializing remotePlayer
6501Playerload() was called while previous load was still in progress
8001PlayerError pulling manifest. bad parameters
8002PlayerError pulling manifest. filters returned no data
8003PlayerError pulling manifest. fetch error
8004PlayerError pulling manifest. parse error
8005PlayerError pulling manifest. stale manifest detected
8006PlayerError updating manifest. internal cache error
8007PlayerError updating manifest. internal error during backoff
8008PlayerError pulling manifest. sidx parsing error
8009PlayerError pulling manifest. internal error

Properties

NameType
detail.errorCodeint
detail.messagestring

Example

remotePlayer.addEventListener("error", (event) => {
  console.error("received remotePlayer error:", event.detail.errorCode, event.detail.message);
});

"license-request"

Fired whenever the platform requires a license to play encrypted content.
The Web App is responsible for passing the (opaque) license request blob to the license server and passing the (opaque) license server response to the CDM by calling the writeLicenseResponse method on the event.

Kind: event emitted by RemotePlayer
Properties

NameTypeDescription
detailobjectObject containing ievent data
detail.licenseRequeststringBase64 coded opaque license request. The app is responsible for decoding the request before sending to the license server. Note that after decoding, the request may still be in Base64 form and this form should be sent to the license server without further decoding
writeLicenseResponsewriteLicenseResponseWrite the license server response to the platform

Example

Whilst the payload structure and access controls are specific to each license server implementation, the Widevine UAT license server requires no authentication and minimal payload formatting and therefore serves as a useful case study that may be adapted.

  remotePlayer.addEventListener("license-request", async (event) => {
      console.log("Got license-request event");
      const requestBuffer = event?.detail?.licenseRequest;
      const requestBufferStr = String.fromCharCode.apply(null, new Uint8Array(requestBuffer));
      console.log("License Request in base64:", requestBufferStr);
      const decodedLicenseRequest = window.atob(requestBufferStr); // from base 64
      const licenseRequestBytes = Uint8Array.from(decodedLicenseRequest, (l) => l.charCodeAt(0));
      // call Google API
      const res = await getLicenseFromServer(licenseRequestBytes.buffer);
      console.log("Writing response to platform ", res.code, res.responseBody);
      event.writeLicenseResponse(res.code, res.responseBody);
  });
  async function getLicenseFromServer(licenseRequest) {
      console.log("Requesting License from Widevine server");
      const response = await fetch("https://proxy.uat.widevine.com/proxy", {
          "method": "POST",
          "body": licenseRequest,
          "headers" : {
              "Content-Type": "application/octet-stream"
          }
      });
      const code = response.status;
      if (code !== 200) {
          console.error("failed to to get response from widevine:", code);
          const responseBody = await response.text();
          console.error(responseBody);
          return {code, responseBody};
      }
      const responseBody = await response.arrayBuffer();
      console.info("Got response: ");
      return {code, responseBody};
  }

"onloadmodechange"

Kind: event emitted by RemotePlayer
Example

remotePlayer.addEventListener("onloadmodechange", () => {
  console.info("remotePlayer load mode changed to", remotePlayer.getLoadMode());
});

writeLicenseResponse(statusCode, response)

Kind: global function

ParamTypeDescription
statusCodenumberLicense server HTTP response code, e.g. 200, 401, etc. Must be 200 to indicate a successful license exchange.
responsestringLicense server response as opaque binary data in an ArrayBuffer.

Config : Object

Kind: global typedef
Properties

NameTypeDescription
preferredAudioLanguagestring
preferredSubtitlesLanguagestring
autoBackgroundbooleanupon start playing automatically move to background
autoPlayboolean(Not implemented yet) upon loading start playing automatically

LoadMode : Object

Kind: global typedef
Properties

NameType
NOT_LOADEDstring
LOADINGstring
LOADEDstring

TextTrack : Object

Kind: global typedef
Properties

NameTypeDescription
idstringA unique ID for the track.
langstringThe track's language.
selectedbooleanWhether the track is selected.
autoTranslateboolean(Optional) Whether the track is an auto-translated language.

AudioTrack : Object

Kind: global typedef
Properties

NameTypeDescription
idstringA unique ID for the track.
langstringThe track's language.
selectedbooleanWhether the track is selected.