Build a custom player on top of AVFoundation
When building an iOS/tvOS application with video playback, the best player solution can be to build a custom one to match the exact application requirements. This article is a technical overview of fundamental points to build a custom player on top of AVFoundation.
The idea is to use
AVPlayer as a playback engine, leveraging its low level efficiency and stability.
To create a great player experience the important points are:
- Provide the best stream to the player
- Observe the player
- Manage the DRM
- Display controls
Provide the best stream to the player
AVPlayer receives its stream through an
To ensure a seamless user experience, the item preparation should be done as soon as possible and as fast as possible.
Apple released a WWDC video about this.
Ahead of providing the
AVPlayer, some preparation can be done:
- Manage the DRM
- Seek to the right playback position
- Select the audio and subtitle language
Manage the DRM
Here, we are talking about Fairplay DRM, the only technology supported by
When an HLS stream is Fairplay protected, the HLS playlist owns a
SESSION_KEY tag with an URI, this information is collected by the item to let the application and the OS load the key to decode the stream.
To fetch the licence, we need to set an
AVAssetResourceLoaderDelegate to the
AVURLAsset. The delegate creates a hook on the
AVURLAsset resource loading (in our case the licence) to let the app fetch the licence at playback.
During the hook, the application needs to:
- Fetch the application certificate
- Generate the request data with
- Ask the application server the content key for the request data
- Ask the
AVAssetResourceLoadingRequestto respond with the content key
- Close the hook
It is also possible to prefetch the licence before playback, instead of on flight, at playback; we won’t talk about prefetch, but the mecanism is basically the same. Prefetching the key is a good way of upgrading the user experience, it avoids the content key fetch, few tenths of seconds length, task before playback start.
Seek to the right playback position
To have the playback starting at the user requested position, the idea is to use the
seekTo methods. The methods take a time tolerance before and after, tolerance parameters should be used to allow a faster seek by taking advantage of the stream encoding.
AVPlayer references its time with a
CMTime type. If the stream is time based, this should be the case for any non-live stream, it is natural to perform a seek from a
TimeInterval representing the position in the stream, starting at 0. The translation from
CMTime is then pretty straight forward.
On the other hand, for live stream, it is natural to navigate through playback with dates, as it represents an ongoing event. To do so, the HLS stream comes with a
EXT-X-PROGRAM-DATE-TIME tag, allowing the player to translate a date to its own
seekToDate method doesn’t complete if the item’s status is not
Select the audio and subtitle language
The player may allow the user to select language and subtitles. It could also be an interesting feature to automatically apply user preferences at playback start. To ensure best performances, this can even be done before playback. To do so, the idea is to load the
mediaSelectionOptions asynchronoulsy and select the proper options among the available ones.
Observe the player
Once the stream is playing on screen, the application will surely need feedback about the playback, first to display relevant controls and probably to monitor player performances and user activities.
Following the playback is important, first, to update the controls transport bar, this can be done with a periodic time observer on
Periodic observation is perfect when the player is nicely playing the video with nothing else happening. But video playback is full of ambushes.
Most of the events come from the
AVPlayerItem, and are catchable through KVO.
seekableTimeRanges are helpful to understand how the player buffer is doing.
status is essential because it defines if the player has been able to fetch the content; this is generally where playback launch error occur.
Some event on
AVPlayer are also important.
rate, to connect to the play/pause button in the controls, or
externalPlaybackActive to activate Airplay.
NotificationCenter emits interesting events too, among which
AudioSessionRouteChange ; registering to those notifications will let the player pause and resume as the application goes into background, or update the UI if Airplay starts/stops.
Once the stream is prepared and the player is playing properly, the last step is to display the controls on top of the player view. To keep the controls up to date with the playback, a good solution is to gather all the playback information into a single property object, publish this object on every playback change and register the view controller in charge of displaying the controls to those changes. It is then easy to connect each UI element to its related information.
Those mandatory steps are a good base to build a custom player; they handle the most important part of the playback. However, some additional work might be required to provide a great user experience. Among the possible extra features, depending on the project specificities, there could be monitoring player performances, providing advanced controls (such as pan to dismiss gesture) or even supporting Airplay or Google Chromecast.