Impressive WebRTC Multitrack Playback by Ant Media Server

WebRTC lets us stream multiple video/audio tracks through a single connection. To use this feature, the SDP semantic should be Unified Plan. Ant Media Server is supporting to play multiple broadcasts as tracks of a WebRTC stream. These broadcasts may be even WebRTC or RTMP streams.

With the help of the multitrack streams, you can play different groups of streams with a single broadcast id. Then you can start to play those groups of streams with one play request and the most importantly through a single WebRTC connection. So that means this decreases resource usage.

In this documentation, we will tell how you can use the multi-track playing feature in Ant Media Server and also its advantages and possible use cases.

Advantages of Multitrack Playback

The main advantage of the multitrack stream is in terms of resource usage. Since you can play multiple streams through a single connection, fewer ports are opened on the server and client and so resource usage will decrease.

Moreover, it provides ease of use. Because you don’t need to send play requests and check the status for the streams separately, you can send only one request for the group. Also dynamically added new streams to the group will be played automatically.

At the runtime, you can enable or disable the playback for a specific broadcast from the group. This will also enable and disable video/audio data transfer from the server.

Use Cases of Multitrack Playback

Ant Media Server provides a perfect SFU conference feature. In that, we send play requests to the server for all participants separately and we used to have N-1 WebRTC peer connections. From now on, we can play all participant streams with a single WebRTC peer connection.

Another use case is multiple language support for a single video. In that case, you will have one video stream and multiple audio streams for each supported language. Then on the client-side, you will be able to enable or disable audio tracks and play the audio with the language that you want to listen to. In the image below you can find the list of streams grouped with the main group id. In the group, there is a video and two audio streams.

Multitrack Streams

Using a multitrack player you can capture the stream list for a group id and select the streams that you want to play.

Multitrack Player

You can also use multitrack playback in such a case and imagine that you have different stream sources that are physically grouped as in the security systems. Then you can switch easily between these groups.

I am sure that there will be different use cases but these are the ones that I have heard from the Ant Media Server users so far.

How to use Multitrack Playback?

Before going into deep, let me make some definitions. We call the stream id of a group as “main track” and Also, the streams in a group are called “sub tracks”.

– First of all, as we mentioned in the intro, we need to switch SDP semantic to Unified Plan first.
There are two things that you need to that. One in the server settings and the other in Html pages.

On the server side:
Add settings.webrtc.sdpSemantics=unifiedPlan line into the /usr/local/antmedia/webapps/WebRTCAppEE/WEB-INF/red5-web.properties file.

On HTML pages:
Add sdpSemantics: '"unified-plan' parameter ​to the pc_config variable.

In order to group the broadcasts into a single broadcast (main track);

• For WebRTC streams, you need to pass group id as main Track while calling publish method of webrtcAdaptor in the SDK
• For RTMP streams, you need to add multiTrack parameter to the stream id as rtmp://{AMS_URL}:1935/{STREAM_ID}?multiTrack={GROUP_ID}

You can play multitrack streams by multitrack-play.html

1. Write the group id in the text box
2. The Request the sub-tracks by clicking the Tracks button
3. Then select the tracks you want to play and click the Play button
4. Please keep in mind if a new subtrack is added to the group, it will be played automatically

You can enable or disable video/audio feed for a sub-track with enableTrack(mainTrackId, trackId, enabled) methods in the webrtc-adaptor SDK. You can also get lists of sub-tracks using getTracks(streamId, token) the method in the SDK.

How to use Multitrack Conference?

We have implemented a new conference page that is compatible with multitrack playback. To use it, you need to make SDP semantic switch first as explained in the previous chapter. Then everything will be automated on the conference HTML page. The main-track (group) id will be the same as the room id. If you check the implementation, you will see that the play request is called once for the room id.

Data Channel usage is also a bit different in the multitrack conference. In a multitrack conference, you need to send the data over publisher peer connection, I mean you need to call sendData method of SDK with the publish stream id of the user. But the messages from the other participants will be received over the single WebRTC play peer connection of the room id.
You can also consider this page as a basis for your own implementations.

Summary

In this blog post, we have introduced the new multitrack playback feature that enables us to play multiple broadcasts in a single WebRTC peer connection. Enjoy with Ant Media Server and with its multitrack playback feature.

Originally published at https://antmedia.io on December 14, 2021.