HLS - AVC/HEVC track formats incorrectly assigned same mime type

[AntonAFA]

Hello. Please consider following manifests.
Both of them contains HEVC and AVC codecs.

1. DASH -
   http://cfvod.kaltura.com/dash/p/2035982/sp/203598200/serveFlavor/entryId/0_3ygb9zlt/v/2/pv/7/flavorId/0_,e8rbw59u,uf2z04po,oipic3cp,pyy89j5i,8o51rwag,i3u1m33j,77bwa8xe,a2xc8gja,ypuer7a3,8cyfpyos,vby88nhm,/name/a.mp4.urlset/manifest.mpd

2. HLS - http://cfvod.kaltura.com/hls/p/2035982/sp/203598200/serveFlavor/entryId/0_3ygb9zlt/v/2/pv/7/flavorId/0_,e8rbw59u,uf2z04po,oipic3cp,pyy89j5i,8o51rwag,i3u1m33j,77bwa8xe,a2xc8gja,ypuer7a3,8cyfpyos,vby88nhm,/name/a.mp4.urlset/master.m3u8

Parsing dash manifest will create 2 video groups. First for hevc Formats and the second for avc. I understand that it happens due to the simple structure of the dash manifest(AdaptationSet). While in HLS it will create only one group and all of the Formats will have the same codec type, in this case hevc(if avc was the first codec type to appear in manifest all of them would be avc). As i saw, hlsParser parse stream right and create each Format with the correct codecType. But in some moment, probably during AtomParer it assign same codecType to all the Formats. Also it creates only one group of video Formats, so all of the avc Formats become unreadable by renderer and can not be played. I understand that possibly it happens because of the complex structure of hls manifest.
It would be grate if you can clarify why that happens and if there are any solution to such a use case.

Thanks, a lot.

[ElizaSapir]

+1

[ojw28]

Answering the question parts:

• It's correct behavior that the DASH manifest results in two TrackGroups where-as the HLS manifests only results in one. This correctly mirrors the different structures in the manifests themselves. For the DASH manifest to result in a single TrackGroup you'd need either (a) AVC and HEVC Representation elements to be contained by a single AdaptationSet element in the manifest, or (b) use of urn:mpeg:dash:adaptation-set-switching:2016 in the manifest to indicate support for switching between the two AVC and HEVC AdaptationSet elements. My understanding is (b) is the recommended approach. See DASH: Support cross-AdaptationSet seamless switching #2431 and the DASH-IF Interoperability Points as quoted in that issue.
• Where AVC and HEVC tracks are contained by the same TrackGroup, you can enable adaptive switching between them using withAllowMixedMimeAdaptiveness as described in Couldn't adaptive different codecs in HLS stream #3500 (comment).

The following looks like a bug, since the codecs should be accurately carried into the formats of the tracks in the TrackGroup. Assigning to @AquilesCanta to take a look, and marking as a bug accordingly:

While in HLS it will create only one group and all of the Formats will have the same codec type

[AquilesCanta]

While in HLS it will create only one group and all of the Formats will have the same codec type

I don't think the codec propagation has any problems. However, I can reproduce an issue with the track's mime type for which a fix will be pushed soon. Please clarify if are referring to a codec specific issue.

[AntonAFA]

Hey. Thank for your reply. I am not sure that i understood your reply regarding to "Please clarify if are referring to a codec specific issue".
I got hls manifest with mixed codec types. I expect that exoplayer will create separate Format groups for each codec. Like it is doing with dash manifest.

[AquilesCanta]

The fix push will appear ref'd below, and you'll be able to try it out. But as @ojw28 explained above, the track group will be only one.

[AquilesCanta]

If the content is actually the same, it is better for you to have all codecs in the same track group. You can either pick one codec and select only tracks with that codec, or both and adapt between them (by allowing mixed mime type adaptation). The latter is not possible with separate track groups. Even if they are in the same track group, you will be able to pick a subset of them (in your case, depending on the video formats's mimetype).

[OrenMe]

@AquilesCanta how can we pick one codec? mappedTrackSelctor? can you please show an example?

[AntonAFA]

Thank you. Do you have any time estimation/side branch for that fix. I just want to track and check it as soon as possible.

[ojw28]

This is fixed in dev-v2.