Support for FLAC & Opus in ISO-BMFF/MP4 for MPEG DASH playback

[ronak2121]

We tried out using Exoplayer 2.8.4 to play streams created from Ffmpeg's experimental support for storing FLAC & Opus audio in an ISO-BMFF container using MPEG DASH.

From the sample Exoplayer app, it looked like Exoplayer was able to parse the container, but it wasn't able to actually play the audio that was inside.

We tried this out in the Android Emulator.

If you need a test stream, you can use:

FLAC: https://d1v9in513d8d86.cloudfront.net/flac_opus-tests/flac_output.mpd
Opus: https://d1v9in513d8d86.cloudfront.net/flac_opus-tests/opus_output.mpd

[pedronveloso]

+1

[ojw28]

@ronak2121 - I don't think the links you've provided work any more (or they're geo-restricted). Could you take a look? Thanks.

[ronak2121]

Apologies, I've updated the above links.

[ojw28]

There are few things missing:

For FLAC:

1. We don't handle codecs="fLaC" in the DASH manifest
2. We don't handle the fLaC atom in the MP4 container

For Opus:

1. We don't handle the Opus or dOps atoms in the MP4 container

Adding in support for these is pretty easy, and gets both streams working. Note that for the streams to work, you either need a device that has platform decoders for FLAC/Opus, or you need to be using the ExoPlayer extensions that bundle them. A platform Opus decoder has been required from Android Lollipop. I think a standalone FLAC decoder has only been bundled in the platform as of relatively recently (Android's supported formats page is a bit misleading on this point, since although support has been around for a while, it was not in the form of a standalone decoder as ExoPlayer requires). I tested on a Galaxy S8, which has platform decoders for both.

I can look at cleaning up the test changes I made and getting them submitted. My question before doing that would be: To what extent is carrying Opus and FLAC in MP4 actually standardized? And to what extent is the choice of codecs value for FLAC in the DASH manifest standardized? The status of these two efforts seemed somewhat unclear from my very brief search.

[ojw28]

Support added in the change above. Note that the underlying decoders still need to be present, as described in my response above.

[ronak2121]

Just to respond to your questions. This is an upcoming standard from Xiph.org:

https://github.com/xiph/flac/blob/master/doc/isoflac.txt
https://www.opus-codec.org/docs/opus_in_isobmff.html
https://www.chromestatus.com/feature/5100845653819392
https://googlechrome.github.io/samples/media/opus-in-mp4-for-mse.html