[question] ffmpeg extension - pcm_mulaw support

[villoren]

Hi,
I'm trying to play a file with an audio stream encoded as pcm_mulaw (container is mp4).

Scenario

ffprobe outputs the following:

Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'y.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: mp42isom
    creation_time   : 2018-05-25 21:38:57
  Duration: 00:00:30.00, start: 0.000000, bitrate: 670 kb/s
    Stream #0:0(und): Video: h264 (Baseline) (avc1 / 0x31637661), yuv420p, 1280x720, 605 kb/s, 15 fps, 15 tbr, 90k tbn, 180k tbc (default)
    Metadata:
      creation_time   : 2018-05-25 21:38:57
      encoder         : JVT/AVC Coding
    Stream #0:1(und): Audio: pcm_mulaw (ulaw / 0x77616C75), 8000 Hz, 1 channels, s16, 64 kb/s (default)
    Metadata:
      creation_time   : 2018-05-25 21:38:57

And since ffmpeg supports pcm_mulaw, I thought the ffmpeg extension would do the trick.
So I built the extension using the following options:

COMMON_OPTIONS="\
    --target-os=android \
    --disable-static \
    --enable-shared \
    --disable-doc \
    --disable-programs \
    --disable-everything \
    --disable-avdevice \
    --disable-avformat \
    --disable-swscale \
    --disable-postproc \
    --disable-avfilter \
    --disable-symver \
    --disable-swresample \
    --enable-avresample \
    --enable-decoder=vorbis \
    --enable-decoder=opus \
    --enable-decoder=flac \
    --enable-decoder=wavpack \
    --enable-decoder=pcm_mulaw \
    " && \
cd "${FFMPEG_EXT_PATH}/jni" && \
...

Then verified FfmpegLibrary.isAvailable() was true, and asked DefaultRenderersFactory to EXTENSION_RENDERER_MODE_PREFER as shown:

// Renderers factory
final DefaultRenderersFactory renderersFactory =
        new DefaultRenderersFactory(this, DefaultRenderersFactory.EXTENSION_RENDERER_MODE_PREFER);

// Track selector
final BandwidthMeter bandwidthMeter = new DefaultBandwidthMeter();
final TrackSelection.Factory trackSelectionFactory = new AdaptiveTrackSelection.Factory(bandwidthMeter);
final DefaultTrackSelector trackSelector = new DefaultTrackSelector(trackSelectionFactory);

// Exoplayer instance
mExoPlayer = ExoPlayerFactory.newSimpleInstance(renderersFactory, trackSelector);

Results

• Noticed audio has no focus (hitting the phone's volume keys updates ringing volume).
• Attaching the debugger, I see FfmpegAudioRenderer is being instantiated and supportsMixedMimeTypeAdaptation() is called, but neither supportsFormatInternal() nor createDecoder() are called.
• Regarding FfmpegLibrary: neither supportsFormat() nor getCodecName() are called, although the fact that MimeType.AUDIO_MLAW isn't listed as a coded could imply it's not supported?

Questions

• If I'm doing something wrong, any thoughts about what it could be?
• If g711-mlaw is supported by ffmpeg, but not by the ffmpeg extension: is there something out there that does?
• If there isn't and I'd have to figure it out by myself, would it be easier if I write my custom Extractor and force its use? (I know the format of the source is always the same)

Thanks,
Rena

[andrewlewis]

Please could you send a link to the MP4 stream you're trying to play to [email protected] with the subject "Issue #4360"?

This requires a few small changes in the FFmpeg JNI wrapper code but should be straightforward to support.

[villoren]

Thanks for looking into this @andrewlewis.
Sending an email shortly.

Also, if there's a way I could contribute, or at least try it out for my self in the meanwhile, I'd love to do so.
Going to start by looking at ffmpeg_jni.cc, which I'm yet not familiar with.

[andrewlewis]

We'll push a change to dev-v2 that adds support for mu-law and A-law PCM via the ffmpeg extension soon, but to fix this issue it will also be necessary to update the MP4 extractor (perhaps AtomParsers.parseAudioSampleEntry) to output the right PCM encoding for this track. So once this issue is updated with a reference to that commit, it would be useful if you could make a pull request for adding MP4 support.

[villoren]

Thanks for the pointers Andrew, will do.

[andrewlewis]

@villoren The commit referenced above has everything needed for this enhancement except for building the right Format in the extractor as described above. It would be great if you could send a PR for dev-v2 to support this for Mp4Extractor, but if you're not planning to do this please let us know and we'll take a look at some point. Thanks!

As a side note, this should now work for WAV files without any further changes.

[villoren]

@andrewlewis Sorry, got caught by something else.
I'll do as soon as I can, even if you guys come up with a solution before, since I'm interested in learning the framework.

I saw this pure-integer approach to get linear samples. Going to see how i can adapt it.
[Edit: handled by ffmpeg, for sure]

Gotcha, mp4 container, no wav.