Presumed incompatibility Authentication Hikvision & Foscam RTSP URL's

[Anioz]

ExoPlayer Version

2.18.2

Devices that reproduce the issue

Samsung Android 10 Note8

Devices that do not reproduce the issue

No response

Reproducible in the demo app?

Yes

Reproduction steps

Create new solution
Prove to yourself that the specific given model of the Foscam and/or Hikvision camera is up and running with any other validation method (for example vlc or a previous xamarin forms project)
Set the mediaelement source as the proven working rtsp url with authentication (use specific hikvision and foscam models)
Run your Android app (debug or release)
Check the log for the above fatal app crash

Expected result

Handles auth headers well and doesnt crash the app

Actual result

Crashes the app at current time with following error:

"
[AndroidRuntime] FATAL EXCEPTION: ExoPlayer:Playback
[AndroidRuntime] Process: com.solid.HomeCon, PID: 18345
[AndroidRuntime] java.lang.IllegalArgumentException
[AndroidRuntime] at com.google.android.exoplayer2.util.Assertions.checkArgument(Assertions.java:39)
[AndroidRuntime] at com.google.android.exoplayer2.source.rtsp.RtspMediaTrack.generatePayloadFormat(RtspMediaTrack.java:260)
[AndroidRuntime] at com.google.android.exoplayer2.source.rtsp.RtspMediaTrack.(RtspMediaTrack.java:161)
[AndroidRuntime] at com.google.android.exoplayer2.source.rtsp.RtspClient.buildTrackList(RtspClient.java:347)
[AndroidRuntime] at com.google.android.exoplayer2.source.rtsp.RtspClient.access$1700(RtspClient.java:75)
[AndroidRuntime] at com.google.android.exoplayer2.source.rtsp.RtspClient$MessageListener.onDescribeResponseReceived(RtspClient.java:690)
[AndroidRuntime] at com.google.android.exoplayer2.source.rtsp.RtspClient$MessageListener.handleRtspResponse(RtspClient.java:598)
[AndroidRuntime] at com.google.android.exoplayer2.source.rtsp.RtspClient$MessageListener.handleRtspMessage(RtspClient.java:507)
[AndroidRuntime] at com.google.android.exoplayer2.source.rtsp.RtspClient$MessageListener.lambda$onRtspMessageReceived$0$com-google-android-exoplayer2-source-rtsp-RtspClient$MessageListener(RtspClient.java:500)
[AndroidRuntime] at com.google.android.exoplayer2.source.rtsp.RtspClient$MessageListener$$ExternalSyntheticLambda0.run(Unknown Source:4)
[AndroidRuntime] at android.os.Handler.handleCallback(Handler.java:883)
[AndroidRuntime] at android.os.Handler.dispatchMessage(Handler.java:100)
[AndroidRuntime] at android.os.Looper.loop(Looper.java:237)
[AndroidRuntime] at android.os.HandlerThread.run(HandlerThread.java:67)
[m.solid.HomeCo] 0xebadde09 skipped times: 0
[libc] exiting due to SIG_DFL handler for signal 11"

Hikvision IP camera model HWI-D140H x 3 works great without any problems (rtsp://user:pswd@ip:554/ch1/sub/live)
Hikvision Doorbell camera model DS-KD8003-IME1 x1 crashes the app (maui toolkit rtsp://user:pswd@ip:554/ch1/sub/live works both in vlc win and xamarin forms)
Foscam IP camera model R4M x 1 crashes the app (maui toolkit rtsp://user:pswd@ip:88/ch1/sub/live, works both in vlc win and xamarin forms)
Foscam IP camera model R4M-B x 1 crashes the app (maui toolkit rtsp://user:pswd@ip:88/ch1/sub/live, works both in vlc win and xamarin forms)

Media

https://1drv.ms/u/s!AqVY9RkzgdHIrRJpCfa_rIgyBW7A?e=8oxTBc (maui toolkit with Exoplayer implementation)

Exoplayer issue that has been started initially in Maui toolkit github repo CommunityToolkit/Maui#972

Bug Report

• You will email the zip file produced by adb bugreport to [email protected] after filing this issue.

[Anioz]

[RtspClient] RTSP/1.0 401 Unauthorized
[RtspClient] CSeq: 1
[RtspClient] WWW-Authenticate: Digest realm="Hikvision", nonce="a1142a0c619202fe28be22a7c8789a6d", random="G66huST+L3nomzn1NFq/isZmLVyHlh7/6qVGYsmoEHwZniHaisXZIRB/PYXiKg9R", stale="FALSE"
[RtspClient] WWW-Authenticate: Basic realm="Hikvision"
[RtspClient] Date: Wed, Feb 08 2023 12:34:06 GMT
[RtspClient] DESCRIBE rtsp://1.4.1.23:554/ch1/sub/live RTSP/1.0
[RtspClient] User-Agent: ExoPlayerLib/2.18.2
[RtspClient] CSeq: 2
[RtspClient] Authorization: Digest username="user", realm="Hikvision", nonce="a1142a0c619202fe28be22a7c8789a6d", uri="rtsp://1.4.1.23:554/ch1/sub/live", response="0de7c2e951f5c280dd5bc7af81df8d95"
[RtspClient]
[RtspClient] RTSP/1.0 200 OK
[RtspClient] CSeq: 2
[RtspClient] Content-Type: application/sdp
[RtspClient] Content-Base: rtsp://1.4.1.23:554/ch1/sub/live/
[RtspClient] Content-Length: 551
[RtspClient]
[RtspClient] v=0
[RtspClient] o=- 1675859646308821 1675859646308821 IN IP4 10.104.1.63
[RtspClient] s=Media Presentation
[RtspClient] e=NONE
[RtspClient] b=AS:5100
[RtspClient] t=0 0
[RtspClient] a=control:rtsp://1.4.1.23:554/ch1/sub/live/
[RtspClient] m=video 0 RTP/AVP 96
[RtspClient] b=AS:5000
[RtspClient] a=control:rtsp://1.4.1.23:554/ch1/sub/live/trackID=1
[RtspClient] a=rtpmap:96 H264/90000
[RtspClient] a=fmtp:96 profile-level-id=420029; packetization-mode=1; sprop-parameter-sets=
[RtspClient] m=audio 0 RTP/AVP 0
[RtspClient] b=AS:50
[RtspClient] a=control:rtsp://1.4.1.23:554/ch1/sub/live/trackID=2
[RtspClient] a=rtpmap:0 PCMU/8000
[RtspClient] a=Media_header:MEDIAINFO=494D4B48010100000400010010710110401F0000000;
[RtspClient] a=appversion:1.0

So basically its IGNORING/NOT USING the given password in the pswd string for some reason altough it works with the other URL's which are the same in several cases (except for the ip and username)

[Anioz]

With rtsp authentication DISABLED on the camera the following log output is gotten:
[RtspClient] OPTIONS rtsp://1.4.3.23:554/ch1/sub/live RTSP/1.0
[RtspClient] User-Agent: ExoPlayerLib/2.18.2
[RtspClient] CSeq: 0
[RtspClient]
[RtspClient] RTSP/1.0 200 OK
[RtspClient] CSeq: 0
[RtspClient] Public: OPTIONS, DESCRIBE, PLAY, PAUSE, SETUP, TEARDOWN
[RtspClient] Date: Wed, Feb 08 2023 13:54:56 GMT
[RtspClient] DESCRIBE rtsp://1.4.3.23:554/ch1/sub/live RTSP/1.0
[RtspClient] User-Agent: ExoPlayerLib/2.18.2
[RtspClient] CSeq: 1
[RtspClient]
[RtspClient] RTSP/1.0 200 OK
[RtspClient] CSeq: 1
[RtspClient] Content-Type: application/sdp
[RtspClient] Content-Base: rtsp://1.4.3.23:554/ch1/sub/live/
[RtspClient] Content-Length: 551
[RtspClient]
[RtspClient] v=0
[RtspClient] o=- 1675864496608871 1675864496608871 IN IP4 1.4.3.23
[RtspClient] s=Media Presentation
[RtspClient] e=NONE
[RtspClient] b=AS:5100
[RtspClient] t=0 0
[RtspClient] a=control:rtsp://1.4.3.23:554/ch1/sub/live/
[RtspClient] m=video 0 RTP/AVP 96
[RtspClient] b=AS:5000
[RtspClient] a=control:rtsp://1.4.3.23:554/ch1/sub/live/trackID=1
[RtspClient] a=rtpmap:96 H264/90000
[RtspClient] a=fmtp:96 profile-level-id=420029; packetization-mode=1; sprop-parameter-sets=
[RtspClient] m=audio 0 RTP/AVP 0
[RtspClient] b=AS:50
[RtspClient] a=control:rtsp://1.4.3.23:554/ch1/sub/live/trackID=2
[RtspClient] a=rtpmap:0 PCMU/8000
[RtspClient] a=Media_header:MEDIAINFO=494D4B48010100000400010010710110401F0000000;
[RtspClient] a=appversion:1.0
[SurfaceView] onWindowVisibilityChanged(0) true android.view.SurfaceView{a65b1cd V.E...... ......I. 0,0-0,0} of ViewRootImpl@2d39d02[MainActivity]
[AndroidRuntime] FATAL EXCEPTION: ExoPlayer:Playback
[AndroidRuntime] Process: com.companyname.rtspcrash, PID: 878
[AndroidRuntime] java.lang.IllegalArgumentException
[AndroidRuntime] at com.google.android.exoplayer2.util.Assertions.checkArgument(Assertions.java:39)
[AndroidRuntime] at com.google.android.exoplayer2.source.rtsp.RtspMediaTrack.processH264FmtpAttribute(RtspMediaTrack.java:384)
[AndroidRuntime] at com.google.android.exoplayer2.source.rtsp.RtspMediaTrack.generatePayloadFormat(RtspMediaTrack.java:261)
[AndroidRuntime] at com.google.android.exoplayer2.source.rtsp.RtspMediaTrack.(RtspMediaTrack.java:161)
[AndroidRuntime] at com.google.android.exoplayer2.source.rtsp.RtspClient.buildTrackList(RtspClient.java:347)
[AndroidRuntime] at com.google.android.exoplayer2.source.rtsp.RtspClient.access$1700(RtspClient.java:75)
[AndroidRuntime] at com.google.android.exoplayer2.source.rtsp.RtspClient$MessageListener.onDescribeResponseReceived(RtspClient.java:690)
[AndroidRuntime] at com.google.android.exoplayer2.source.rtsp.RtspClient$MessageListener.handleRtspResponse(RtspClient.java:598)
[AndroidRuntime] at com.google.android.exoplayer2.source.rtsp.RtspClient$MessageListener.handleRtspMessage(RtspClient.java:507)
[AndroidRuntime] at com.google.android.exoplayer2.source.rtsp.RtspClient$MessageListener.lambda$onRtspMessageReceived$0$com-google-android-exoplayer2-source-rtsp-RtspClient$MessageListener(RtspClient.java:500)
[AndroidRuntime] at com.google.android.exoplayer2.source.rtsp.RtspClient$MessageListener$$ExternalSyntheticLambda0.run(Unknown Source:4)
[AndroidRuntime] at android.os.Handler.handleCallback(Handler.java:883)
[AndroidRuntime] at android.os.Handler.dispatchMessage(Handler.java:100)
[AndroidRuntime] at android.os.Looper.loop(Looper.java:237)
[AndroidRuntime] at android.os.HandlerThread.run(HandlerThread.java:67)

[microkatz]

Hello @Anioz,

Thank you for reporting your issue! We submitted a fix into the dev-v2 branch for the app crashes related to your camera's RTSP DescribeResponse's invalid fmpt attribute. The app should no longer crash as a result. That being said, your cameras do not provide sprop-parameter-sets values which are required for decoding the H264 stream. Therefore the camera streams will not be supported.

Some suggestions:

• You can file a bug with Foscam and Hikvision for them to start providing the values via a firmware update. Also check if there currently are any firmware updates for your devices.
• Try changing the stream media type to mjpeg and see if that enables your cameras.

[Anioz]

How does libvlc manage to get those streams working if this is a manufacturer issue?

Im fine with whatever work around might be possible like setting the codecs manually if thats possible? I dont even mind writing custom rtsp responses but is there some way to get this working in exoplayer without going the route of asking the manufacturers individually to come up with compatibility updates for a player?

[microkatz]

@Anioz
I do find it odd that one of your Hikvision cameras has the issue while the other does not. The Hikvision camera that errored may have a firmware update available that fixes this.

As far as libvlc working? It may be that your devices provide the sprop-parameter-set's sps and pps values within non-VCL NAL units inside of the bitstream.

Currently, the RTSP implentation in Exoplayer expects these values in the session description protocol.

If you would like to test reading them from the stream, you can clone the repo and try the following changes.

• Comment out the sps related checks and code in RtspMediaTrack::processH264FmtpAttribute. Instead you will add these sps-related attributes to the RtpPayloadFormat.format variable when parsing the bitstream in RtpH264Reader
• You will need to parse for the sps and pps data in RtpH264Reader. You can use the code in H264Reader as a basis as that extracts the sps data from the NAL units from the bitstream. You can update payloadFormat.format with the Format.Builder::buildUpon method after extracting the required data.
• Comment out the line castNonNull(trackOutput).format(payloadFormat.format) in RtpH264Reader::createTracks. This method outputs the format onto the SampleQueue which will create the decoder. You want to postpone this method call until after you have extracted the required sps and pps data.
• Just like in H264Reader::endNalUnit, once you have extracted the required sps/pps data in RtpH264Reader, you will then invoke castNonNull(trackOutput).format(payloadFormat.format) to put the format onto the SampleQueue to configure the decoder.

Hopefully that works for your devices!

If you are successful, you are more than welcome to make a contribution to Exoplayer with these changes. We consider high quality pull requests. These should merge into the dev-v2 branch. Before a pull request can be accepted you must submit a Contributor License Agreement.

[microkatz]

@Anioz
As a side note, you are more than welcome to create a new Github issue labeled as an enhancement for adding the parsing of sprop-parameter-set's sps and pps values within the non-VCL NAL units inside of the bitstream to ExoPlayer. I cannot promise if/when this would be completed but at the very least then the feature will be in our queue.