Support non standard HTTP communication

[pfeigl]

Hi,

we are trying to proxy NTRIP connections through YARP. NTRIP is a (20 year old) protocol based on HTTP to transmit RTCM data (positioning data) in a streaming fashion.

Unfortunately the people designing the protocol back than derivated a little bit from the HTTP specs at some point which now gives me a few headaches when trying to proxy these streams over YARP.

This article here gives a good overview of the differences by describing the changes from rev1 to rev2 of the NTRIP protocol (where they tried to stay with the HTTP spec in rev2):
https://www.use-snip.com/kb/knowledge-base/ntrip-rev1-versus-rev2-formats/

To get a little bit more concrete on what they changed, the biggest problem is, that the response starts with something like this
ICY 200 OK or SOURCETABLE 200 OK instead of the expectd HTTP/x.y 200 OK.

Unfortunately the parsing in the HttpClient library is very strict and throws when these lines do not exactly match the HTTP spec.
(See here https://github.com/dotnet/runtime/blob/a08d9ce2caf02455c0b825bcdc32974bdf769a80/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/HttpConnection.cs#L983)

I guess my question is: Does anyone see any chance on how to get this working? I was looking at using SocketsHttpHandler.ConnectCallback to replace the underlying stream, with a wrapper stream that changes the content before relaying it to the HttpClient. But this feels like the wrong thing to do?

(I would have loved to just override the HttpClient's parsing implementation, but thats all just private internal code).

Thanks for your thoughts!

[Tratcher]

It's unlikely. HTTP is complicated enough without trying to bend it into other protocols. Abstracting and exposing the internal parser would give up a lot of flexibility and performance.

Plugging in an alternative HTTP stack would be more viable. @MihaZupan, do you have that LLHTTP prototype available?

[MihaZupan]

I am not familiar with this protocol, but is the main difference between it and HTTP the modified status line (e.g. ICY instead of HTTP) in the server response?

I see some mentions online that v1 of the protocol uses only Content-Length. If so, it should be relatively straightforward to overwrite just that part in a custom stream (SocketsHttpHandler.PlaintextStreamFilter would be the appropriate place to plug that in).

[pfeigl]

I did quite some tests meanwhile and it really seems that the modified status line is the only real difference.

I successfully managed to proxy NTRIP traffic through nginx. It only gives a few warnings because of the unexpected status line but seems to fall back to HTTP/1.1 and than successfully proxies the traffic.

We need some scripting capabilities and I would have loved to do it in .NET (but can fall back to njs in nginx). I'm going to look into the PlaintextStreamFilter. Thanks for the headsup!