Request for Comments: Implement RFC 5987

[sigmavirus24]

RFC 5987 provides a way of encoding arbitrary character encodings in ISO-8859-1 for inclusion in header values compliant with RFC 2616. Recently there have been issues opened by people expecting unicode to be usable in header values (see: https://github.com/kennethreitz/requests/issues/1926 and httpie/cli#212). Since requests is following the specification, it makes sense to me that the toolbelt could provide some functionality for dealing with this.

Thoughts @Lukasa, @piotr-dobrogost, or @shazow?

[Lukasa]

+1, let's do it.

[piotr-dobrogost]

Well, this should have been implemented a long time ago :) However, it should be in requests not toolbelt.

[Lukasa]

@piotr-dobrogost Disagree.

Requests should not be checking encodings or assuming that endpoints support RFC 5234. Many don't, and that kind of failure is really nasty to work around. Better to have it be an explicit opt-in. Remember the commitment of the requests project to the 90% use-case. =D This project is for the remaining 10%.

[sigmavirus24]

Actually, I'd like to think of this supporting 90% of that 10% ;).

[sigmavirus24]

@piotr-dobrogost the other thing is that we reserve the right to eventually move elements of the toolbelt into the core library if Kenneth wants to. The toolbelt is licensed the same as requests for just that purpose.

[sigmavirus24]

So contrary to what @piotr-dobrogost seemed to hint at in kennethreitz/requests#1926 this does not affect the actual value of the header. This just allows you to add extra information to the header in a different encoding.

[piotr-dobrogost]

I really don't get what you meant to say above.

[sigmavirus24]

RFC 5987 does not allow for you to do the following:

X-Some-Header: ΓΞΨθζϋ𐆊

It does, however allow you to do this:

X-Some-Header: some-latin-1; field=UTF-8''(ΓΞΨθζϋ𐆊 translated to Latin-1 and then percent encoded)