Unhandled UnicodeDecodeError for malformed data

[bastianraschke]

Hey @miguelgrinberg,

when developing my Flask application and upgraded from 3.3.0 to 4.2.0 I noticed that my unit tests got flaky. The flaky one was the test that tests a route that only accepts password based authentication. But instead of a proper username and password, a JWT is send to expect the unauthorized response. The test passed and failed randomly and I found the problem after some analysis:

If the sent JWT (including all three Base 64 encoded components concatenated by dots) is "fortuitously" accepted by the line 205:

username, password = b64decode(credentials).split(b':', 1)

without throwing a ValueError (it needs some kind of valid Base 64 and the decoded data must also contain a :), the problem comes into place: If the decoded data contains invalid characters, the decode('utf-8') throw an unhandled UnicodeDecodeError and generate a 500 Internal Server Error:

127.0.0.1 - - [28/Jan/2021 21:48:28] "GET / HTTP/1.1" 500 -
Traceback (most recent call last):
  File "/home/bastian/Desktop/venv/lib/python3.8/site-packages/flask/app.py", line 2464, in __call__
    return self.wsgi_app(environ, start_response)
  File "/home/bastian/Desktop/venv/lib/python3.8/site-packages/flask/app.py", line 2450, in wsgi_app
    response = self.handle_exception(e)
  File "/home/bastian/Desktop/venv/lib/python3.8/site-packages/flask/app.py", line 1867, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/home/bastian/Desktop/venv/lib/python3.8/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/home/bastian/Desktop/venv/lib/python3.8/site-packages/flask/app.py", line 2447, in wsgi_app
    response = self.full_dispatch_request()
  File "/home/bastian/Desktop/venv/lib/python3.8/site-packages/flask/app.py", line 1952, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/home/bastian/Desktop/venv/lib/python3.8/site-packages/flask/app.py", line 1821, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/home/bastian/Desktop/venv/lib/python3.8/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/home/bastian/Desktop/venv/lib/python3.8/site-packages/flask/app.py", line 1950, in full_dispatch_request
    rv = self.dispatch_request()
  File "/home/bastian/Desktop/venv/lib/python3.8/site-packages/flask/app.py", line 1936, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/home/bastian/Desktop/venv/lib/python3.8/site-packages/flask_httpauth.py", line 138, in decorated
    auth = self.get_auth()
  File "/home/bastian/Desktop/venv/lib/python3.8/site-packages/flask_httpauth.py", line 210, in get_auth
    'password': password.decode('utf-8')})
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb2 in position 1: invalid start byte

The reason why it worked some time is simple: If the randomly generated JWT was too "bad" in terms of format for the b64decode call, it throws an exception which was catched fine and the test resulted in an expected unauthorized response.

You can simply test it with the basic_auth.py example. Just start it and send a header with the following malformed data (it's not a JWT anymore but fulfills the above mentioned requirements):

$ curl -H 'Authorization: Basic eyJhbGciOieyJp==' -X GET http://localhost:5000/

I fixed the issue with the following pull request: #121 Besides the fix, I also added a test case for that issue.

I also still thinking if it would be a good idea to additionally pass a validate=True to the b64decode call to ensure even more a correct Base 64 input. What do you think?

Best regards from Munich,
Bastian