Response caching issue

[codingjoe]

Pickel do not work since version 3.2.* on a restframework response. Thus caching responses doesn't work if you use the pickel serializer for caching.

TypeError: __new__() takes exactly 3 arguments (2 given)
  File "django/core/handlers/base.py", line 111, in get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "newrelic/hooks/framework_django.py", line 499, in wrapper
    return wrapped(*args, **kwargs)
  File "django/views/decorators/csrf.py", line 57, in wrapped_view
    return view_func(*args, **kwargs)
  File "rest_framework/viewsets.py", line 87, in view
    return self.dispatch(request, *args, **kwargs)
  File "newrelic/hooks/component_djangorestframework.py", line 27, in _nr_wrapper_APIView_dispatch_
    return wrapped(*args, **kwargs)
  File "rest_framework/views.py", line 466, in dispatch
    response = self.handle_exception(exc)
  File "rest_framework/views.py", line 463, in dispatch
    response = handler(request, *args, **kwargs)
  File "rest_framework_extensions/cache/decorators.py", line 43, in inner
    kwargs=kwargs,
  File "rest_framework_extensions/cache/decorators.py", line 60, in process_cache_response
    response = self.cache.get(key)
  File "django_redis/cache.py", line 25, in _decorator
    return method(self, *args, **kwargs)
  File "django_redis/cache.py", line 73, in get
    client=client)
  File "django_redis/client/default.py", line 205, in get
    return self.decode(value)
  File "django_redis/client/default.py", line 303, in decode
    value = self._serializer.loads(value)
  File "django_redis/serializers/pickle.py", line 43, in loads
    return pickle.loads(force_bytes(value))

[xordoquy]

@codingjoe thanks for reporting. Is it possible to have strip things and have a minimum test case for this specific issue ?

[codingjoe]

Yes, here 😉
https://github.com/chibisov/drf-extensions/blob/master/rest_framework_extensions/cache/decorators.py#L60

[tomchristie]

We do have unit tests for response caching.

Django master has a response issue that's resolved in the 3.3 branch, which we plan to have that merged in prior to 1.9.

In the absence of any actually helpful minimal reproducible case, I'm going to assume that running against Django master is the issue. If that's not the case and you still believe there's an issue then I'd appreciate a little more work on narrowing down a minimal reproducible case, so that we can help.

[mlavin]

I'm seeing this issue as well and I think I can add a bit more clarity. It appears that the Hyperlink objects from rest_framework.relations cannot be pickled and break serialization of responses which contain them.

>>> import pickle
>>> from rest_framework.relations import Hyperlink
>>> link = Hyperlink('http://example.com', 'test')
>>> pickle.loads(pickle.dumps(link))
Traceback (most recent call last):
  File "<console>", line 1, in <module>
TypeError: __new__() missing 1 required positional argument: 'name'

[tomchristie]

Think it needs reopening then?

[mlavin]

A new issue that Hyperlink objects cannot be pickled seems more actionable and testable. I'll open a new one.

[tomchristie]

Gotcha