Allow per-view Schema Generation customisation

[carltongibson]

A recurring issue is the need to slightly customise the Schema Generation. The default behaviour is correct in the main; what we’re missing is the ability to customise the behaviour for specific views.

For full control it is possible to write schemas by hand, but we want to be able to keep the automatic generation, in the main, whilst tweaking the aspects that are not correct (for our project).

Some proposals involve adopting a specific markdown syntax in order to encode (ever) more complex data structures in docstrings. We would then parse these, using Sphinx or pyparsing or a custom parser.

This is not the way to go. (External dependencies, fragility, and so on.) Better is to use Core Schema to clearly/precisely/etc specify the details we require.

The current implementation is Top Down — the generator inspects each view in turn to create the schema. Its limitation is that it does not (easily) allow customisation on a per-view/serialiser/etc basis.

In order to avoid adding more API to the APIView class, we will move the bulk of the generator’s introspection logic to a descriptor class that will be attached to the view. It will be accessed from the view and perform the necessary schema generation:

view.schema.get_schema()

(Or such)

The descriptor class will expose the required hooks to allow per-view customisation of the schema generation.

For example, it is not currently possible to provide a description for a non-model path parameter on a URL. An implementation of (the equivalent to the current) get_path_fields adding this would go something like:

fields = super().get_path_fields(...)
extra_field = coreapi.Field(...)
fields.append(extra_field)
return fields

i.e. We take the default implementation and make the (small) changes we want.

Users would be free to take whatever steps were needed to keep this DRY in their own projects.

The top-level schema generator will be maintained. It’s responsibilities will be reduced to accepting the patterns to be processed and gathering the resulting schema document.

Tasks

• Migrate introspection logic from SchemaGenerator to descriptor class for APIView.
  • Make sure it still works.
• Document interface and usage.

Related Issues

• Documentation for Non-Model Views/Viewsets #4685: Difficulty customising docs in Non-Model contexts
• field_to_schema should handle ListField like ListSerializer #5311: field_to_schema needs to handle ListField types
• Have schemas.is_list_view recognise RetrieveAPIView subclasses #5165: is_list_view error with RetrieveAPIView subclass

(There are others.)

Related PRs

• Restructure SchemaGenerator for easier subclassing #5271: Moves is_custom_action & is_list_view into SchemaGenerator
• Remove wildcard from api endpoint documentation #5290: Adds special-case to should_include_endpoint
• Remove wildcard from api endpoint documentation #5290: Adds get_title introspection.
• default value for path field should be empty string instead of None #5224: Customises get_path_fields to set Swagger-friendly default value
• Schema generation overriding #5335: Adds per-field customisation.

[tomchristie]

I'm wondering if in addition to the programmatic overrides, we might want to support a fields or schema_fields attribute on views that allows users to set a list of fields to use, declaratively, but without having to write any method code. We'd hook that into the existing behavior https://github.com/encode/django-rest-framework/blob/master/rest_framework/schemas.py#L445 by checking for any explicitly declared fields, and then replacing the automatically generated one (if there was a field with the same name) or extending the list.

The benefit of doing that is that we'd be able to meet he "extend the list, or override a field" case without having to override methods, and it'd also be extensible to FBVs where we could apply it as an attribute.

[Place1]

Will this also work as a solution to documenting 3rd party endpoints? Say I've included a reusable app in my project and i've included some API endpoints from it (urls) but the generator doesn't correctly detect the inputs/outputs. Would I be able to dynamically add a .schema to the view before I add it to my urlconf to manually define the endpoint?

[carltongibson]

@tomchristie: Yep. That sounds good. I'd rather pass kwargs to the descriptor class than add attributes to APIView (or children) but the idea is 👍. I guess it will look something like:

class MyView(SomeSuperView):
  schema = SchemaDescriptor(       # Name Pending but this would be the provided class
     fields = {
      "my_extra": coreapi.Field( ... ), 
    },
    ...
  )

There's an extra line here but it keeps all the schema logic in the descriptor class and out of the view. If it gets too big inline it's very easy to extract and refactor to a subclass.

@Place1:

Would I be able to dynamically add a .schema to the view before I add it to my urlconf to manually define the endpoint?

Yeah, I think so: it's Python 🙂

[carltongibson]

I've opened #5354 which addresses this.

It's fully functional (I think 😀) so do try it and feedback if you're wanting to customise the schema generation.