Support for parameter interdependencies

[mikekistler]

Moonwalk should allow API designers to express interdependencies between the parameters of an operation.

This is part of the initial Moonwalk Proposal and is broken out here to refine the ideas before creating the ADR.

The Problem to be Solved

All prior versions of OpenAPI used an array to define parameters for an operation. This approach does not provide any formal mechanism for describing interdependencies between parameters. Research [1] has shown that interdependencies are not uncommon in real-world APIs and come in a variety of forms. Issue #256 (opened in 2015) requested this feature and has received over 600 up-votes.

1. "API Specs and Inter-Parameter Dependencies: The Elephant in the Room", Alberto Martin, API Specifications Conference 2022

Proposed solution

Rather than using an array to define the parameters for an operation, we should use a JSON Schema. The parameters schema must be an object schema and must not have additionalProperties. Each named property of the parameters schema defines an operation parameter. Property names of the parameter schema have the form [<in>.]name, where <in> may be "query", "header", "path", or "cookie". When not specified, <in> defaults to "query".

Note: Combining the <in> and name allows parameters with the same name but different <in>, as is permitted in prior versions of OpenAPI. It also avoids the need to augment JSON Schema for this purpose.

The parameterSchema field would be supported on Operations and on Path Items. If specified on a Path Item, the effective parameter schema for an operation would be the allOf of the path item schema and the operation schema.

The parameter schema may contain JSON Schema assertions to express interdependencies between parameters. Some common interdependencies are described below.

Requires

The presence of a parameter (or a specific parameter value) requires the presence of another parameter.

Example: answerCount must be specified if promote is specified.

    anyOf:
    - not:
      required: [promote]
    - required: [answerCount]

Or

Given a set of parameters, one or more of them must be included.

Example: either title or description must be specified.

    anyOf:
    - required: [title]
    - required: [description]

OnlyOne

Given a set of parameters, one and only one of them must be included.

Example: either from or messagingServiceId must be specified but not both.

    oneOf:
    - required: [from]
    - required: [messagingServiceId]

AllOrNone

Given a set of parameters, either all of them are provided or none of them is.

Example: subject_type and subject_id are both specified or neither is specified.

    oneOf:
    - required: [subject_type,subject_id]
    - not:
        anyOf:
          - required: [subject_type]
          - required: [subject_id]

Example: value is present if and only if type is "bucket"

    oneOf:
    - properties:
        type: 
          enum: ["bucket"]
      required: [type,value]
    - not:
        properties:
          type: 
            enum: ["bucket']      
        required: [type,value]

ZeroOrOne

Given a set of parameters, at most one of can be included.

Example: specify zero or one of forContentOwner, forDeveloper, forMine, or relatedToVideoId:

    oneOf:
    - required: [forContentOwner]
      not:  
        anyOf:
        - required: [forDeveloper]
        - required: [forMine]
        - required: [relatedToVideoId]
    - required: [forDeveloper]
      not:  
        anyOf:
        - required: [forContentOwner]
        - required: [forMine]
        - required: [relatedToVideoId]
    - required: [forMine]
      not:  
        anyOf:
        - required: [forContentOwner]
        - required: [forDeveloper]
        - required: [relatedToVideoId]
    - required: [relatedToVideoId]
      not:  
        anyOf:
        - required: [forContentOwner]
        - required: [forDeveloper]
        - required: [forMine]
    - not:
        anyOf:
        - required: [forContentOwner]
        - required: [forDeveloper]
        - required: [forMine]
        - required: [relatedToVideoId]

Note: There is probably an easier way to express this. Happy for feedback here.

Compatibility with OpenAPI 3.x

I believe that no functionality of OpenAPI 3.x is lost with this proposal. It should be straightforward to transform an existing parameters array to an equivalent parameterSchema.

[mikekistler]

Just realized that I have not accounted for the OpenAPI 3.x style attribute -- need to think about how to represent that in the parameter schema.

[miqui]

@mikekistler - so just focusing on the use case: (to keep it simple for now).
"The presence of a parameter (or a specific parameter value) requires the presence of another parameter."

SHOULD we use anything but JSON Schema?

[miqui]

I feel like we SHOULD use JSON Schema and not throw another DSL into the mix. I had suggested in our last call OPA (uses Rego DSL as a mechanism to evaluate constraints)

[miqui]

I think OPA is not so such a good idea.

[miqui]

{
  "properties": {
    "promote": {
      "type": "boolean"
    },
    "answerCount": {
      "type": "integer"
    }
  },
  "if": {
    "properties": {
      "promote": {
        "const": true
      }
    },
    "required": ["promote"]
  },
  "then": {
    "required": ["answerCount"]
  }
}

So, something like this can be ref'd like so ?

paths:
  /pets:
    get:
      summary: Get a list of pets
      parameters:
        - name: name
          in: query
          description: Name of the pet
          schema:
            type: string
        - name: breed
          in: query
          description: Breed of the pet
          schema:
            type: string
        - name: age
          in: query
          description: Age of the pet
          schema:
            type: integer
            minimum: 0
        - name: limit
          in: query
          description: Maximum number of results to return
          schema:
            type: integer
            minimum: 1
            maximum: 100
            default: 20
        - name: offset
          in: query
          description: Index of the first result to return
          schema:
            type: integer
            minimum: 0
            default: 0
        schema:
            $ref: '#/components/schemas/paramConstraint1'

[miqui]

note: the schema keyword is at the parameter level to imply that this constraint is to be applied to all parameters for this particular request.

[mikekistler]

Notes from Moonwalk-sig meeting 4/9

• Marsh (and others) expressed concern about the usability of some of these JSON Schema expressions for tooling vendors.
• Dan expressed concern with using JSON schema to describe a part of the HTTP message (the query string) that is not actually JSON
  • e.g. how to represent repeated keys
  • e.g. value is always a string
• How to handle number values? Why can't we define them
• Dan: You have to have some sort of type-hinting before you get to the schema.
• Mike: Should we consider Darrel's suggestion of leaving the parameters array in place and using a JSON schema only to describe the interdependencies between parameters.
• We need a data model that maps a query string to JSON.
  • Dan is already working on this. Perhaps we can discuss this next week.
  • https://hudlow.github.io/query-parse-suite/
• From Henry: There was an abandoned effort at an RFC, but I don't think anyone uses it: https://datatracker.ietf.org/doc/html/draft-hoehrmann-urlencoded-01

[handrews]

In today's meeting I mentioned an old draft RFC for query strings, so here is a link to that:

• https://datatracker.ietf.org/doc/html/draft-hoehrmann-urlencoded-01

It wasn't developed very far but might be of interest given that the WHATWG spec only defines algorithms and not grammars. I imagine they don't produce identical sets of valid strings, but that itself might be interesting.

There are several other relevant Moonwalk discussions, including ones on the scope of query string / headers / cookies, which is a decision I suggested could be made separately (maybe I'll look into that- I'll update that discussion if so) and also one about the similar issues around response headers, which is part of what I was talking about for the scope (request headers are params in OAS 3, but response headers are separate):

• One parameterSchema or one each for URL Parameters, headers and body? #20
• How to handle response headers? #22
• Arithmetic and relational inter-parameter dependencies and readability concerns #24
• Support for validation across parameters #31

The other point I made is that the query string can be any string, it does not need to be in application/x-www-form-urlencoded format. This has come up in the main spec repo:

• Support for arbitrary query strings OpenAPI-Specification#1502

[karenetheridge]

I like that it would now be possible to specify dependencies between other parameter types (previously it was only possible with 'query' paramters, see below), and also between parameters of different types (e.g. if a particular path parameter is seen, require or disallow a particular header).

I have questions regarding the proposal as written:

Query parameters can be named as (for example) query.petType or just petType. How then will we deserialize query parameters into an object (for passing to json schema evaluation)? All the examples refer to just the parameter name, without the query. prefix. Can the schema say (for example) "required": ["query.petType"], or will this property only be called petType in the deserialized data ?

The current mechanism for deserializing all query parameters to a single object for schema verification can be done like so (following the example in the 3.1 spec):

- name: color
  style: form
  explode: true
  schema:
    type: object
    required: [R, G, B]

The uri: https://example.com/foo?R=100&G=200&B=150 will deserialize to { "R": "100", "G": "200", "B": "150" }.

But if style and explode functionality is removed or changed, then we'll need to think about how the behaviour in the above example will change. It may still be desired to group all query parameters into a single object, at a lower level than all other parameters (path, header, cookie), e.g. to say "no other query parameters but these are permitted" (we can't easily restrict all parameters, because arbitrary headers are often included in requests that we can't or don't want to specify in the openapi doc).

[mikekistler]

It is surprising and a bit disturbing that style: form, explode: true parameters with a type: object schema work this way, but it does seem to open the door for expressing parameter interdependencies within a particular parameter type.

Looking back through the examples from Alberto's talk, these all seem to be cases of interdependencies between query parameters. So, e.g., the "Requires" example could be described in OpenAPI 3.0.x with:

  parameters:
  - name: _
    in: query
    style: form
    explode: true
    schema:
      type: object
      properties:
        promote:
          type: boolean
        answerCount:
          type: integer
          format: int32
      # `answerCount` must be specified if `promote` is specified.
      anyOf:
      - not:
        required: [promote]
      - required: [answerCount]

I will not argue that this is easy for tooling ... but it is worth noting (as @karenetheridge did above) that it is already possible in OpenAPI 3.0.x.

[earth2marsh]

Inter-dependency implies a bunch of logic checks in tooling that would require work before tooling authors could claim full-support. For example, what would we expect from tools like Swagger UI or Postman when consumers are crafting a request? [I think inter-dependency would at least show some sort of warning if you've created a malformed request (like an onblur check that the other inter-dependent input field had a value).]

If this complexity is worth that investment, we should do it--I'd just prefer it to be obvious that this creates much more value in the world than it costs. It does seem like there is enthusiasm in this thread for such a feature. I wonder if we might see this utility demonstrated with x-extensions with tooling support that complements the historic interest we've seen for this.

[handrews]

@earth2marsh this exact complexity already exists for payloads. Why is it more difficult for parameters?

[earth2marsh]

I'd posit that we didn't intend to introduce that complexity, it just came along with JSON Schema. 😀

[handrews]

In fact, I would be shocked if tools treat the Schema Object within a Parameter Object differently from any other Schema Object. Why would they? That is the more complex implementation choice. The simple choice is to treat all Schema Objects the same, in which case Schema Object interdependency features work exactly as well (or poorly) in parameters as elsewhere. It's just a schema, just like any other schema.

The fact that after schema validation you serialize it into query strings, headers, and path templates is orthogonal to the schema complexity. And even that's not unique, as mentioned in today's call, application/x-www-form-urlencoded request bodies already work exactly like this! Plus an Encoding Object, just to make things even more complicated, which is not a concern here.

[handrews]

@earth2marsh I do get that you put a smiley on there, but in all seriousness, does that matter, and if so why?