Check multiple response codes

[javier-sierra-sngular]

Proposal

Objective

We want to verify if the implementation of the API complies with the contract established with the OpenAPI document.

In an OpenAPI document you can specify more than one response code with a different body in each case. It would be nice if we could test each one by making multiple requests for the same endpoint, one for each status code.

The problem that arises from this idea is how to trigger a different response code for the same endpoint.

We have thought about defining an additional file, with the intention of decoupling it from the original document. In this file, the final user specifies how to trigger a response code through a request. The format of this document will also follow the OpenAPI specification, using the “examples” field. This way, it will be easy to process the information in the workflow generation and we can use the Swagger validator to check if the document is well-formed.

Use example

The examples file could be specified by using a flag or as a second parameter of stepci generate:

stepci generate path/contract.yml --request-examples path/contract.examples.yml
stepci generate path/contract.yml path/contract.examples.yml

OpenAPI contract (starter point)

The following OpenAPI document is the specification of the API that will be referred to in the subsequent examples.

openapi: 3.0.0
info:
  title: Example API
  version: 1.0.0
servers:
  - url: http://localhost:1234
paths:
  /example-headers:
    get:
      responses:
        '200':
          description: This is the response for the HTTP status code 200.
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/SuccessResponse'
        '500':
          description: This is the response for the HTTP status code 500.
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ErrorResponse'

  /example-query:
    get:
      parameters:
        - name: foo
          in: query
          schema:
            type: string
      responses:
        '200':
          description: This is the response for the HTTP status code 200.
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/SuccessResponse'
        '500':
          description: This is the response for the HTTP status code 500.
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ErrorResponse'

  /example-request-body:
    post:
      requestBody:
        content:
          application/json:
            schema:
              type: object
              properties:
                property1:
                  type: string
                property2:
                  type: string
      responses:
        '200':
          description: This is the response for the HTTP status code 200.
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/SuccessResponse'
        '500':
          description: This is the response for the HTTP status code 500.
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ErrorResponse'

components:
  schemas:
    SuccessResponse:
      type: object
      properties:
        data:
          type: string
    ErrorResponse:
      type: object
      properties:
        error:
          type: integer

Examples document structure

Several options are provided for the user to specify how to trigger a specific HTTP response code through an HTTP request. All of them share the use of examples field.

Object Examples

The object key represents an HTTP response code, and the value is an object with the following properties:

• summary (optional): Explanatory text.

• value (required): Payload value to be sent. It must follow the same format as the associated schema, according to the OpenAPI specification:

  *“In all cases, the example value is expected to be compatible with the type schema of its associated value. Tooling implementations MAY choose to validate compatibility automatically, and reject the example value(s) if incompatible.”*

  https://spec.openapis.org/oas/v3.1.0#example-object

Placement

1. parameters (https://spec.openapis.org/oas/v3.1.0#parameter-object)

   1. header
      The user specifies the header to be added to the request, with the goal of having the API interpret it and return the expected response code. The header name, its value, and interpretation are on the user's side.

      In this example, Prefer: statusCode=200 and Prefer: statusCode=500 headers are used.

      get:
        parameters:
          - name: Prefer          
            in: header
            schema:
              type: string
            examples:
              '200':
                summary: Expects a 200 status code from the API
                value: 'statusCode=200'
              '500':
                summary: Expects a 500 status code from the API
                value: 'statusCode=500'

   2. query

   get:
     parameters:
       - name: foo          
         in: query
         schema:
           type: string
         examples:
           '200':
             summary: This is a correct parameter example
             value: bar
           '500':
             summary: This is a wrong parameter example
             value: xxx

2. requestBody (https://spec.openapis.org/oas/v3.1.0#request-body-object)

   post:
     requestBody:
       content:
         application/json:
           examples:
             '200':
               summary: This is a correct parameter example
               value:
                 property1: 'foo'
                 property2: 'bar'
             '500':
               summary: This is a wrong parameter example
               value: 'xxx'

Complete example

openapi: 3.0.0
info:
  title: Example API
  version: 1.0.0
paths:
  /example-headers:
    get:
      parameters:
        - name: Prefer
          in: header
          schema:
            type: string
          examples:
            '200':
              summary: Expects a 200 status code from the API
              value: 'statusCode=200'
            '500':
              summary: Expects a 500 status code from the API
              value: 'statusCode=500'
      responses:
        '200':
          description: This is the response for the HTTP status code 200.
        '500':
          description: This is the response for the HTTP status code 500.

  /example-query:
    get:
      parameters:
        - name: foo
          in: query
          schema:
            type: string
          examples:
            '200':
              summary: This is a correct parameter example
              value: bar
            '500':
              summary: This is a wrong parameter example
              value: xxx
      responses:
        '200':
          description: This is the response for the HTTP status code 200.
        '500':
          description: This is the response for the HTTP status code 500.

  /example-request-body:
    post:
      requestBody:
        content:
          'application/json':
            examples:
              '200':
                summary: This is a correct parameter example
                value:
                  property1: 'foo'
                  property2: 'bar'
              '500':
                summary: This is a wrong parameter example
                value: 'xxx'
      responses:
        '200':
          description: This is the response for the HTTP status code 200.
        '500':
          description: This is the response for the HTTP status code 500.