pubsub-service-api.yml

openapi: "3.0.2"
info:
  title: "Pubsub service API"
  version: "1.0.0"
  description: |
    ## Goal
    
    The goal of this API is to be able to use pubsub on any device and under any condition to build decentralized application. Powerful and well connected machine can use pubsub directly, but different reasons can make that difficult. Notable one are mobile or IOT devices: bad networking (NAT, ...), battery life, available compute or application lifespan.
    
    This API allow to delegate to a more powerful instance or a service provider this role of interacting with pubsub. As this API is asynchronous at its core, it allows to instruct that provider to listen to topics, sleep, and come back later to retrieve messages. 
    
    ## Design principles
    
    - effort is made to allow client to operate with the minimum of requests, and within a limited time window. An example of that is /join that returns before the topic is actually subscribed and ready.
    - as the real world as shown, there is a gradient of necessary validation of received message, ranging from generic or protocol layer (ex:  deduplicate messages, max message size) to application specific (non-conforming message, abuse ...). All of those would usually be done in the same place if the application directly perform pubsub, but this is not the case here. The choice here is to split those into server side (the generic part, including some form of spam prevention) and the client side (anything that require specific logic or data), and have a way for the client to hint the server about those bad message or PeerID.
    - provide a versioned API from the start
    
    ## Versioning
    
    This API is expected to be hosted at an URL with a path ending with `/v1`, so that future major versions, if any, can be hosted side by side. An example of that would be `https://pubsub-service.example.com/v1`.
    
    ## Expected usage pattern
    
    ### Subscribing to a topic
    
    This first step to use a pubsub service is to join a topic, and optionally define limits. For example, the following will join the `foo` topic for a maximum of 3600s, have a queue of 10 messages of maximum 1000 bytes, and drop old messages if the queue fills up:
    
    `POST /join?topic=foo&queue-length=10&queue-policy=drop-old&timeout=3600&max-message-size=1000`
    
    The service will confirm with a 202, and returns the effective limits as a confirmation. Those limits can be restricted by the service if they are outside of the maximum advertised in the /discovery endpoints:
    
    ```json
    {
      "queue-length": 10,
      "queue-policy": "drop-old",
      "timeout": 3600,
      "max-message-size": 1000
    }
    ```
  
    ### Publish and read messages
    
    We now can publish a message, by passing it as the body of the following request:
    
    `POST /publish?topic=foo`
    
    We can read messages in the queue:
    
    `GET /read?topic=foo&count=2`
    
    Here, we ask for a maximum of 2 messages. The service returns them, and inform us that there is 8 more in the queue, and that 2 have been dropped due to the queue filling up:
    
    ```json
    {
      "message-dropped": 2,
      "remaining-messages": 8,
      "messages": [
        {
          "sender": "QmdXGaeGiVA745XorV1jr11RHxB9z4fqykm6xCUPX1aTJo",
          "content": "aGVsbG8gd29ybGQK"
        },
        {
          "sender": "QmdXGaeGiVA745XorV1jr11RHxB9z4fqykm6xCUPX1aTJo",
          "content": "aG93IGFyZSB0aGluZ3MgdG9kYXk/Cg=="
        }
      ]
    }
    ```
    
    ### Stay subscribed
    
    To avoid keeping alive irrelevant topic subscriptions, services have a timeout defined per topic. If the client is not active on that topic (/join, /publish, /read or /read-all), the service consider that the client is gone and teardown that subscription. To avoid that, clients should signal liveness with a query to one of those endpoints, which will reset the timeout to the maximum duration defined during /join.
    
    Which endpoint to use is up to the client. For example, a client that does read message might want to periodically /read or /read-all, while a client only writing messages might want to periodically use /join again if not enough messages are published.

servers:
  - url: https://pubsub-service.example.com/v1

paths:
  /discovery:
    get:
      summary: Describe the service configuration and limits
      description: "Describe the service configuration and limits.
      
      This endpoint returns the maximum acceptable settings that a client can ask for."
      responses:
        "200":
          description: "Returns the service configuration"
          content:
            application/json:
              schema:
                $ref: "#/components/schemas/DiscoveryResponse"
        '400':
          $ref: '#/components/responses/BadRequest'
        '401':
          $ref: '#/components/responses/Unauthorized'
        '409':
          $ref: '#/components/responses/InsufficientFunds'
        '429':
          $ref: '#/components/responses/TooManyRequests'
        '4XX':
          $ref: '#/components/responses/CustomServiceError'
        '5XX':
          $ref: '#/components/responses/InternalServerError'

  /join:
    post:
      summary: Subscribe to a pubsub topic; also signal liveness
      description: "Subscribe to a pubsub topic.
      
      The client can define configuration for that topic (queue length, timeout...). Those configuration can be restricted by the server to conform to the server capability (see /discovery). The effective configuration is returned as part of the response.
      
      Join is only effective for some duration, defined by the previously described process. In order to keep listening to that topic, the client should perform a /join, /read, /read-all or /publish before expiration, to signal liveliness. This operation is idempotent if the given parameters result in the same configuration, but a different operation should be effective and drop messages if necessary.
      
      Note that subscription is not an instantaneous operation and can happen after this request returns. It may take some time before the subscription is processed and effective in the server."
      parameters:
        - $ref: "#/components/parameters/topic"
        - $ref: '#/components/parameters/queue-length'
        - $ref: '#/components/parameters/queue-policy'
        - $ref: '#/components/parameters/timeout'
        - $ref: '#/components/parameters/max-message-size'
      responses:
        "202":
          description: "Join accepted. The service will proceed and start to listen."
          content:
            application/json:
              schema:
                $ref: "#/components/schemas/JoinResponse"
        '400':
          $ref: '#/components/responses/BadRequest'
        '401':
          $ref: '#/components/responses/Unauthorized'
        '409':
          $ref: '#/components/responses/InsufficientFunds'
        '429':
          $ref: '#/components/responses/TooManyRequests'
        '4XX':
          $ref: '#/components/responses/CustomServiceError'
        '5XX':
          $ref: '#/components/responses/InternalServerError'

  /leave:
    post:
      summary: Unsubscribe from a pubsub topic
      description: Unsubscribe from a pubsub topic, and destroy any stored message for that topic.
      parameters:
        - $ref: "#/components/parameters/topic"
      responses:
        "200":
          description: "Leave is effective"
        '400':
          $ref: '#/components/responses/BadRequest'
        '401':
          $ref: '#/components/responses/Unauthorized'
        '404':
          $ref: '#/components/responses/TopicNotFound'
        '409':
          $ref: '#/components/responses/InsufficientFunds'
        '429':
          $ref: '#/components/responses/TooManyRequests'
        '4XX':
          $ref: '#/components/responses/CustomServiceError'
        '5XX':
          $ref: '#/components/responses/InternalServerError'

  /publish:
    post:
      summary: Publish a message in a pubsub topic; also signal liveness
      parameters:
        - $ref: '#/components/parameters/topic'
      requestBody:
        required: true
        content:
          application/octet-stream:
            schema:
              type: string
              format: binary
      responses:
        "200":
          description: "OK"
        '400':
          $ref: '#/components/responses/BadRequest'
        '401':
          $ref: '#/components/responses/Unauthorized'
        '404':
          $ref: '#/components/responses/TopicNotFound'
        '409':
          $ref: '#/components/responses/InsufficientFunds'
        '413':
          $ref: '#/components/responses/MessageTooBig'
        '429':
          $ref: '#/components/responses/TooManyRequests'
        '4XX':
          $ref: '#/components/responses/CustomServiceError'
        '5XX':
          $ref: '#/components/responses/InternalServerError'

  /read:
    get:
      summary: Read message from a pubsub topic; also signal liveness
      parameters:
        - $ref: '#/components/parameters/topic'
        - $ref: '#/components/parameters/max-message'
        - $ref: '#/components/parameters/include-signature'
      responses:
        "200":
          description: "Returns the read messages as well as pagination information"
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ReadResponse'
        '400':
          $ref: '#/components/responses/BadRequest'
        '401':
          $ref: '#/components/responses/Unauthorized'
        '404':
          $ref: '#/components/responses/TopicNotFound'
        '409':
          $ref: '#/components/responses/InsufficientFunds'
        '429':
          $ref: '#/components/responses/TooManyRequests'
        '4XX':
          $ref: '#/components/responses/CustomServiceError'
        '5XX':
          $ref: '#/components/responses/InternalServerError'

  /read-all:
    get:
      summary: Read messages from multiple topic at once; also signal liveness
      parameters:
        - $ref: '#/components/parameters/max-message'
        - $ref: '#/components/parameters/filter-prefix'
        - $ref: '#/components/parameters/filter-suffix'
        - $ref: '#/components/parameters/include-signature'
      responses:
        "200":
          description: "Returns the read messages as well as pagination information"
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ReadAllResponse'
        '400':
          $ref: '#/components/responses/BadRequest'
        '401':
          $ref: '#/components/responses/Unauthorized'
        '409':
          $ref: '#/components/responses/InsufficientFunds'
        '429':
          $ref: '#/components/responses/TooManyRequests'
        '4XX':
          $ref: '#/components/responses/CustomServiceError'
        '5XX':
          $ref: '#/components/responses/InternalServerError'

  /filter-peerid:
    post:
      summary: Instruct the service about read messages that the client side consider bogus or malicious.
      description: One necessary component of pubsub is message filtering. This could be required for various reasons, ranging from technical (re-broadcasted message, maximum size ...) to malicious (spam, abuse ..). Some of those, the most generic, are expected to be handled by the service. On the other side of the spectrum, application logic and data is required to evaluate the validity of a message. This endpoint exist so that client can provide that feedback, which will allow the service to filter messages or prune bad peers.
      parameters:
        - $ref: "#/components/parameters/topic"
        - $ref: "#/components/parameters/peerid"
      responses:
        "202":
          description: "Service understood the instruction and will proceed"
        '400':
          $ref: '#/components/responses/BadRequest'
        '401':
          $ref: '#/components/responses/Unauthorized'
        '404':
          $ref: '#/components/responses/TopicNotFound'
        '409':
          $ref: '#/components/responses/InsufficientFunds'
        '429':
          $ref: '#/components/responses/TooManyRequests'
        '4XX':
          $ref: '#/components/responses/CustomServiceError'
        '5XX':
          $ref: '#/components/responses/InternalServerError'

  /list:
    get:
      summary: List subscribed topics
      parameters:
        - $ref: '#/components/parameters/filter-prefix'
        - $ref: '#/components/parameters/filter-suffix'
        - $ref: '#/components/parameters/max-topic'
        - $ref: '#/components/parameters/after-topic'
      responses:
        "200":
          description: "Returns a list of subscribed topic with characteristics"
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ListResponse'
        '400':
          $ref: '#/components/responses/BadRequest'
        '401':
          $ref: '#/components/responses/Unauthorized'
        '409':
          $ref: '#/components/responses/InsufficientFunds'
        '429':
          $ref: '#/components/responses/TooManyRequests'
        '4XX':
          $ref: '#/components/responses/CustomServiceError'
        '5XX':
          $ref: '#/components/responses/InternalServerError'

components:
  schemas:
    # Note, each config is optional, as the service can opt-in to not have limits
    DiscoveryResponse:
      description: Response for /discovery
      type: object
      properties:
        queue-length:
          $ref: '#/components/schemas/QueueLengthProperty'
        timeout:
          $ref: '#/components/schemas/TimeoutProperty'
        max-message-size:
          $ref: '#/components/schemas/MaxMessageSizeProperty'
        maximum-topic:
          $ref: '#/components/schemas/MaximumTopicProperty'

    JoinResponse:
      description: Response for /join
      type: object
      required:
        - queue-length
        - queue-policy
        - timeout
        - max-message-size
      properties:
        queue-length:
          $ref: '#/components/schemas/QueueLengthProperty'
        queue-policy:
          $ref: '#/components/schemas/QueuePolicyProperty'
        timeout:
          $ref: '#/components/schemas/TimeoutProperty'
        max-message-size:
          $ref: '#/components/schemas/MaxMessageSizeProperty'

    ReadResponse:
      description: response for /read
      type: object
      required:
        - messages-dropped
        - remaining-message
        - messages
      properties:
        messages-dropped:
          $ref: '#/components/schemas/MessagesDroppedProperty'
        remaining-message:
          $ref: '#/components/schemas/RemainingMessageProperty'
        messages:
          $ref: '#/components/schemas/MessagesProperty'

    ReadAllResponse:
      description: response for /read-all
      type: array
      items:
        required:
          - topic
          - messages-dropped
          - remaining-message
          - messages
        properties:
          topic:
            $ref: '#/components/schemas/TopicProperty'
          messages-dropped:
            $ref: '#/components/schemas/MessagesDroppedProperty'
          remaining-message:
            $ref: '#/components/schemas/RemainingMessageProperty'
          messages:
            $ref: '#/components/schemas/MessagesProperty'

    ListResponse:
      description: Response for /list
      type: array
      items:
        type: object
        required:
          - topic
        properties:
          topic:
            $ref: '#/components/schemas/TopicProperty'
          queue-length:
            $ref: '#/components/schemas/QueueLengthProperty'
          queue-policy:
            $ref: '#/components/schemas/QueuePolicyProperty'
          timeout:
            $ref: '#/components/schemas/TimeoutProperty'
          max-message-size:
            $ref: '#/components/schemas/MaxMessageSizeProperty'
          remaining-message:
            $ref: '#/components/schemas/RemainingMessageProperty'
          messages-dropped:
            $ref: '#/components/schemas/MessagesDroppedProperty'

    Failure:
      description: Response for a failed request
      type: object
      required:
        - error
      properties:
        error:
          type: object
          required:
            - reason
          properties:
            reason:
              type: string
              description: Mandatory string identifying the type of error
              example: "ERROR_CODE_FOR_MACHINES"
            details:
              type: string
              description: Optional, longer description of the error; may include UUID of transaction for support, links to documentation etc
              example: "Optional explanation for humans with more details"

    QueueLengthProperty:
      description: The maximum number of pubsub message stored by the service.
      type: integer
      format: int32
      minimum: 1
      example: 10

    TimeoutProperty:
      description: The maximum lifetime during which a pubsub topic is subscribed to after a /join, in seconds.
      type: integer
      format: int32
      minimum: 1
      example: 600

    MaxMessageSizeProperty:
      description: The maximum size of a single pubsub message, in byte.
      type: integer
      format: int32
      minimum: 1
      example: 100000

    MaximumTopicProperty:
      description: The maximum number of pubsub topic subscribed to by a client.
      type: integer
      format: int32
      minimum: 1
      example: 200

    QueuePolicyProperty:
      description: What happens when the number of pubsub message stored by the service fills up
      type: string
      enum:
        - drop-old
        - drop-new
      example: drop-old

    MessagesDroppedProperty:
      description: The number of message dropped due to queue size constraint
      type: integer
      format: int32
      minimum: 0
      example: 5

    RemainingMessageProperty:
      description: The number of remaining message in the read queue
      type: integer
      format: int32
      minimum: 0
      example: 7

    MessagesProperty:
      description: The read messages.
      type: array
      items:
        type: object
        required:
          - sender
          - content
        properties:
          sender:
            description: the Peer Id of the message's sender
            type: string
            example: "QmcXhKGEAJT9DwwzKeEZsWrYTTGkzBdgeKif2MdAXBswJv"
          content:
            description: the base64 encoded message's content,
            type: string
            example: "YWJj"
          pubkey:
            description: the public key associated to the sender's peer ID, formatted as a base64-encoded x509 DER public key
            format: byte
            example: "MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAmreiLkzVz2qlc2a4z8FbdUTzvka3vQVqPX0QRqr+Lgb6NOw0RVR8WTySavlg8zjOqgyRcEXMvmL1R34jMped68SBaRUyBBmvhT8ptzOZhZdSB4PPRwAfhILWyHnUm9JBPEmjN+Xi9wNt0WH8ds5S+RRnBoD1BagVxAykzjCJnZzAtKQYiU4jIG8zm7jwaWTZjukKaVsHAY7kjPF/8i2u3u+xltKGXYCwKbg621fTaEKBzC2mQORod9fw0qLhrOT/N1CuC5xCrlqvaDv8J4yKvJMd6XBG3lcsP/yhcCw9FjHdpolsRsomeSofvl7/OdaPMDimpdT012ohZAmPntUyNQIDAQAB"
          signature:
            description: the signature for that message
            format: byte
            example: "An7j/dPng3efQoyBMiFy/owSc5kb4pVteI9BZEdL67Krt/W4zEUidKuTi67X580p5dqrd/y04iODgZ99/6sMxb6/7h0++K/M9jkVr2544+0GT5I7fozyPZavbL5bMTy5azeWpZMgI8PiaPJChkHOaRelc7fuqeVB/lIds0ojUhBASOdReNOOMXMvhMjwe/Hh+jACXlJK0SraBV5UET4SPqy/NBhdYdkDOmoB7E2EWHWx6QQk80WPVEAZsKyVkCTz2QT3Hzb4HRxMpLYZ41ZeDggkWtnkG62X6faTX33tBwiCtO2Xac0awf56PxEeXCXhYBwi04Sk2HtV5GBo7gPeCw=="

    TopicProperty:
      description: The topic name
      type: string
      example: foo

  parameters:
    topic:
      description: The name of a pubsub topic
      name: topic
      in: query
      required: true
      schema:
        type: string
        minLength: 1
        maxLength: 1000
      example: "foo"

    queue-length:
      description: The maximum number of pubsub message stored by the service.
      name: queue-length
      in: query
      required: false
      schema:
        type: integer
        default: 20
      example: 20

    queue-policy:
      description: What happens when the number of pubsub message stored by the service fills up
      name: queue-policy
      in: query
      required: false
      schema:
        type: string
        enum:
          - drop-old
          - drop-new
      example: drop-old

    timeout:
      description: The maximum lifetime during which a pubsub topic is subscribed to after a /join, in seconds.
      name: timeout
      in: query
      required: false
      schema:
        type: integer
        default: 600
      example: 600

    max-message-size:
      description: The maximum length in byte for a pubsub message.
      name: max-message-size
      in: query
      required: false
      schema:
        type: integer
      example: 100000

    max-message:
      description: The maximum number of message to read.
      name: max-message
      in: query
      required: false
      schema:
        type: integer
      example: 5

    include-signature:
      description: If set, public key and signature for the received messages is also returned.
      name: include-signature
      in: query
      required: false
      schema:
        type: boolean
        default: false
      example: true

    max-topic:
      description: The maximum number of topic to return.
      name: max-topic
      in: query
      required: false
      schema:
        type: integer
      example: 5

    filter-prefix:
      description: A string to filter topics based on prefix-matching the name.
      name: filter-prefix
      in: query
      required: false
      schema:
        type: string
        minLength: 1
        maxLength: 1000
        example: '/foo/bar'

    filter-suffix:
      description: A string to filter topics based on suffix-matching the name.
      name: filter-suffix
      in: query
      required: false
      schema:
        type: string
        minLength: 1
        maxLength: 1000
        example: '/events'

    peerid:
      description: A libp2p peer identity.
      name: peerid
      in: query
      required: true
      schema:
        type: string
        maxLength: 1000
        example: 'QmdXGaeGiVA745XorV1jr11RHxB9z4fqykm6xCUPX1aTJo'

    after-topic:
      description: A pagination cursor to only list topics after a specific one.
      name: after-topic
      in: query
      required: false
      schema:
        type: string
        minLength: 1
        maxLength: 1000

  responses:
    TooManyTopic:
      description: Error response (Too many topics; the limit of number of subscribed topic by a client was reached)
      content:
        application/json:
          schema:
            $ref: '#/components/schemas/Failure'
          examples:
            BadRequestExample:
              $ref: '#/components/examples/TooManyTopicExample'

    MessageTooBig:
      description: Error response (Message too big; the given message size exceed the service limits)
      content:
        application/json:
          schema:
            $ref: '#/components/schemas/Failure'
          examples:
            BadRequestExample:
              $ref: '#/components/examples/MessageTooBigExample'

    BadRequest:
      description: Error response (Bad request)
      content:
        application/json:
          schema:
            $ref: '#/components/schemas/Failure'
          examples:
            BadRequestExample:
              $ref: '#/components/examples/BadRequestExample'

    Unauthorized:
      description: Error response (Unauthorized; access token is missing or invalid)
      content:
        application/json:
          schema:
            $ref: '#/components/schemas/Failure'
          examples:
            UnauthorizedExample:
              $ref: '#/components/examples/UnauthorizedExample'

    TopicNotFound:
      description: Error response (The specified topic was not found)
      content:
        application/json:
          schema:
            $ref: '#/components/schemas/Failure'
          examples:
            NotFoundExample:
              $ref: '#/components/examples/TopicNotFoundExample'

    InsufficientFunds:
      description: Error response (Insufficient funds)
      content:
        application/json:
          schema:
            $ref: '#/components/schemas/Failure'
          examples:
            InsufficientFundsExample:
              $ref: '#/components/examples/InsufficientFundsExample'

    TooManyRequests:
      description: Error response (Too many requests; rate limiting)
      headers:
        Retry-After:
          required: false
          description: hint from the server to only retry after a number of seconds
          example: 60
          schema:
            type: integer
            minimum: 1
      content:
        application/json:
          schema:
            $ref: "#/components/schemas/Failure"
          examples:
            TooManyRequestsExample:
              $ref: "#/components/examples/TooManyRequestsExample"

    CustomServiceError:
      description: Error response (Custom service error)
      content:
        application/json:
          schema:
            $ref: '#/components/schemas/Failure'
          examples:
            CustomServiceErrorExample:
              $ref: '#/components/examples/CustomServiceErrorExample'

    InternalServerError:
      description: Error response (Unexpected internal server error)
      content:
        application/json:
          schema:
            $ref: '#/components/schemas/Failure'
          examples:
            InternalServerErrorExample:
              $ref: '#/components/examples/InternalServerErrorExample'

  examples:
    TooManyTopicExample:
      value:
        error:
          reason: "TOO_MANY_TOPIC"
          details: "Too many topic are already subscribed"
      summary: Response when too many topics are already subscribed and the service refuse to subscribe to a new one

    MessageTooBigExample:
      value:
        error:
          reason: "MESSAGE_TOO_BIG"
          details: "The message is too big"
      summary: Response when the client ask to send a message that is too big for the topic configured limit.

    BadRequestExample:
      value:
        error:
          reason: "BAD_REQUEST"
          details: "Explanation for humans with more details"
      summary: A sample response to a bad request; reason will differ

    UnauthorizedExample:
      value:
        error:
          reason: "UNAUTHORIZED"
          details: "Access token is missing or invalid"
      summary: Response to an unauthorized request

    TopicNotFoundExample:
      value:
        error:
          reason: "TOPIC_NOT_FOUND"
          details: "The specified topic was not found"
      summary: Response to a request for a topic that is not joined

    InsufficientFundsExample:
      value:
        error:
          reason: "INSUFFICIENT_FUNDS"
          details: "Unable to process request due to the lack of funds"
      summary: Response when access token run out of funds

    TooManyRequestsExample:
      value:
        error:
          reason: "TOO_MANY_REQUESTS"
          details: "Request denied due to too many requests sent to the server"

    CustomServiceErrorExample:
      value:
        error:
          reason: "CUSTOM_ERROR_CODE_FOR_MACHINES"
          details: "Optional explanation for humans with more details"
      summary: Response when a custom error occurred

    InternalServerErrorExample:
      value:
        error:
          reason: "INTERNAL_SERVER_ERROR"
          details: "Explanation for humans with more details"
      summary: Response when unexpected error occurred