IPIP-378: Delegated Routing HTTP POST API

IPIP/0337-delegated-routing-http-api.md

@@ -86,7 +86,7 @@ and increasing data availability.
 IPFS Stewards will implement this API in [go-delegated-routing](https://github.com/ipfs/go-delegated-routing), using breaking changes in a new minor version.
 Because the existing Reframe spec can't be safely used in JavaScript and we won't be investing time and resources into changing the wire format implemented in edelweiss to fix it,
 the experimental support for Reframe in Kubo will be deprecated in the next release and delegated content routing will subsequently use this HTTP API.
-We may decide to re-add Reframe support in the future once these issues have been resolved.-
+We may decide to re-add Reframe support in the future once these issues have been resolved.
 
 #### Forwards Compatibility
 

IPIP/0378-delegated-routing-http-api-provide-records.md

@@ -0,0 +1,65 @@
+# IPIP-378: Delegated Content Routing HTTP Provide Records API
+
+- Start Date: 2023-02-14
+- Related Issues:
+  - https://github.com/ipfs/specs/pull/378
+
+## Summary
+
+This IPIP extends the [IPIP-337 HTTP Delegated Routing API](0337-delegated-routing-http-api.md) to provide records over `PUT` requests.
+
+The work here was originally proposed as part of IPIP-337, and eventually was separated into its own IPIP in order to reduce the scope of original work, while enabling iterative release of the HTTP delegated routing APIs.
+
+## Motivation
+
+The IPFS interaction with DHT includes both read and write operations.
+A user can provide records, advertising the presence of content, as well as looking up providers for a given CID.
+The specification proposed by [IPIP-337](0337-delegated-routing-http-api.md) offers an idiomatic first-class support for offloading the lookup portion of this interaction onto other processes and/or servers.
+Following the same motivations that inspired [IPIP-337](0337-delegated-routing-http-api.md), this document expands the HTTP APIs to also
+offload the ability to provide records noto a third-party system.
+
+## Detailed design
+
+The API extensions are added to the [Delegated Content Routing HTTP API spec/`PUT`](../routing/DELEGATED_CONTENT_ROUTING_HTTP.md#put-routingv1providers) section, along with complimentary sections that outline known formats followed by example payload.
+
+## Design rationale
+
+The rationale for the design of `PUT` operations closely follows the reasoning listed in [IPIP-337](0337-delegated-routing-http-api.md#design-rationale).
+The design uses a human-readable request/response structure with extensibility in mind.
+The specification imposes no restrictions on the schema nor the protocol advertised in provider records.
+The hope is that such extensibility will encourage and inspire innovation for better transfer protocols.
+In order to reduce barrier for adoption, the existing
+
+### User benefit
+
+Expanding the user benefits listed as part of [IPIP-337](0337-delegated-routing-http-api.md#user-benefit), in the context of content routing write operations are typically more expensive than read operations. They involve book keeping such as TTL, gossip propagation, etc.
+Therefore, it is highly desirable to reduce the burden of advertising provider records onto the network by means of delegation through simple to use HTTP APIs.
+
+### Compatibility
+
+#### Backwards Compatibility
+
+##### DHT
+
+The `PUT` APIs proposed here require a new data format for specifying provider records.
+Since the records must include a valid signature, records published through HTTP delegated routing must be resigned.
+
+##### Reframe
+
+See [IPIP-337/Backwards Compatibility](0337-delegated-routing-http-api.md#backwards-compatibility).
+
+#### Forwards Compatibility
+
+See [IPIP-337/Forwads Compatibility](0337-delegated-routing-http-api.md#forwards-compatibility).
+
+### Security
+
+See [IPIP-337/Security](0337-delegated-routing-http-api.md#security).
+
+### Alternatives
+
+- Reframe (general-purpose RPC) was evaluated, see "Design rationale" section for rationale why it was not selected.
+
+### Copyright
+
+Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).

routing/DELEGATED_CONTENT_ROUTING_HTTP.md

@@ -66,13 +66,13 @@ Specifications for some transfer protocols are provided in the "Transfer Protoco
 
 ### `GET /routing/v1/providers/{CID}`
 
-#### Response codes
+#### `GET` Response codes
 
 - `200` (OK): the response body contains 0 or more records
 - `404` (Not Found): must be returned if no matching records are found
 - `422` (Unprocessable Entity): request does not conform to schema or semantic constraints
 
-#### Response Body
+#### `GET` Response Body
 
 ```json
 {
@@ -90,6 +90,47 @@ Response limit: 100 providers
 
 Each object in the `Providers` list is a *read provider record*.
 
+### `PUT /routing/v1/providers`
+
+#### `PUT` Response codes
+
+- `200` (OK): the server processed the full list of provider records (possibly unsuccessfully, depending on the semantics of the particular records)
+- `400` (Bad Request): the server deems the request to be invalid and cannot process it
+- `422` (Unprocessable Entity): request does not conform to schema or semantic constraints
+- `501` (Not Implemented): the server does not support providing records
+
+#### `PUT` Request Body
+
+```json
+{
+    "Providers": [
+        {
+            "Protocol": "<protocol_name>",
+            "Schema": "bitswap",
+            ...
+        }
+    ]
+}
+```
+
+Each object in the `Providers` list is a *write provider record*.
+
+#### `PUT` Response Body
+
+  ```json
+  {
+      "ProvideResults": [
+          { ... }
+      ]
+  }
+  ```
+
+- `ProvideResults` is a list of results in the same order as the `Providers` in the request, and the schema of each object is determined by the `Protocol` of the corresponding write object (called "Write Provider Records Response" in the Known Transfer Protocols section)
+  - This may contain output information such as TTLs, errors, etc.
+  - It is undefined whether the server will allow partial results
+- The work for processing each provider record should be idempotent so that it can be retried without excessive cost in the case of full or partial failure of the request
+- Default limit of 100 keys per request
+
 ## Pagination
 
 This API does not support pagination, but optional pagination can be added in a backwards-compatible spec update.
@@ -118,7 +159,7 @@ limits, allowing every site to query the API for results:
 
 ```plaintext
 Access-Control-Allow-Origin: *
-Access-Control-Allow-Methods: GET, OPTIONS
+Access-Control-Allow-Methods: GET, PUT, OPTIONS
 ```
 
 ## Known Transfer Protocols
@@ -148,6 +189,60 @@ Specification: [ipfs/specs/BITSWAP.md](https://github.com/ipfs/specs/blob/main/B
 
 The server should respect a passed `transport` query parameter by filtering against the `Addrs` list.
 
+#### Bitswap Write Provider Records
+
+```json
+{
+    "Protocol": "transport-bitswap",
+    "Schema": "bitswap",
+    "Signature": "<signature>",
+    "Payload": "<payload>"
+}
+```
+
+- `Signature`: a multibase-encoded signature of the sha256 hash of the `Payload` field, signed using the private key of the Peer ID specified in the `Payload` JSON. Signing details for specific key types should follow [libp2p/peerid specs](https://github.com/libp2p/specs/blob/master/peer-ids/peer-ids.md#key-types), unless stated otherwise.
+
+  - Servers may ignore this field if they do not require signature verification.
+- `Payload`: a string containing a serialized JSON object which conforms with the following schema:
+
+  ```json
+   {
+       "Keys": ["cid1", "cid2"],
+       "Timestamp": 0,
+       "AdvisoryTTL": 0,
+       "ID": "12D3K...",
+       "Addrs": ["/ip4/..."]
+   }
+   ```
+
+- `Keys` is a list of the CIDs being provided
+- `Timestamp` is the current time
+- `AdvisoryTTL` is the time by which the caller expects the server to keep the record available
+  - If this value is unknown, the caller may use a value of 0
+- `ID` is the peer ID that was used to sign the record
+- `Addrs` is a list of string-encoded multiaddrs
+
+A [400 Bad Request](https://httpwg.org/specs/rfc9110.html#status.400)  response code should be returned if the signature check fails.
+
+Note that this only supports Peer IDs expressed as identity multihashes. Peer IDs with older key types that exceed 42 bytes are not verifiable since they only contain a hash of the key, not the key itself.
+Normally, if the Peer ID contains only a hash of the key, then the key is obtained out-of-band (e.g. by fetching the block via IPFS).
+If support for these Peer IDs is needed in the future, this spec can be updated to allow the client to provide the key and key type out-of-band by adding optional `PublicKey` and `PublicKeyType` fields, and if the Peer ID is a CID, then the server can verify the public key's authenticity against the CID, and then proceed with the rest of the verification scheme.
+
+The `Payload` field is a string, not a proper JSON object, to prevent its contents from being accidentally parsed and re-encoded by intermediaries, which may change the order of JSON fields and thus cause the record to fail validation.
+
+#### Write Provider Records Response
+
+```json
+{
+    "AdvisoryTTL": 0
+}
+```
+
+- `AdvisoryTTL` is the time at which the server expects itself to drop the record
+  - If less than the `AdvisoryTTL` in the request, then the client should re-issue the request by that point
+  - If greater than the `AdvisoryTTL` in the request, then the server expects the client to be responsible for the content for up to that amount of time (TODO: this is ambiguous)
+  - If 0, the server makes no claims about the lifetime of the record
+
 ### Filecoin Graphsync
 
 Multicodec name: `transport-graphsync-filecoinv1`
@@ -173,3 +268,7 @@ Specification: [ipfs/go-graphsync/blob/main/docs/architecture.md](https://github
 - `PieceCID`: the CID of the [piece](https://spec.filecoin.io/systems/filecoin_files/piece/#section-systems.filecoin_files.piece) within which the data is stored
 - `VerifiedDeal`: whether the deal corresponding to the data is verified
 - `FastRetrieval`: whether the provider claims there is an unsealed copy of the data available for fast retrieval
+
+#### Filecoin Graphsync Write Provider Records
+
+There is currently no specified schema.