Skip to content

Commit

Permalink
chore(ipip-410): final editorials
Browse files Browse the repository at this point in the history
  • Loading branch information
lidel committed May 31, 2023
1 parent e55ca4a commit e55e319
Show file tree
Hide file tree
Showing 2 changed files with 35 additions and 12 deletions.
14 changes: 13 additions & 1 deletion src/ipips/ipip-0410.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
title: "IPIP-0410: Streaming Routing V1 HTTP API"
title: "IPIP-0410: Streaming NDJSON in Routing HTTP API"
date: 2023-05-12
ipip: proposal
editors:
Expand All @@ -15,6 +15,8 @@ order: 410
tags: ['ipips']
---

## Summary

Introduce backwards-compatible streaming support to the Routing V1 HTTP API.
For this, we use the `Accept` HTTP header (:cite[rfc9110]) for content type negotiation, as well
as the Newline Delimited JSON ([NDJSON]) format.
Expand All @@ -27,6 +29,9 @@ the client requests a list of providers for a CID from the server. Then, the cli
has to wait for the server to collect their final list of providers. After that,
the server can respond with the full list of providers.

This is a big source of latency when `/routing/v1` is used for delegating DHT lookups,
where the client is forced to wait for the server to finish DHT walk.

With streaming support, the server is able to respond with provider records as soon
as they are available. This reduces latency and allows for faster content discovery.

Expand Down Expand Up @@ -65,6 +70,13 @@ to respond more promptly to provider record requests. Instead of waiting for the
list to be constructed, servers can now return each provider record one by one,
in a streaming fashion.

The client will be able to close connection at any time, reducing load on both ends.

The main use cases for this IPIP are light clients and services which want to
delegate DHT lookups to external service. With streaming, clients will be able
to receive results as soon the delegated service learns about new record, which
directly impacts the content load speeds perceived by the end user.

### Compatibility

The introduced changes are backwards-compatible. The introduced header is completely
Expand Down
33 changes: 22 additions & 11 deletions src/routing/http-routing-v1.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ order: 0
tags: ['routing']
---

Delegated routing is a mechanism for IPFS implementations to use for offloading content routing and naming to another process/server. This specification describes an HTTP API for delegated content routing.
Delegated routing is a mechanism for IPFS implementations to use for offloading content routing and naming to another process/server. This specification describes a vendor-agnostic HTTP API for delegated content routing.

## API Specification

Expand Down Expand Up @@ -147,23 +147,34 @@ This API does not support pagination, but optional pagination can be added in a

## Streaming

JSON-based endpoints support streaming requests made by sending an `Accept` HTTP Header containing
`application/x-ndjson`. The response will be formatted as [Newline Delimited JSON (ndjson)](https://github.com/ndjson/ndjson-spec),
with one *read provider record* per line:
JSON-based endpoints support streaming requests made
with `Accept: application/x-ndjson` HTTP Header.

Steaming responses are formatted as
[Newline Delimited JSON (ndjson)](https://github.com/ndjson/ndjson-spec),
with one result per line:

```json
{"Protocol": "<protocol_name>", "Schema": "<schema>", ...}
{"Protocol": "<protocol_name>", "Schema": "<schema>", ...}
{"Protocol": "<protocol_name>", "Schema": "<schema>", ...}
{"Schema": "<schema>", ...}
{"Schema": "<schema>", ...}
{"Schema": "<schema>", ...}
...
```

Streaming is backwards-compatible with clients that do not support streaming. Please note the following:
:::note

- Requests without an `Accept` header MUST default to regular, non-streaming, responses.
- The server MAY respond with non-streaming response even if the client requests streaming.
- The server MUST NOT respond with streaming response if the client did not request so.
Streaming is opt-in and backwards-compatibile with clients and servers that do
not support streaming:

- Requests without the `Accept: application/x-ndjson` header MUST default to
regular, non-streaming, JSON responses.
- Legacy server MAY respond with non-streaming `application/json` response even
if the client requested streaming. It is up to the client to inspect
the `Content-Type` header before parsing the response.
- The server MUST NOT respond with streaming response if the client did not
explicitly request so.

:::

## Error Codes

Expand Down

0 comments on commit e55e319

Please sign in to comment.