Gateway responds in JSON when requested

[markg85]

Rationale

Say you're building a command line application where you want to get a listing of files from a given CID. The gateway currently does that just fine in the browser! But that very same gateway can't be used on the command line because it outputs html.

My usecase

I'm still vigorously working on getting ipfs:// in media players, but i'm also working on getting encrypted data to be played.
The structure i'm crafting up right now is this:

<folder cid>
|   -> <random string> encrypted video file
|   -> <random string>.<first 16 byte of encrypted file>.json

In order to handle that folder structure i made a small bash script that "understands" that structure and fills in all the details in the mpv command line. So a command that used to look like:
mpv lavf://crypto:<cid> --stream-lavf-o=key=<aes key>,iv=<aes iv>
then looks like:
mpv $(wrapper ipfs://<folder cid>)

Note This is proof of concept "getting it working" logic. It should eventually go into a library which any media application should then use. And yes, the keys themselves should be encrypted too. WIP. But that's the future.

But for this to work i found myself in the tricky position of getting a folder listing on the command line. While that is possible when you have a local gateway running by using the http api (on port 5001), it's not ideal. It means you must have access to an http API which is much more limiting then having access to a gateway. This made me want the gateway to output it's folder listing as json data when requested.

Now i will change my scripts to basically add a third file linking the other two together and essentially say "CID X is the data, CID Y is the encryption details". This third file would be provided in the mpv line. It would make my case work just fine without any changes required to the gateway.

Honor the Accept header

The idea is simply to honor whatever the Accept header is from the client doing the request to the gateway. So if for example you pass -H "Accept: application/json" in a curl request to any given gateway then it's output should be in JSON.

I suppose this is the output of /api/v0/files/ls but i couldn't quite find that in the gateway codebase. It seems to be doing some calls on unixfs but i don't quite get how it works.

Regardless of internal specifics, the spec proposal should be clear enough.

[aschmahmann]

I have some good news, bad news, and more good news for you:

• good news 1: Your spec proposal seems to be about being able to get the output of /api/v0/files/ls with a JSON accept header when getting a UnixFS directory. There's no specification about what happens with files when you pass the header or anything like that, I assume since you mostly don't care about that. So you can just do something like https://ipfs.io/api/v0/ls?arg=/ipns/ipfs.io and you should be fine since it's already exposed on many gateways.
• bad news: Relying on /api/v0 endpoints on gateways is generally not great. Plenty of people do it for a variety of good reasons, but not every gateway exposes them and the goal is to get to a point where the important APIs people are using this way are handled in a more standardized way designed for gateways rather than just matching what go-ipfs internally uses for its HTTP RPC API.
• good news 2: There's interest in making this happen and generally having concrete specifications for IPFS gateways. An issue describing some proposals and directions here is Gateway support for /ipfs/{cid}?format=car|raw|... kubo#8234 (although you're right that that issue should really have been in this repo).

cc @lidel

[markg85]

Hi @aschmahmann

Great to see that there were ideas already :)
I'm a bit puzzled as to where the best place is to discuss it. The issue you mentioned is more detailed imho..

[lidel]

Let's continue in ipfs/kubo#8823 (it includes Logical Format of dag-pb being returned as dag-json)
When we have that, we will document new response types using IPIP process.