Add getBlobs endpoint

[nflaig]

As suggested in #524 (comment), adds new getBlobs endpoint which only returns blobs.

[rolfyone]

LGTM

[jimmygchen]

I'm not sure how useful is it for users to get a list of unordered blobs without indices. If this is intended for L2s, we might as well implement retrieval by version hashes?
#332

[nflaig]

seems like from the previous discussion it might be better, would we still keep the block_id in that case as noted in #332 (comment)? We index data columns sidecars by block root or slot so dropping this would require additional indexing

[jimmygchen]

yep you're right, I'm bringing this up because the intent of this endpoint is for L2 users - I'm not exactly sure how they use it, but i assume they have block_id and version_hash, and so they can retrieve the blob they need?

With the getBlobSidecar endpoint, they're able to do this by computing the version_hash of individual sidecars from kzg_commitment - but without the sidecar metadata this won't work anymore.

I see a few possible options:

1. return KZG commitments / version hash together with the blob
2. implement version hash filtering on the server side
3. returned an ordered list, so they're able to figure out which blobs they're after from the block.kzg_commitments

I think #3 may be the least work on the beacon node, and also keep the API clean?

[nflaig]

but i assume they have block_id and version_hash, and so they can retrieve the blob they need?

Not sure how they use it either but since the blob sidecars endpoint required block_id already it should be fine

I think #3 may be the least work on the beacon node, and also keep the API clean?

(3) is the simplest for us and that's how I implemented it for now but using version_hashes for filtering should be easy change, if we go with (3) it makes sense to add explicit note that an ordered list must be returned

[Thegaram]

For Scroll, it would be more convenient to have a "get blob(s) by versioned hash(es)" endpoint (without block id).

[tersec]

It seems like from @Tristan-Wilson 's description slot numbers have concrete debugging and diagnostic advantages, too.

[Thegaram]

I think it's best to keep block_id

In that case, from our perspective, I don't see much benefit to adding a new API.

Or is the idea here that later when we'll have much more blobs in each block, the response would grow too large? In that case, filtering based on versioned hashes would be useful.

[nflaig]

In that case, from our perspective, I don't see much benefit to adding a new API.

the reason for the new api is that we no longer use and store BlobSidecars on the CL side once Fusaka goes live, see #524 (comment)

Or is the idea here that later when we'll have much more blobs in each block, the response would grow too large?

yeah, just to avoid returning data you don't need and with increasing blob count this matters more

[Enigmatic331]

Chiming in a little - I understand the OP folks are having some issues with plans to deprecate the blob_sidecars endpoint

Fulu deprecates the /eth/v1/beacon/blob_sidecars Beacon node endpoint, replacing it with /eth/v1/beacon/blobs. However, this new endpoint only provides the raw blobs, without the new kzg commitments (for cell proofs), without which verification doesn't work. So we need to investigate how we want to support this new endpoint for untrusted beacon nodes.

This is an endpoint we are currently using in production for data indexing as well, and would appreciate if it could be retained.

[nflaig]

This is an endpoint we are currently using in production for data indexing as well, and would appreciate if it could be retained.

The reason for deprecating the blob_sidecars endpoint is because blob sidecars no longer exist after Fulu, we only have data column sidecars.

I'd rather update the response of the blobs endpoint to include additional data as previously suggested by @jimmygchen

[james-prysm]

going to check with l2s for this endpoint

[nflaig]

Pushed some updates to the PR based on feedback

• use versioned_hashes instead of indices as filter (in addition to block_id)
• updated description that the blobs are ordered based on the order of kzg commitments in the block

[rolfyone]

looks like the discussion is completed here, approving