Blobs by range failures should penalize the peer

[eserilev]

Issue Addressed

Users in discord reported syncing issues on holesky

Oct 18 11:53:06.002 WARN Syncing deposit contract block cache    est_blocks_remaining: initializing deposits, service: slot_notifier
Oct 18 11:53:06.860 WARN Blobs missing in response to range request, slot: 2775009, block_root: 0xaf15bfd1aa990db73f83f204737c536ec581bac3c18a4b9074ef226acf727c92, service: beacon

Digging deeper into the logs it seems like LH keeps trying to query the same peer for the same range of blocks and blobs. It seems this peer just simply doesn't have the blobs available. The user reports that LH is connected to at least 30 peers so we should be trying a different peer after this failure occurs.

Proposed Changes

This PR updates our logic to penalize the peer for not having blobs available within the DA window. Alternatively find a way to retry the by range requests with a different peer.

The deneb spec mentions that

Peers that are unable to reply to blob sidecar requests within the range blob_serve_range SHOULD respond with error code 3: ResourceUnavailable. Such peers that are unable to successfully reply to this range of requests MAY get descored or disconnected at any time.

Note: The above requirement implies that nodes that start from a recent weak subjectivity checkpoint MUST backfill the local blobs database to at least the range blob_serve_range to be fully compliant with BlobSidecarsByRange requests.

Note: Although clients that bootstrap from a weak subjectivity checkpoint can begin participating in the networking immediately, other peers MAY disconnect and/or temporarily ban such an un-synced or semi-synced client.

So it seems that penalizing the peer is spec compliant

[dapplion]

I would like to go for a more complete solution like this one

• Stricter match of BlockError in lookup sync #6321

[dapplion]

lighthouse/beacon_node/beacon_chain/src/data_availability_checker/error.rs

Lines 40 to 41 in 6eaa370

	| Error::MissingBlobs
	| Error::MissingCustodyColumns

The root of the issue are these lines here that consider MissingBlobs an internal (not penalizable) fault

[eserilev]

Thanks for the additional info Lion, closing in favor of #6321