Fusaka Defense(beacon client): support `/eth/v1/beacon/blobs`

[sebastianst]

Description

Fulu deprecates the /eth/v1/beacon/blob_sidecars Beacon node endpoint, replacing it with /eth/v1/beacon/blobs. However, this new endpoint only provides the raw blobs, without the new kzg commitments (for cell proofs), without which verification doesn't work. So we need to investigate how we want to support this new endpoint for untrusted beacon nodes.

See below discussion for more details as they emerge.

Original Description

Our derivation needs to be updated to do cell proof verifications.

Inside op-service/sources/l1_beacon_client.go there's blobsFromSidecars, which calls eth.VerifyBlobProof, which just forwards to kzg4844.VerifyBlobProof, but we probably need to switch on the active L1 fork, (or just inspect the number of proofs as a heuristic) to call kzg4844.VerifyCellProofs instead (it's a batched verification).

[geoknee]

I don't think it as simple as switching to cell proofs. We have been working on the assumption that there is no change to the beacon API and therefore no change needed in our beacon client -- see these notes from @protolambda.
Inspecting the response from lighthouse , we see that blob 0 has:

"kzg_proof": "0xc00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000",

The data type is indicative of a single blob proof, not multiple cell proofs. But it is obviously not correct.

There is a new endpoint /eth/v1/debug/beacon/data_column_sidecars/{block_id} in a prerelease of the beacon API, but it's in the debug namespace and not designed to be exposed by https://github.com/ethereum/beacon-APIs/releases/tag/v4.0.0-alpha.1

Our e2e tests mock out the beacon node, so we need to update that mock to reflect any changes with Fulu; but we need to be sure what those changes are (I still think by the beacon specs there should be no changes).

TLDR this could be a bug in lighthouse, or it could be that we need a beacon node with --supernode set (see e.g. these release notes https://github.com/sigp/lighthouse/releases/tag/v8.0.0-rc.0) (edited)

However, if I query a devnet 3 beacon endpoint curl https://beacon.fusaka-devnet-3.ethpandaops.io/eth/v1/beacon/blob_sidecars/head I get

{
  "code": 500,
  "message": "INTERNAL_SERVER_ERROR: Insufficient data columns to reconstruct blobs: required 64, but only 9 were found.",
  "stacktraces": []
}

Which is what I would expect for a non supernode beacon node. Returning all the blobs but an incorrect blob proof does seem like a bug.

[geoknee]

It seems that the /eth/v1/beacon/blob_sidecars/ API we rely on it being deprecated with Fulu.

This API should be used instead ethereum/beacon-APIs#546, although it is not yet running on testnets.

Have updated the ticket title accordingly.

[geoknee]

Of note: this endpoint does not expose a blob proof or cell proof. So we won't be able to validate the blobs, meaning that we must trust the beacon endpoint.

We may be able to use other endpoints to pull cell proofs from the beacon endpoint, but this is TBC and there is no official, non-debug endpoint available to my knowledge.

[pandainzoo]

Launch the Lighthouse client with the --supernode startup parameter, after the Fusaka upgrade, queries through the getBlob interface still return an invalid value for the kzg_proof field. this is their implementation
https://github.com/sigp/lighthouse/blob/stable/beacon_node/beacon_chain/src/kzg_utils.rs#L352
and temporarily bypassing blob verification might be a workaround solution.

[niran]

Instead of relying on the beacon node to validate the blob and skipping blob verification when that's not possible, can't we just regenerate the KZG commitment, derive the versioned hash, and compare it to what's in the blob transaction? That seems to be how op-challenger handles this already.

[geoknee]

Yes I think you are right @niran, the KZG proof shouldn't be necessary if we have the whole blob, it's not really designed for the simple task of checking a blob matches a commitment. I think the current way we verify blobs in op-node is actually overdoing it.