Paginated getLogs alternative

[sakulstra]

Currently working with eth_getLogs is painful in multiple ways. It has been painful 4 years ago when i first entered the space, and not much has improved since then.

• every client and thus node service provider(llamarpc, quicknode, drpc, alchemy) have different limitations on block-range.
• the actual behavior accross clients is different as well (some error on unbounded block ranges, some return zero results, alchemy does the custom thing of suggesting a block range)
• handling of reorgs is always a userland solution

With faster chains, the blockrange of e.g. alchemy(2_000) makes it essentially impossible to use eth_getLogs to index a contract in a reasonable amount of calls.

At the same time with bigger blocks some rpcs which implement limits based on nr of events, indexing all events could become impossible.

The big providers like alchemy and quicknode, now offer custom products (i think both call it streams) to index events and handle reorgs, but i feel like there should be an appropriate method on the node level not tied to a vendor lock-in.

---

Therefore, shouldn't it be possible to provide a paginated eth_getLogs not limited by an arbitrary block range?
I'm not suggesting any exact api here as i have far to little knowledge on the topic, but I guess some solution based on block-hash, should work?

Instead of providing a fromBlock number, i'd provide the last (exclusive)blockhash.
If there was a reorg, it should be easily detectable on the node and i could receive the reorged events as removed: true - if i don't provide a toBlock it return as many as possible and as part of the response i get a blockhash to continue with on the next call.

---

Sorry if this is the wrong place for starting this discussion, as reth was the innovator behind pushing ethereum/execution-apis#639 forward i guess it's a good idea to start the discussion here.

[sakulstra]

Did a bit more research on the topic, as i was curious why noone ever proposed this before.
It seems like people proposed it before - just not on ethmagicians and the other places i looked at:

• JSONRPC responses with cursor/paging ethereum/execution-apis#617 here is a proposal at execution-apis repo (i think all answers are spambots, but the initial proposal is legit)
• Proposal: Paginated Filter Queries ethereum/go-ethereum#17487 seems like someone proposed this years ago but it never got any traction