Disable batching? ("Too many eth_getLogs methods in the batch")

[therealjmj]

Ethers Version

6.4.2

Search Terms

No response

Describe the Problem

Ethers crashes the program during initial connection (contract.on listener assignment), with the following error.

Using a set of polling JsonRpcProviders within a FallbackProvider.

Is there a way to limit the requests, so we don't hit this error?

Code Snippet

Using the following config:

const config: FallbackProviderJsonConfig = {
  chainId: 137,
  providers: [
    {
      provider: 'https://rpc.ankr.com/polygon',
      priority: 2,
      weight: 2,
      stallTimeout: 2500,
    },
    {
      provider: 'https://polygon-rpc.com',
      priority: 3,
      weight: 1,
    },
    {
      provider: 'https://rpc-mainnet.maticvigil.com',
      priority: 3,
      weight: 1,
    },
  ],
};

Contract ABI

No response

Errors

Error: no response from server (value=[ { "error": { "code": -32600, "message": "Too many [\"eth_getLogs\"] methods in the batch" }, "id": null, "jsonrpc": "2.0" } ], info={ "payload": { "id": 29, "jsonrpc": "2.0", "method": "eth_chainId", "params": [  ] } }, code=BAD_DATA, version=6.4.0)

Environment

No response

Environment (Other)

No response

[ricmoo]

The provider property must be a Provider, not a JSON-RPC string, but my guess is that your backends do not support batching. You can use the following for your configuration:

// Create your network
const network = Network.from(137)

// Return a Provider with batching disabled and with static network optimizations enabled
function getProvider(url) {
    return new ethers.JsonRpcProvider(url, null, { batchMaxSize: 1, staticNetwork: network });
}

const backends = [
    {
      provider: getProvider('https://rpc.ankr.com/polygon'),
      priority: 2,
      weight: 2,
      stallTimeout: 2500,
    },
    {
      provider: getProvider('https://polygon-rpc.com'),
      priority: 3,
      weight: 1,
    },
    {
      provider: getProvider('https://rpc-mainnet.maticvigil.com'),
      priority: 3,
      weight: 1,
    },
];

That will provide you with the best performing FallbackProvider for your purposes, I believe.

[therealjmj]

Thanks! However, even with the above JsonRpcProvider settings, I'm still getting this error.

It's causing a full app crash, which is also kind of an unexpected behavior.

Would there be a difference with setting isPolling: true, and multiple JsonRpcProviders inside a FallbackProvider?

––

Error: missing response for request (value=[ { "error": { "code": -32600, "message": "Too many [\"eth_getLogs\"] methods in the batch" }, "id": null, "jsonrpc": "2.0" } ], info={ "payload": { "id": 40, "jsonrpc": "2.0", "method": "eth_chainId", "params": [  ] } }, code=BAD_DATA, version=6.5.1)
    at makeError (.../ethers@6.5.1/node_modules/ethers/lib.commonjs/utils/errors.js:125:21)
    at .../ethers@6.5.1/node_modules/ethers/lib.commonjs/providers/provider-jsonrpc.js:257:72
    at processTicksAndRejections (node:internal/process/task_queues:96:5) {
  code: 'BAD_DATA',
  value: [
    {
      jsonrpc: '2.0',
      error: {
        code: -32600,
        message: 'Too many ["eth_getLogs"] methods in the batch'
      },
      id: null
    }
  ],
  info: {
    payload: { method: 'eth_chainId', params: [], id: 40, jsonrpc: '2.0' }
  }
}```

[ricmoo]

That payload is not a batch though, it indicates the payload in the error. Does the backend work for getLogs normally? Maybe they have disabled logs entirely and the message is just deceiving?

A FallbackProvider uses polling internally regardless right now…

[ricmoo]

Actually. Looking at the payload, it isn’t even a getLogs request…

[therealjmj]

Right, it's strange behavior. Our RPCs are actually getting completely overrun with these kinds of failing 'batch' requests.
Millions of requests on Polygon from a couple of test frontends. I wonder if it's related.

[ricmoo]

Could it be that getLogs is entirely disabled on those nodes and the error message is a red herring? I know some of the fast networks have disabled getLogs.

[therealjmj]

[therealjmj]

@ricmoo

This is a surprising batch request that was triggered through Ethers v6.5.1... especially since we have staticNetwork set for this JsonRpcProvider on Polygon. I wouldn't expect it to request chainId, and requesting more than once is definitely wrong.

I wonder if the 'payload' from the original post (above) is only showing the first index of the batch? That would explain why the payload shows a chainId request, and the error itself shows getLogs.

[therealjmj]

Also, the overall request fails with "Bad parameters in JSONRPC request: eth_getLogs".

Here are the params for an example request:
params: [{address: "0x19b620929f97b7b990801496c3b361ca5def8c71", fromBlock: "0x29cd3a9", toBlock: "0x29cd3e5"}]

It seems strange that the getLogs params would be formatted as an array with a single object - could that be the problem?

[ricmoo]

It is weird it is asking for the chain ID at all, little line within the same batch. I can try connecting to those endpoints and see what I get with that getLogs request.

[therealjmj]

Here's another request made with Ethers v6.5.1.

Not only are there dozens of chainId requests, and 60 requests total, but there are tons of duplicate getLogs requests as well.

This results in the error "Too many eth_getLogs methods in the batch".

[
    {
        "method": "eth_chainId",
        "params": [],
        "id": 2644,
        "jsonrpc": "2.0"
    },
    {
        "method": "eth_chainId",
        "params": [],
        "id": 2645,
        "jsonrpc": "2.0"
    },
    {
        "method": "eth_getLogs",
        "params": [
            {
                "address": "0x19b620929f97b7b990801496c3b361ca5def8c71",
                "topics": [
                    "0x781745c57906dc2f175fec80a9c691744c91c48a34a83672c41c2604774eb11f"
                ],
                "fromBlock": "0x29cd945",
                "toBlock": "0x29cd981"
            }
        ],
        "id": 2646,
        "jsonrpc": "2.0"
    },
    {
        "method": "eth_chainId",
        "params": [],
        "id": 2647,
        "jsonrpc": "2.0"
    },
    {
        "method": "eth_chainId",
        "params": [],
        "id": 2648,
        "jsonrpc": "2.0"
    },
    {
        "method": "eth_getLogs",
        "params": [
            {
                "address": "0x19b620929f97b7b990801496c3b361ca5def8c71",
                "topics": [
                    "0x3a5b9dc26075a3801a6ddccf95fec485bb7500a91b44cec1add984c21ee6db3b"
                ],
                "fromBlock": "0x29cd945",
                "toBlock": "0x29cd981"
            }
        ],
        "id": 2649,
        "jsonrpc": "2.0"
    },
    {
        "method": "eth_chainId",
        "params": [],
        "id": 2650,
        "jsonrpc": "2.0"
    },
    {
        "method": "eth_chainId",
        "params": [],
        "id": 2651,
        "jsonrpc": "2.0"
    },
    {
        "method": "eth_getLogs",
        "params": [
            {
                "address": "0x19b620929f97b7b990801496c3b361ca5def8c71",
                "topics": [
                    "0x56a618cda1e34057b7f849a5792f6c8587a2dbe11c83d0254e72cb3daffda7d1"
                ],
                "fromBlock": "0x29cd945",
                "toBlock": "0x29cd981"
            }
        ],
        "id": 2652,
        "jsonrpc": "2.0"
    },
    {
        "method": "eth_chainId",
        "params": [],
        "id": 2653,
        "jsonrpc": "2.0"
    },
    {
        "method": "eth_chainId",
        "params": [],
        "id": 2654,
        "jsonrpc": "2.0"
    },
    {
        "method": "eth_getLogs",
        "params": [
            {
                "address": "0x19b620929f97b7b990801496c3b361ca5def8c71",
                "topics": [
                    "0xd93cf895c7d5b2cd7dc7a098b678b3089f37d91f48d9b83a0800a91cbdf05284"
                ],
                "fromBlock": "0x29cd945",
                "toBlock": "0x29cd981"
            }
        ],
        "id": 2655,
        "jsonrpc": "2.0"
    },
    {
        "method": "eth_chainId",
        "params": [],
        "id": 2656,
        "jsonrpc": "2.0"
    },
    {
        "method": "eth_chainId",
        "params": [],
        "id": 2657,
        "jsonrpc": "2.0"
    },
    {
        "method": "eth_getLogs",
        "params": [
            {
                "address": "0x19b620929f97b7b990801496c3b361ca5def8c71",
                "topics": [
                    "0x781745c57906dc2f175fec80a9c691744c91c48a34a83672c41c2604774eb11f"
                ],
                "fromBlock": "0x29cd945",
                "toBlock": "0x29cd982"
            }
        ],
        "id": 2658,
        "jsonrpc": "2.0"
    },
    {
        "method": "eth_chainId",
        "params": [],
        "id": 2659,
        "jsonrpc": "2.0"
    },
    {
        "method": "eth_chainId",
        "params": [],
        "id": 2660,
        "jsonrpc": "2.0"
    },
    {
        "method": "eth_getLogs",
        "params": [
            {
                "address": "0x19b620929f97b7b990801496c3b361ca5def8c71",
                "topics": [
                    "0x3a5b9dc26075a3801a6ddccf95fec485bb7500a91b44cec1add984c21ee6db3b"
                ],
                "fromBlock": "0x29cd945",
                "toBlock": "0x29cd982"
            }
        ],
        "id": 2661,
        "jsonrpc": "2.0"
    },
    {
        "method": "eth_chainId",
        "params": [],
        "id": 2662,
        "jsonrpc": "2.0"
    },
    {
        "method": "eth_chainId",
        "params": [],
        "id": 2663,
        "jsonrpc": "2.0"
    },
    {
        "method": "eth_getLogs",
        "params": [
            {
                "address": "0x19b620929f97b7b990801496c3b361ca5def8c71",
                "topics": [
                    "0x56a618cda1e34057b7f849a5792f6c8587a2dbe11c83d0254e72cb3daffda7d1"
                ],
                "fromBlock": "0x29cd945",
                "toBlock": "0x29cd982"
            }
        ],
        "id": 2664,
        "jsonrpc": "2.0"
    },
    {
        "method": "eth_chainId",
        "params": [],
        "id": 2665,
        "jsonrpc": "2.0"
    },
    {
        "method": "eth_chainId",
        "params": [],
        "id": 2666,
        "jsonrpc": "2.0"
    },
    {
        "method": "eth_getLogs",
        "params": [
            {
                "address": "0x19b620929f97b7b990801496c3b361ca5def8c71",
                "topics": [
                    "0xd93cf895c7d5b2cd7dc7a098b678b3089f37d91f48d9b83a0800a91cbdf05284"
                ],
                "fromBlock": "0x29cd945",
                "toBlock": "0x29cd982"
            }
        ],
        "id": 2667,
        "jsonrpc": "2.0"
    },
    {
        "method": "eth_chainId",
        "params": [],
        "id": 2668,
        "jsonrpc": "2.0"
    },
    {
        "method": "eth_chainId",
        "params": [],
        "id": 2669,
        "jsonrpc": "2.0"
    },
    {
        "method": "eth_getLogs",
        "params": [
            {
                "address": "0x19b620929f97b7b990801496c3b361ca5def8c71",
                "topics": [
                    "0x781745c57906dc2f175fec80a9c691744c91c48a34a83672c41c2604774eb11f"
                ],
                "fromBlock": "0x29cd945",
                "toBlock": "0x29cd983"
            }
        ],
        "id": 2670,
        "jsonrpc": "2.0"
    },
    {
        "method": "eth_chainId",
        "params": [],
        "id": 2671,
        "jsonrpc": "2.0"
    },
    {
        "method": "eth_chainId",
        "params": [],
        "id": 2672,
        "jsonrpc": "2.0"
    },
    {
        "method": "eth_getLogs",
        "params": [
            {
                "address": "0x19b620929f97b7b990801496c3b361ca5def8c71",
                "topics": [
                    "0x3a5b9dc26075a3801a6ddccf95fec485bb7500a91b44cec1add984c21ee6db3b"
                ],
                "fromBlock": "0x29cd945",
                "toBlock": "0x29cd983"
            }
        ],
        "id": 2673,
        "jsonrpc": "2.0"
    },
    {
        "method": "eth_chainId",
        "params": [],
        "id": 2674,
        "jsonrpc": "2.0"
    },
    {
        "method": "eth_chainId",
        "params": [],
        "id": 2675,
        "jsonrpc": "2.0"
    },
    {
        "method": "eth_getLogs",
        "params": [
            {
                "address": "0x19b620929f97b7b990801496c3b361ca5def8c71",
                "topics": [
                    "0x56a618cda1e34057b7f849a5792f6c8587a2dbe11c83d0254e72cb3daffda7d1"
                ],
                "fromBlock": "0x29cd945",
                "toBlock": "0x29cd983"
            }
        ],
        "id": 2676,
        "jsonrpc": "2.0"
    },
    {
        "method": "eth_chainId",
        "params": [],
        "id": 2677,
        "jsonrpc": "2.0"
    },
    {
        "method": "eth_chainId",
        "params": [],
        "id": 2678,
        "jsonrpc": "2.0"
    },
    {
        "method": "eth_getLogs",
        "params": [
            {
                "address": "0x19b620929f97b7b990801496c3b361ca5def8c71",
                "topics": [
                    "0xd93cf895c7d5b2cd7dc7a098b678b3089f37d91f48d9b83a0800a91cbdf05284"
                ],
                "fromBlock": "0x29cd945",
                "toBlock": "0x29cd983"
            }
        ],
        "id": 2679,
        "jsonrpc": "2.0"
    },
    {
        "method": "eth_chainId",
        "params": [],
        "id": 2680,
        "jsonrpc": "2.0"
    },
    {
        "method": "eth_chainId",
        "params": [],
        "id": 2681,
        "jsonrpc": "2.0"
    },
    {
        "method": "eth_getLogs",
        "params": [
            {
                "address": "0x19b620929f97b7b990801496c3b361ca5def8c71",
                "topics": [
                    "0x781745c57906dc2f175fec80a9c691744c91c48a34a83672c41c2604774eb11f"
                ],
                "fromBlock": "0x29cd945",
                "toBlock": "0x29cd984"
            }
        ],
        "id": 2682,
        "jsonrpc": "2.0"
    },
    {
        "method": "eth_chainId",
        "params": [],
        "id": 2683,
        "jsonrpc": "2.0"
    },
    {
        "method": "eth_chainId",
        "params": [],
        "id": 2684,
        "jsonrpc": "2.0"
    },
    {
        "method": "eth_getLogs",
        "params": [
            {
                "address": "0x19b620929f97b7b990801496c3b361ca5def8c71",
                "topics": [
                    "0x3a5b9dc26075a3801a6ddccf95fec485bb7500a91b44cec1add984c21ee6db3b"
                ],
                "fromBlock": "0x29cd945",
                "toBlock": "0x29cd984"
            }
        ],
        "id": 2685,
        "jsonrpc": "2.0"
    },
    {
        "method": "eth_chainId",
        "params": [],
        "id": 2686,
        "jsonrpc": "2.0"
    },
    {
        "method": "eth_chainId",
        "params": [],
        "id": 2687,
        "jsonrpc": "2.0"
    },
    {
        "method": "eth_getLogs",
        "params": [
            {
                "address": "0x19b620929f97b7b990801496c3b361ca5def8c71",
                "topics": [
                    "0x56a618cda1e34057b7f849a5792f6c8587a2dbe11c83d0254e72cb3daffda7d1"
                ],
                "fromBlock": "0x29cd945",
                "toBlock": "0x29cd984"
            }
        ],
        "id": 2688,
        "jsonrpc": "2.0"
    },
    {
        "method": "eth_chainId",
        "params": [],
        "id": 2689,
        "jsonrpc": "2.0"
    },
    {
        "method": "eth_chainId",
        "params": [],
        "id": 2690,
        "jsonrpc": "2.0"
    },
    {
        "method": "eth_getLogs",
        "params": [
            {
                "address": "0x19b620929f97b7b990801496c3b361ca5def8c71",
                "topics": [
                    "0xd93cf895c7d5b2cd7dc7a098b678b3089f37d91f48d9b83a0800a91cbdf05284"
                ],
                "fromBlock": "0x29cd945",
                "toBlock": "0x29cd984"
            }
        ],
        "id": 2691,
        "jsonrpc": "2.0"
    },
    {
        "method": "eth_chainId",
        "params": [],
        "id": 2692,
        "jsonrpc": "2.0"
    },
    {
        "method": "eth_chainId",
        "params": [],
        "id": 2693,
        "jsonrpc": "2.0"
    },
    {
        "method": "eth_getLogs",
        "params": [
            {
                "address": "0x19b620929f97b7b990801496c3b361ca5def8c71",
                "topics": [
                    "0x781745c57906dc2f175fec80a9c691744c91c48a34a83672c41c2604774eb11f"
                ],
                "fromBlock": "0x29cd945",
                "toBlock": "0x29cd985"
            }
        ],
        "id": 2694,
        "jsonrpc": "2.0"
    },
    {
        "method": "eth_chainId",
        "params": [],
        "id": 2695,
        "jsonrpc": "2.0"
    },
    {
        "method": "eth_chainId",
        "params": [],
        "id": 2696,
        "jsonrpc": "2.0"
    },
    {
        "method": "eth_getLogs",
        "params": [
            {
                "address": "0x19b620929f97b7b990801496c3b361ca5def8c71",
                "topics": [
                    "0x3a5b9dc26075a3801a6ddccf95fec485bb7500a91b44cec1add984c21ee6db3b"
                ],
                "fromBlock": "0x29cd945",
                "toBlock": "0x29cd985"
            }
        ],
        "id": 2697,
        "jsonrpc": "2.0"
    },
    {
        "method": "eth_chainId",
        "params": [],
        "id": 2698,
        "jsonrpc": "2.0"
    },
    {
        "method": "eth_chainId",
        "params": [],
        "id": 2699,
        "jsonrpc": "2.0"
    },
    {
        "method": "eth_getLogs",
        "params": [
            {
                "address": "0x19b620929f97b7b990801496c3b361ca5def8c71",
                "topics": [
                    "0x56a618cda1e34057b7f849a5792f6c8587a2dbe11c83d0254e72cb3daffda7d1"
                ],
                "fromBlock": "0x29cd945",
                "toBlock": "0x29cd985"
            }
        ],
        "id": 2700,
        "jsonrpc": "2.0"
    },
    {
        "method": "eth_chainId",
        "params": [],
        "id": 2701,
        "jsonrpc": "2.0"
    },
    {
        "method": "eth_chainId",
        "params": [],
        "id": 2702,
        "jsonrpc": "2.0"
    },
    {
        "method": "eth_getLogs",
        "params": [
            {
                "address": "0x19b620929f97b7b990801496c3b361ca5def8c71",
                "topics": [
                    "0xd93cf895c7d5b2cd7dc7a098b678b3089f37d91f48d9b83a0800a91cbdf05284"
                ],
                "fromBlock": "0x29cd945",
                "toBlock": "0x29cd985"
            }
        ],
        "id": 2703,
        "jsonrpc": "2.0"
    }
]

[ricmoo]

Can you include how you are configuring the provider?

[therealjmj]

We use a fallback provider, made up of Polling JsonRpcProviders.

Here's an example that fails consistently on Polygon:

const config = {
  chainId: 137,
  providers: [
    {
      provider: 'https://rpc.ankr.com/polygon',
      priority: 2,
      weight: 2,
      disableBatching: true,
    },
  ],
};

The provider itself is instantiated like this:

class PollingJsonRpcProvider extends JsonRpcProvider {
    constructor(url, network, disableBatching = false) {
        const options = {
            polling: true,
            staticNetwork: network,
        };
        if (disableBatching) {
            options.batchMaxCount = 1;
        }
        super(url, network, options);
    }
}

And the fallback provider:

const network = Network.from(Number(config.chainId));

    const providers: FallbackProviderConfig[] = config.providers.map(
      ({
        provider: providerURL,
        priority,
        weight,
        stallTimeout,
        disableBatching,
      }) => {
        const isWebsocket = providerURL.startsWith('wss');
        const provider = isWebsocket
          ? new WebSocketProvider(providerURL, network)
          : new PollingJsonRpcProvider(providerURL, network, disableBatching);

        const fallbackProviderConfig: FallbackProviderConfig = {
          provider,
          priority,
          weight,
          stallTimeout,
        };
        return fallbackProviderConfig;
      },
    );

    return new FallbackProvider(providers, network);

[therealjmj]

I think we have 5 issues to resolve here.

I'm dedicating today + tomorrow to try to hammer on these.

1. "Too many ["eth_getLogs"] methods in the batch error crashes the application in NodeJS. (Expected: no crash, but a console warning / error).
2. Multiple chainId calls are in the batch, despite options.staticNetwork setting. (Expected: no 'chainId' calls with this setting).
3. Repeated chainId calls in the same batch. (Expected: no repeated 'chainId' calls in a single batch).
4. The call is batched despite options.batchMaxCount = 1; setting. (Expected: no 'batch' calls with this setting).
5. Multiple repeated eth_getLogs calls, with the same start and end block numbers. (Expected: no repeated 'getLogs' calls).

[therealjmj]

1 - Still crashing
2 - [Fixed] with proper staticNetwork setting
3 - Unknown - not an issue with staticNetwork setting
4 - [Fixed] with proper options setting
5 - [Fixed] by using '*' for getLogs query, and filtering multiple events in post.