The engine port is invalid and cannot be sync.

[rayn316]

System information

Geth version: 1.15.10-stable-2bf8a789
CL client & version: e.g. v7.0.0-54f7bc5
OS & Version: Linux
Commit hash : (if develop)

Expected behaviour

Actual behaviour

As an RPC node, after a period of time, the node suddenly did not synchronize the block, and found that geth had the following log at the end
3000 Apr 28 16:06:02 geth[495860]: WARN [04-28|16:06:02.980] Beacon client online, but no consensus updates received in a while. Please fix your beacon client to follow the chain!

Then I restarted the lighthouse program, but after the restart, it still couldn't sync, prompting the log.
ERRO Error during execution engine upcheck

Finally, I restarted the geth program and was able to sync.

Steps to reproduce the behaviour

Backtrace

[backtrace]

When submitting logs: please submit them as text and not screenshots.

[s1na]

Do you still have the full logs for when this happened? It seems for some reason the CL and EL got disconnected and then couldn't connect back. Specifically the logs for around the time when EL stopped syncing and then from the time you tried to restart CL.

[rayn316]

@s1na
geth: 14:14:58 22,365,705
The display is stuck after this block, but lighthoue prompts a timeout problem very early

ethereum.log
ethereum-lighthouse.log

[s1na]

What I can see is that geth is syncing albeit slow. The timestamp you provided is this line. We can see a block import took 9s.

Apr 28 14:14:58 geth[650882]: INFO [04-28|14:14:58.113] Imported new potential chain segment     number=22,365,705 hash=3254bd..12d541 blocks=1 txs=126 mgas=14.726 elapsed=9.316s        mgasps=1.581   snapdiffs=5.69MiB  triediffs=230.59MiB triedirty=228.06MiB

I can see that CL->Geth communication is timing out frequently. I think it's because the node is being hammered with RPC requests + it is indexing the logs (which should take a few hours). I imagine it will get better after that.

[s1na]

Possibly related to #31718

[rjl493456442]

Can you reproduce the issue? If so, please dump out the go stacks and block profile.

• geth attach --exec 'debug.stacks()' ./geth.ipc > stack.debug
• geth attach --exec 'debug.blockProfile("block.profile", 10)' ./geth.ipc
• geth attach --exec 'debug.mutexProfile("mutex.profile", 10)' ./geth.ipc

[blockPI-rayn]

Attach to the geth binary at ./bin/geth and execute the command 'debug.blockProfile("block.profile", 10)' on the IPC file ./geth/geth.ipc
null

Attach to the geth binary at ./bin/geth and execute the command 'debug.mutexProfile("mutex.profile", 10)' on the IPC file ./geth/geth.ipc
null
``
stack.debug    http://46.4.139.66/stack.debug

[rjl493456442]

We have merged a fix for mitigating the overhead, #31752

Please have a try.