Flaky test: nimiq-consensus::state_live_sync revert_chunks_for_state_live_sync #3409
Description
In #3407 with entirely unrelated changes: https://github.com/nimiq/core-rs-albatross/actions/runs/15441825305/job/43461240427:
FAIL [ 1.037s] nimiq-consensus::state_live_sync revert_chunks_for_state_live_sync
stdout ───
running 1 test
2025-06-04T12:24:06.878637Z INFO nimiq_database::mdbx::database: MDBX memory map size cur_mapsize=2199023255552
2025-06-04T12:24:06.888926Z DEBUG nimiq_database::mdbx::database: Creating table: ValidityTxnHashes, flags: TableFlags(CREATE)
2025-06-04T12:24:06.891283Z DEBUG nimiq_database::mdbx::database: Creating table: ValidityBlockTxnHashes, flags: TableFlags(DUP_SORT | INTEGER_KEY | DUP_FIXED | CREATE)
2025-06-04T12:24:06.894162Z DEBUG nimiq_database::mdbx::database: Creating table: HistoryTrees, flags: TableFlags(DUP_SORT | INTEGER_KEY | DUP_FIXED | CREATE)
2025-06-04T12:24:06.897960Z DEBUG nimiq_database::mdbx::database: Creating table: HistoricTransactions, flags: TableFlags(DUP_SORT | INTEGER_KEY | CREATE)
2025-06-04T12:24:06.898680Z DEBUG nimiq_database::mdbx::database: Creating table: LastLeafIndexesByBlock, flags: TableFlags(INTEGER_KEY | CREATE)
2025-06-04T12:24:06.899201Z DEBUG nimiq_database::mdbx::database: Creating table: LeafIndexByTxHash, flags: TableFlags(CREATE)
2025-06-04T12:24:06.899640Z DEBUG nimiq_database::mdbx::database: Creating table: TxHashesByAddress, flags: TableFlags(DUP_SORT | DUP_FIXED | CREATE)
2025-06-04T12:24:06.900025Z DEBUG nimiq_database::mdbx::database: Creating table: Head, flags: TableFlags(CREATE)
2025-06-04T12:24:06.900520Z DEBUG nimiq_database::mdbx::database: Creating table: ChainData, flags: TableFlags(CREATE)
2025-06-04T12:24:06.901021Z DEBUG nimiq_database::mdbx::database: Creating table: PushedBlockTable, flags: TableFlags(CREATE)
2025-06-04T12:24:06.901498Z DEBUG nimiq_database::mdbx::database: Creating table: StoredBlockTable, flags: TableFlags(CREATE)
2025-06-04T12:24:06.901943Z DEBUG nimiq_database::mdbx::database: Creating table: HeightIndex, flags: TableFlags(DUP_SORT | INTEGER_KEY | DUP_FIXED | CREATE)
2025-06-04T12:24:06.902329Z DEBUG nimiq_database::mdbx::database: Creating table: Receipts, flags: TableFlags(INTEGER_KEY | CREATE)
2025-06-04T12:24:06.902762Z DEBUG nimiq_database::mdbx::database: Creating table: AccountsDiff, flags: TableFlags(CREATE)
2025-06-04T12:24:06.903275Z DEBUG nimiq_database::mdbx::database: Creating table: AccountsTrie, flags: TableFlags(CREATE)
2025-06-04T12:24:06.920472Z INFO nimiq_database::mdbx::database: MDBX memory map size cur_mapsize=2199023255552
2025-06-04T12:24:06.920550Z DEBUG nimiq_database::mdbx::database: Creating table: ValidityTxnHashes, flags: TableFlags(CREATE)
2025-06-04T12:24:06.921951Z DEBUG nimiq_database::mdbx::database: Creating table: ValidityBlockTxnHashes, flags: TableFlags(DUP_SORT | INTEGER_KEY | DUP_FIXED | CREATE)
2025-06-04T12:24:06.923491Z DEBUG nimiq_database::mdbx::database: Creating table: HistoryTrees, flags: TableFlags(DUP_SORT | INTEGER_KEY | DUP_FIXED | CREATE)
2025-06-04T12:24:06.926949Z DEBUG nimiq_database::mdbx::database: Creating table: HistoricTransactions, flags: TableFlags(DUP_SORT | INTEGER_KEY | CREATE)
2025-06-04T12:24:06.927477Z DEBUG nimiq_database::mdbx::database: Creating table: LastLeafIndexesByBlock, flags: TableFlags(INTEGER_KEY | CREATE)
2025-06-04T12:24:06.929936Z DEBUG nimiq_database::mdbx::database: Creating table: LeafIndexByTxHash, flags: TableFlags(CREATE)
2025-06-04T12:24:06.930577Z DEBUG nimiq_database::mdbx::database: Creating table: TxHashesByAddress, flags: TableFlags(DUP_SORT | DUP_FIXED | CREATE)
2025-06-04T12:24:06.930995Z DEBUG nimiq_database::mdbx::database: Creating table: Head, flags: TableFlags(CREATE)
2025-06-04T12:24:06.931382Z DEBUG nimiq_database::mdbx::database: Creating table: ChainData, flags: TableFlags(CREATE)
2025-06-04T12:24:06.931782Z DEBUG nimiq_database::mdbx::database: Creating table: PushedBlockTable, flags: TableFlags(CREATE)
2025-06-04T12:24:06.932282Z DEBUG nimiq_database::mdbx::database: Creating table: StoredBlockTable, flags: TableFlags(CREATE)
2025-06-04T12:24:06.932725Z DEBUG nimiq_database::mdbx::database: Creating table: HeightIndex, flags: TableFlags(DUP_SORT | INTEGER_KEY | DUP_FIXED | CREATE)
2025-06-04T12:24:06.933070Z DEBUG nimiq_database::mdbx::database: Creating table: Receipts, flags: TableFlags(INTEGER_KEY | CREATE)
2025-06-04T12:24:06.933695Z DEBUG nimiq_database::mdbx::database: Creating table: AccountsDiff, flags: TableFlags(CREATE)
2025-06-04T12:24:06.934110Z DEBUG nimiq_database::mdbx::database: Creating table: AccountsTrie, flags: TableFlags(CREATE)
2025-06-04T12:24:07.008321Z INFO nimiq_database::mdbx::database: MDBX memory map size cur_mapsize=2199023255552
2025-06-04T12:24:07.010014Z DEBUG nimiq_database::mdbx::database: Creating table: ValidityTxnHashes, flags: TableFlags(CREATE)
2025-06-04T12:24:07.011494Z DEBUG nimiq_database::mdbx::database: Creating table: ValidityBlockTxnHashes, flags: TableFlags(DUP_SORT | INTEGER_KEY | DUP_FIXED | CREATE)
2025-06-04T12:24:07.012830Z DEBUG nimiq_database::mdbx::database: Creating table: HistoryTrees, flags: TableFlags(DUP_SORT | INTEGER_KEY | DUP_FIXED | CREATE)
2025-06-04T12:24:07.014394Z DEBUG nimiq_database::mdbx::database: Creating table: HistoricTransactions, flags: TableFlags(DUP_SORT | INTEGER_KEY | CREATE)
2025-06-04T12:24:07.015025Z DEBUG nimiq_database::mdbx::database: Creating table: LastLeafIndexesByBlock, flags: TableFlags(INTEGER_KEY | CREATE)
2025-06-04T12:24:07.015702Z DEBUG nimiq_database::mdbx::database: Creating table: LeafIndexByTxHash, flags: TableFlags(CREATE)
2025-06-04T12:24:07.018134Z DEBUG nimiq_database::mdbx::database: Creating table: TxHashesByAddress, flags: TableFlags(DUP_SORT | DUP_FIXED | CREATE)
2025-06-04T12:24:07.021298Z DEBUG nimiq_database::mdbx::database: Creating table: Head, flags: TableFlags(CREATE)
2025-06-04T12:24:07.021739Z DEBUG nimiq_database::mdbx::database: Creating table: ChainData, flags: TableFlags(CREATE)
2025-06-04T12:24:07.022296Z DEBUG nimiq_database::mdbx::database: Creating table: PushedBlockTable, flags: TableFlags(CREATE)
2025-06-04T12:24:07.022807Z DEBUG nimiq_database::mdbx::database: Creating table: StoredBlockTable, flags: TableFlags(CREATE)
2025-06-04T12:24:07.023403Z DEBUG nimiq_database::mdbx::database: Creating table: HeightIndex, flags: TableFlags(DUP_SORT | INTEGER_KEY | DUP_FIXED | CREATE)
2025-06-04T12:24:07.024980Z DEBUG nimiq_database::mdbx::database: Creating table: Receipts, flags: TableFlags(INTEGER_KEY | CREATE)
2025-06-04T12:24:07.025622Z DEBUG nimiq_database::mdbx::database: Creating table: AccountsDiff, flags: TableFlags(CREATE)
2025-06-04T12:24:07.026117Z DEBUG nimiq_database::mdbx::database: Creating table: AccountsTrie, flags: TableFlags(CREATE)
2025-06-04T12:24:07.258389Z DEBUG nimiq_blockchain::blockchain::push: Accepted block block=#201:MI:5703c71738 num_transactions=0 kind="extend" duration=4.420818ms
2025-06-04T12:24:07.372338Z DEBUG nimiq_blockchain::blockchain::push: Accepted block block=#201:MI:fb03b7577c num_transactions=0 kind="extend" duration=4.661053ms
2025-06-04T12:24:07.493502Z DEBUG nimiq_blockchain::blockchain::push: Accepted block block=#202:MI:b05a6a3597 num_transactions=0 kind="extend" duration=1.811632ms
2025-06-04T12:24:07.493626Z INFO state_live_sync: Incomplete blockchain: at #200 - 4f8f18fcfe98ccacee0745b6aa69365cd28c836f962d577de57f04aa17fe54a1, accounts: Some(ε)
2025-06-04T12:24:07.493818Z DEBUG nimiq_consensus::sync::live::block_queue::queue: Received block via gossipsub block=#201:MI:5703c71738 peer_id=2
2025-06-04T12:24:07.494127Z INFO nimiq_test_utils::mock_node: Received request RequestTrieDiff { block_hash: 5703c71738b1f435431cac1977f0aed3eaba054d2324d5d981ab4c17bfc04f54 }, has custom handler: false
2025-06-04T12:24:07.584933Z DEBUG nimiq_blockchain::blockchain::push: Accepted block block=#201:MI:5703c71738 num_transactions=0 kind="extend" duration=6.493406ms
2025-06-04T12:24:07.588967Z INFO state_live_sync: Applying chunk #0
2025-06-04T12:24:07.589020Z INFO nimiq_test_utils::mock_node: Received request RequestChunk { start_key: ε, limit: 3 }, has custom handler: false
2025-06-04T12:24:07.589593Z INFO nimiq_consensus::sync::live::state_queue: Received state sync chunk, ~0.00% complete start_key=ε
2025-06-04T12:24:07.590971Z INFO state_live_sync: Incomplete blockchain: at #201 - 5703c71738b1f435431cac1977f0aed3eaba054d2324d5d981ab4c17bfc04f54, accounts: Some(e)
2025-06-04T12:24:07.685499Z INFO nimiq_blockchain_interface::chain_ordering: New block in Unknown chain fork_block_number=200 current_block_number=201 new_block_number=201
2025-06-04T12:24:07.685578Z DEBUG nimiq_blockchain::blockchain::push: Storing block - on fork block=#201:MI:fb03b7577c
2025-06-04T12:24:07.795961Z INFO nimiq_blockchain_interface::chain_ordering: New block in Superior chain fork_block_number=200 current_block_number=201 new_block_number=202
2025-06-04T12:24:07.798959Z DEBUG nimiq_blockchain::blockchain::push: Rebranching block="#202:MI:b05a6a3597"
2025-06-04T12:24:07.799115Z DEBUG nimiq_blockchain::blockchain::push: Found common ancestor block="#202:MI:b05a6a3597" common_ancestor=#200:MA:4f8f18fcfe no_blocks_up=2
2025-06-04T12:24:07.799246Z DEBUG nimiq_blockchain::blockchain::accounts: Reverting block block=#201:MI:5703c71738 is_skip=false num_transactions=0 num_equivocation_proofs=0
2025-06-04T12:24:07.802920Z DEBUG nimiq_blockchain::blockchain::push: Reverted block block=#201:MI:5703c71738 num_transactions=0
2025-06-04T12:24:07.802949Z DEBUG nimiq_blockchain::blockchain::push: Accepted block block=#201:MI:fb03b7577c num_transactions=0 kind="rebranch"
2025-06-04T12:24:07.802961Z DEBUG nimiq_blockchain::blockchain::push: Accepted block block=#202:MI:b05a6a3597 num_transactions=0 kind="rebranch"
2025-06-04T12:24:07.802975Z DEBUG nimiq_blockchain::blockchain::push: Rebranched block=#202:MI:b05a6a3597 num_reverted_blocks=1 num_adopted_blocks=2 duration=3.992053ms
2025-06-04T12:24:07.803024Z INFO state_live_sync: Requesting chunks #1
2025-06-04T12:24:07.803155Z INFO nimiq_test_utils::mock_node: Received request RequestChunk { start_key: e, limit: 3 }, has custom handler: false
2025-06-04T12:24:07.803420Z INFO nimiq_consensus::sync::live::state_queue: Received state sync chunk, ~87.50% complete start_key=e
2025-06-04T12:24:07.803508Z DEBUG nimiq_consensus::sync::live::block_queue::queue: Received block via gossipsub block=#202:MI:b05a6a3597 peer_id=2
2025-06-04T12:24:07.803618Z DEBUG nimiq_consensus::sync::live::block_queue::queue: Requesting missing blocks block_number=201 block_hash=fb03b7577c96689ca663c00fa34d2f4e03bf047a1069baa7a8e056764fc9b650 head_hash=5703c71738b1f435431cac1977f0aed3eaba054d2324d5d981ab4c17bfc04f54 macro_height=200
2025-06-04T12:24:07.803769Z INFO nimiq_test_utils::mock_node: Received request RequestMissingBlocks { target_hash: fb03b7577c96689ca663c00fa34d2f4e03bf047a1069baa7a8e056764fc9b650, include_body: true, locators: [5703c71738b1f435431cac1977f0aed3eaba054d2324d5d981ab4c17bfc04f54, 4f8f18fcfe98ccacee0745b6aa69365cd28c836f962d577de57f04aa17fe54a1], direction: Forward }, has custom handler: false
2025-06-04T12:24:07.804263Z INFO nimiq_test_utils::mock_node: Received request RequestTrieDiff { block_hash: fb03b7577c96689ca663c00fa34d2f4e03bf047a1069baa7a8e056764fc9b650 }, has custom handler: false
2025-06-04T12:24:07.804393Z DEBUG nimiq_consensus::sync::live::queue: Pushing block #201:MI:fb03b7577c from missing blocks response
2025-06-04T12:24:07.885583Z INFO nimiq_blockchain_interface::chain_ordering: New block in Unknown chain fork_block_number=200 current_block_number=201 new_block_number=201
2025-06-04T12:24:07.885649Z DEBUG nimiq_blockchain::blockchain::push: Storing block - on fork block=#201:MI:fb03b7577c
test revert_chunks_for_state_live_sync ... FAILED
failures:
failures:
revert_chunks_for_state_live_sync
test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 5 filtered out; finished in 1.02s
stderr ───
thread 'revert_chunks_for_state_live_sync' panicked at consensus/tests/state_live_sync.rs:546:5:
assertion `left == right` failed
left: 1
right: 0
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace