Deterministic Block Templates

[mcelrath]

I posted this idea on Delving Bitcoin, but reproduced here for more focused discussion.

One of my fears is that the risk and liability of transaction selection is so bad that no one will want to do it, leading to full centralization around the block template providers. We may already be there. Matt made a lot of hay about this in his recent TFTC interview and he’s right. The thesis that we can achieve decentralization by pool hopping may be false if there’s only one tx set being used by pools, or if there are too few pools to hop to, and having many pools has a negative consequence in terms of variance reduction. Tx selection is an all downside and no upside problem. Every jurisdiction has someone they don’t like, whether it be political rivals, business rivals, criminals or freedom fighters. If Bitcoin is to be censorship resistant, we’re failing.

An alternative is to have a deterministic algorithm select transactions. The reason we can’t do this today is that we do not all have the same set of txs in our mempool, we have no consensus on the contents of the mempool, and miners can and do introduce txs that have never been seen by anyone in their blocks.

But if we did have consensus on the mempool, it’s straightforward to use a deterministic algorithm to select a subset of those txs.

Step one is to have a committed mempool with consensus. This can be accomplished fairly straightforwardly in a decentralized mining pool like P2Pool or Braidpool by adding a metadata block to the weak blocks (“shares”) used by the pool containing transactions. These metadata “mempool” txs must not conflict with other txs committed to the mempool in ancestor shares, and you can regard this “mempool” as a single large self-consistent block with no block size restriction.

Once we have this committed tx set, every weak block in the share chain defines its own mempool in its ancestors by following the highest work-weighted path to the previous Bitcoin block. (This definition is valid both for blockchain-based P2Pool and DAG-based Braidpool).

One advantage of this approach is that by defining the deterministic block template algorithm, it’s no longer necessary to actually communicate the block template to sharechain nodes since it can be independently computed, vastly speeding up share/block validation and reducing communications bandwidth to be basically equivalent to today’s mempool bandwidth plus the weak block shares, which would add a few hundred bytes about once a second.

Now this doesn’t fully solve the problem of transaction censorship, but rather moves it to the share chain. If you want to prevent a tx from being mined in the main chain, you now have to prevent it from being mined in the share chain. However being mined in the share chain is not finalization. The UTXO set of the parent chain has not updated and in a double spend can still occur. In a DAG this happens via diamond-type graphs with conflicting txs on opposite sides - the algorithm must follow the highest work-weighted path and will pick up only one side of the diamond, but work can be added to the opposite side of the diamond to effect a change in the highest work-weighted path. In a P2Pool-like chain, the same can occur by orphaning a share chain block.

[Ravish990]

Hello I am Ravish,

I am learning and getting to know about the project in detail and trying find the proper implementation.

[Ujjawal179]

This is definitely a deep and complex topic. I’m currently deep-diving into it and studying the implications, especially after reading the discussion. The idea of a deterministic transaction selection algorithm seems promising, but I can see how it could shift the problem of censorship to the share chain level. I’m really trying to wrap my head around it and evaluate it carefully.

[R27-pixel]

This concept of deterministic transaction selection using a committed mempool is fascinating. I’d like to explore further how we can balance the decentralization benefits with potential challenges like ensuring the scalability of mempool consensus and mitigating vulnerabilities in share chain censorship. Could implementing additional cryptographic techniques or incentive mechanisms in the share chain help address these challenges?

[rajanarahul93]

This is a fascinating discussion! A committed mempool with consensus sounds promising, especially for decentralized mining pools like Braidpool. It helps reduce block template centralization, but I’m curious—how realistic is it to achieve global consensus on mempool transactions while keeping things scalable and resistant to manipulation? It’d be great to dive deeper into the trade-offs between efficiency and censorship resistance in this approach!