Skip to content

Commit e95ebaa

Browse files
tac0turtleMarko BaricevicMarko Baricevic
authored
feat: add priority mempool (#1703)
## Description This PR adds back the priority mempool to comet --------- Co-authored-by: Marko Baricevic <markobaricevic@Markos-MacBook-Pro.local> Co-authored-by: Marko Baricevic <markobaricevic@MacBookPro.localdomain>
1 parent bbd3b9b commit e95ebaa

File tree

15 files changed

+2775
-227
lines changed

15 files changed

+2775
-227
lines changed

‎abci/types/types.pb.go

Lines changed: 218 additions & 219 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎config/config.go

Lines changed: 26 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -40,9 +40,13 @@ const (
4040
DefaultNodeKeyName = "node_key.json"
4141
DefaultAddrBookName = "addrbook.json"
4242

43-
MempoolTypeFlood = "flood"
44-
MempoolTypeNop = "nop"
45-
MempoolTypeCAT = "cat"
43+
MempoolTypeFlood = "flood"
44+
MempoolTypeNop = "nop"
45+
MempoolTypePriority = "priority"
46+
MempoolTypeCAT = "cat"
47+
LegacyMempoolTypeFlood = "v0"
48+
LegacyMempoolTypePriority = "v1"
49+
LegacyMempoolTypeCAT = "v2"
4650
)
4751

4852
// NOTE: Most of the structs & relevant comments + the
@@ -794,6 +798,22 @@ type MempoolConfig struct {
794798
// Only applicable to the v2 / CAT mempool
795799
// Default is 200ms
796800
MaxGossipDelay time.Duration `mapstructure:"max-gossip-delay"`
801+
802+
// TTLDuration, if non-zero, defines the maximum amount of time a transaction
803+
// can exist for in the mempool.
804+
//
805+
// Note, if TTLNumBlocks is also defined, a transaction will be removed if it
806+
// has existed in the mempool at least TTLNumBlocks number of blocks or if it's
807+
// insertion time into the mempool is beyond TTLDuration.
808+
TTLDuration time.Duration `mapstructure:"ttl-duration"`
809+
810+
// TTLNumBlocks, if non-zero, defines the maximum number of blocks a transaction
811+
// can exist for in the mempool.
812+
//
813+
// Note, if TTLDuration is also defined, a transaction will be removed if it
814+
// has existed in the mempool at least TTLNumBlocks number of blocks or if
815+
// it's insertion time into the mempool is beyond TTLDuration.
816+
TTLNumBlocks int64 `mapstructure:"ttl-num-blocks"`
797817
}
798818

799819
// DefaultMempoolConfig returns a default configuration for the CometBFT mempool
@@ -812,6 +832,8 @@ func DefaultMempoolConfig() *MempoolConfig {
812832
MaxTxBytes: 1024 * 1024, // 1MB
813833
ExperimentalMaxGossipConnectionsToNonPersistentPeers: 0,
814834
ExperimentalMaxGossipConnectionsToPersistentPeers: 0,
835+
TTLDuration: 0 * time.Second,
836+
TTLNumBlocks: 0,
815837
}
816838
}
817839

@@ -836,7 +858,7 @@ func (cfg *MempoolConfig) WalEnabled() bool {
836858
// returns an error if any check fails.
837859
func (cfg *MempoolConfig) ValidateBasic() error {
838860
switch cfg.Type {
839-
case MempoolTypeFlood, MempoolTypeNop:
861+
case MempoolTypeFlood, MempoolTypeNop, MempoolTypePriority, LegacyMempoolTypePriority, LegacyMempoolTypeFlood:
840862
case "": // allow empty string to be backwards compatible
841863
default:
842864
return fmt.Errorf("unknown mempool type: %q", cfg.Type)

‎mempool/priority/README.md

Lines changed: 242 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,242 @@
1+
# Priority Mempool
2+
3+
The Priority Mempool is a specialized transaction pool for Celestia Core (based on CometBFT) that manages pending transactions with a focus on transaction prioritization. This document provides an overview of how the priority mempool works, its key components, and transaction flows.
4+
5+
## Core Concepts
6+
7+
The Priority Mempool prioritizes transactions based on their assigned priority values. When selecting transactions for inclusion in a block or when evicting transactions due to size constraints, priority values are crucial:
8+
9+
- **Higher-priority** transactions are chosen **first** for block inclusion
10+
- **Lower-priority** transactions are evicted **sooner** when mempool is full
11+
12+
Within the mempool, transactions are ordered by time of arrival for gossiping to the rest of the network.
13+
14+
## Key Components
15+
16+
### 1. `TxMempool`
17+
18+
The main mempool structure that handles transaction storage, validation, and lifecycle:
19+
20+
```ascii
21+
┌─────────────────────────────────────────┐
22+
│ TxMempool │
23+
├─────────────────────────────────────────┤
24+
│ ┌─────────┐ ┌───────────┐ ┌────────┐ │
25+
│ │ Cache │ │ txByKey │ │ txs │ │
26+
│ └─────────┘ └───────────┘ └────────┘ │
27+
│ │
28+
│ ┌───────────┐ ┌─────────────────────┐ │
29+
│ │txBySender │ │ evictedTxs │ │
30+
│ └───────────┘ └─────────────────────┘ │
31+
└─────────────────────────────────────────┘
32+
```
33+
34+
- **Cache**: Tracks seen transactions to prevent duplicates
35+
- **txs**: Linked list of valid transactions that passed CheckTx
36+
- **txByKey**: Map of transaction keys to list elements for fast lookups
37+
- **txBySender**: Map of sender IDs to transactions for sender-based filtering
38+
- **evictedTxs**: Cache of recently evicted transactions
39+
40+
### 2. `WrappedTx`
41+
42+
Wrapper around raw transactions with additional metadata:
43+
44+
```ascii
45+
┌───────────────────────────────┐
46+
│ WrappedTx │
47+
├───────────────────────────────┤
48+
│ tx : Raw transaction │
49+
│ hash : Transaction hash │
50+
│ height : Initial check │
51+
│ timestamp : Entry time │
52+
│ │
53+
│ gasWanted : Gas requirement │
54+
│ priority : Priority value │
55+
│ sender : Sender identifier │
56+
│ peers : Source peer IDs │
57+
└───────────────────────────────┘
58+
```
59+
60+
### 3. `Reactor`
61+
62+
Handles peer-to-peer communication for transaction broadcasting:
63+
64+
```ascii
65+
┌───────────────────────────────┐
66+
│ Reactor │
67+
├───────────────────────────────┤
68+
│ mempool : TxMempool ref │
69+
│ ids : Peer ID tracker │
70+
│ config : Config settings │
71+
│ traceClient : Tracer instance │
72+
└───────────────────────────────┘
73+
```
74+
75+
## Transaction Flow
76+
77+
### External Transaction Flow
78+
79+
```ascii
80+
┌──────────────┐ 1. Submit Tx ┌───────────────┐
81+
│ │----------------->│ │
82+
│ Client │ │ Node │
83+
│ │<-----------------│ │
84+
└──────────────┘ 2. Tx Response └───────┬───────┘
85+
│
86+
│ 3. Gossip to peers
87+
▼
88+
┌─────────────────────────┐
89+
│ Network │
90+
│ (Other Validators) │
91+
└─────────────────────────┘
92+
```
93+
94+
1. Client submits transaction to a node
95+
2. Node validates and returns response
96+
3. If valid, node gossips transaction to peers
97+
98+
### Internal Transaction Processing
99+
100+
```ascii
101+
┌────────────────────────────────────────────────────┐
102+
│ TxMempool │
103+
│ │
104+
┌─────────┐ │ ┌──────────┐ ┌───────────┐ ┌────────────┐ │
105+
│ │ │ │ 1. Size │ │ 2. PreCheck│ │3. Cache Check│ │
106+
│ Tx │──────────▶ Check │───▶│ Hook │───▶│ │ │
107+
│ │ │ │ │ │ │ │ │ │
108+
└─────────┘ │ └──────────┘ └───────────┘ └──────┬───────┘ │
109+
│ │ │
110+
│ │ │
111+
│ ▼ │
112+
│ ┌────────────┐ ┌───────────┐ ┌─────────────┐ │
113+
│ │ 6. Update │ │5. PostCheck│ │4. Application│ │
114+
│ │ Mempool │◀──│ Hook │◀──│ CheckTx │ │
115+
│ │ │ │ │ │ │ │
116+
│ └─────┬──────┘ └───────────┘ └──────────────┘ │
117+
│ │ │
118+
└────────┼─────────────────────────────────────────────┘
119+
│
120+
│ Full Mempool?
121+
▼
122+
┌────────────────┐
123+
│ Eviction │
124+
│ Logic │
125+
└────────────────┘
126+
```
127+
128+
1. **Size Check**: Reject if transaction exceeds the configured max size
129+
2. **PreCheck Hook**: Run optional validation hook
130+
3. **Cache Check**: Check if transaction is already in cache or mempool
131+
4. **Application CheckTx**: Application checks transaction, assigns priority
132+
5. **PostCheck Hook**: Run optional post-validation hook
133+
6. **Update Mempool**: Add transaction to mempool, possibly evicting lower-priority transactions if full
134+
135+
### Eviction Process
136+
137+
```ascii
138+
┌───────────────────────────────────────────────────────────┐
139+
│ Full Mempool │
140+
│ ┌────┐ ┌────┐ ┌────┐ ┌────┐ ┌────┐ ┌────┐ ┌────┐ │
141+
│ │Tx 1│ │Tx 2│ │Tx 3│ │Tx 4│ │Tx 5│ │Tx 6│ │Tx 7│ │
142+
│ │p=10│ │p=25│ │p=7 │ │p=15│ │p=3 │ │p=2 │ │p=30│ │
143+
│ └────┘ └────┘ └────┘ └────┘ └────┘ └────┘ └────┘ │
144+
└───────────────────────────────────────────────────────────┘
145+
▲
146+
│ New transaction (p=20)
147+
│
148+
┌───────────────────────────┐
149+
│ Find lowest priority txs │
150+
│ until space is available │
151+
└───────────────────────────┘
152+
│
153+
▼
154+
┌───────────────────────────────────────────────────────────┐
155+
│ Updated Mempool │
156+
│ ┌────┐ ┌────┐ ┌────┐ ┌────┐ ┌────┐ ┌────┐ ┌────┐ │
157+
│ │Tx 1│ │Tx 2│ │Tx 3│ │Tx 4│ │New │ │Tx 7│ │ │ │
158+
│ │p=10│ │p=25│ │p=7 │ │p=15│ │p=20│ │p=30│ │ │ │
159+
│ └────┘ └────┘ └────┘ └────┘ └────┘ └────┘ └────┘ │
160+
└───────────────────────────────────────────────────────────┘
161+
Tx 5 & Tx 6 evicted (lowest priority)
162+
```
163+
164+
When the mempool is full and a new transaction arrives with higher priority than existing transactions:
165+
166+
1. Identify lowest-priority transactions
167+
2. Evict them until sufficient space is available
168+
3. Add the new transaction to the mempool
169+
170+
### Block Proposal
171+
172+
```ascii
173+
┌───────────────────────────────┐
174+
│ Mempool │
175+
│ │
176+
│ ┌────┐┌────┐┌────┐┌────┐┌────┐│
177+
│ │p=30││p=25││p=20││p=15││p=10││
178+
│ └────┘└────┘└────┘└────┘└────┘│
179+
└─────────────┬─────────────────┘
180+
│
181+
│ ReapMaxBytesMaxGas()
182+
│ or ReapMaxTxs()
183+
▼
184+
┌───────────────────────────────┐
185+
│ Proposed Block │
186+
│ ┌────┐┌────┐┌────┐ │
187+
│ │p=30││p=25││p=20│ │
188+
│ └────┘└────┘└────┘ │
189+
└───────────────────────────────┘
190+
```
191+
192+
During block proposal:
193+
194+
1. `ReapMaxBytesMaxGas()` or `ReapMaxTxs()` is called
195+
2. Transactions are selected in priority order
196+
3. Selected transactions are included in the proposed block
197+
198+
## TTL Mechanisms
199+
200+
The Priority Mempool supports two mechanisms for transaction expiration:
201+
202+
1. **Time-based TTL**: Transactions expire after a configured duration
203+
2. **Block-height TTL**: Transactions expire after a specific number of blocks
204+
205+
```ascii
206+
┌───────────────────────────────────────────────────────────┐
207+
│ Mempool │
208+
│ │
209+
│ ┌────┐ ┌────┐ ┌────┐ ┌────┐ ┌────┐ ┌────┐ ┌────┐ │
210+
│ │Tx 1│ │Tx 2│ │Tx 3│ │Tx 4│ │Tx 5│ │Tx 6│ │Tx 7│ │
211+
│ │h=10│ │h=10│ │h=11│ │h=11│ │h=12│ │h=12│ │h=13│ │
212+
│ └────┘ └────┘ └────┘ └────┘ └────┘ └────┘ └────┘ │
213+
└───────────────────────────────────────────────────────────┘
214+
215+
Current height = 20
216+
TTLNumBlocks = 9
217+
218+
┌───────────────────────────────────────────────────────────┐
219+
│ Updated Mempool │
220+
│ │
221+
│ ┌────┐ ┌────┐ ┌────┐ ┌────┐ │
222+
│ │Tx 5│ │Tx 6│ │Tx 7│ │ │ │
223+
│ │h=12│ │h=12│ │h=13│ │ │ │
224+
│ └────┘ └────┘ └────┘ └────┘ │
225+
└───────────────────────────────────────────────────────────┘
226+
227+
Tx 1-4 expired (height < 11)
228+
```
229+
230+
## Configuration Options
231+
232+
The Priority Mempool can be configured with several options:
233+
234+
- **Size**: Maximum number of transactions
235+
- **MaxTxBytes**: Maximum transaction size
236+
- **CacheSize**: Size of the transaction cache
237+
- **TTLDuration**: Time-based expiration duration
238+
- **TTLNumBlocks**: Block-height-based expiration limit
239+
240+
## Conclusion
241+
242+
The Priority Mempool provides a sophisticated transaction management system that ensures higher-priority transactions are included in blocks and maintained in the mempool while lower-priority transactions are evicted when necessary. This design helps optimize block space usage and transaction processing efficiency.

0 commit comments

Comments
 (0)