@@ -168,18 +168,33 @@ In order to support PMem storage, we add some more config fields to the Kafka [s
168
168
Sample config in config/server.properties is as follows:
169
169
170
170
storage.pmem.path=/mnt/pmem/kafka/
171
- storage.pmem.size=107374182400
172
- log.pmem.pool.ratio=0.9
171
+ storage.pmem.size=21474836480
172
+ log.pmem.pool.ratio=0.6
173
173
log.channel.type=pmem
174
174
# log.preallocate have to set to true if pmem is used
175
175
log.preallocate=true
176
176
177
- #### Start Kafka
177
+ #### Start Pafka
178
178
Follow instructions in https://kafka.apache.org/quickstart . Basically:
179
179
180
- bin/zookeeper-server-start.sh config/zookeeper.properties
181
- bin/kafka-server-start.sh config/server.properties
180
+ bin/zookeeper-server-start.sh config/zookeeper.properties > zk.log 2>&1 &
181
+ bin/kafka-server-start.sh config/server.properties > pafka.log 2>&1 &
182
+
183
+ #### Benchmark Pafka
184
+
185
+ ##### Producer
186
+
187
+ ``` bash
188
+ # bin/kafka-producer-perf-test.sh --topic $TOPIC --throughput $MAX_THROUGHPUT --num-records $NUM_RECORDS --record-size $RECORD_SIZE --producer.config config/producer.properties
189
+ bin/kafka-producer-perf-test.sh --topic test --throughput 1000000 --num-records 1000000 --record-size 1024 --producer.config config/producer.properties
190
+ ```
191
+
192
+ ##### Consumer
182
193
194
+ ``` bash
195
+ # bin/kafka-consumer-perf-test.sh --topic $TOPIC --consumer.config config/consumer.properties --bootstrap-server localhost:9092 --messages $NUM_RECORDS --show-detailed-stats --reporting-interval $REPORT_INTERVAL --timeout $TIMEOUT_IN_MS
196
+ bin/kafka-consumer-perf-test.sh --topic test --consumer.config config/consumer.properties --bootstrap-server localhost:9092 --messages 1000000 --show-detailed-stats --reporting-interval 1000 --timeout 100000
197
+ ```
183
198
184
199
## Limitations
185
200
@@ -204,7 +219,7 @@ as we do not see much performance gain if we move the indexes to PMem.
204
219
205
220
| Version | Status | Features |
206
221
| ---| ---| ---|
207
- | v0.1.0 | Released| - Use PMem for data storage <br /> - Significant performance boost compared with Kafka |
222
+ | v0.1.1 | Released| - Use PMem for data storage <br /> - Significant performance boost compared with Kafka |
208
223
| v0.2.0| To be released in June/July 2021| - Layered storage with PMem and other storage medium|
209
224
210
225
0 commit comments