Skip to content

Commit 33df4ae

Browse files
achow101achow101
committed
Merge bitcoin/bitcoin#31551: [IBD] batch block reads/writes during AutoFile serialization
8d801e3 optimization: bulk serialization writes in `WriteBlockUndo` and `WriteBlock` (Lőrinc) 520965e optimization: bulk serialization reads in `UndoRead`, `ReadBlock` (Lőrinc) 056cb3c refactor: clear up blockstorage/streams in preparation for optimization (Lőrinc) 67fcc64 log: unify error messages for (read/write)[undo]block (Lőrinc) a4de160 scripted-diff: shorten BLOCK_SERIALIZATION_HEADER_SIZE constant (Lőrinc) 6640dd5 Narrow scope of undofile write to avoid possible resource management issue (Lőrinc) 3197155 refactor: collect block read operations into try block (Lőrinc) c77e310 refactor: rename leftover WriteBlockBench (Lőrinc) Pull request description: This change is part of [[IBD] - Tracking PR for speeding up Initial Block Download](bitcoin/bitcoin#32043) ### Summary We can serialize the blocks and undos to any `Stream` which implements the appropriate read/write methods. `AutoFile` is one of these, writing the results "directly" to disk (through the OS file cache). Batching these in memory first and reading/writing these to disk is measurably faster (likely because of fewer native fread calls or less locking, as [observed](bitcoin/bitcoin#28226 (comment)) by Martinus in a similar change). ### Unlocking new optimization opportunities Buffered writes will also enable batched obfuscation calculations (implemented in bitcoin/bitcoin#31144) - especially since currently we need to copy the write input's std::span to do the obfuscation on it, and batching enables doing the operations on the internal buffer directly. ### Measurements (micro benchmarks, full IBDs and reindexes) Microbenchmarks for `[Read|Write]BlockBench` show a ~**30%**/**168%** speedup with `macOS/Clang`, and ~**19%**/**24%** with `Linux/GCC` (the follow-up XOR batching improves these further): <details> <summary>macOS Sequoia - Clang 19.1.7</summary> > Before: | ns/op | op/s | err% | total | benchmark |--------------------:|--------------------:|--------:|----------:|:---------- | 2,271,441.67 | 440.25 | 0.1% | 11.00 | `ReadBlockBench` | 5,149,564.31 | 194.19 | 0.8% | 10.95 | `WriteBlockBench` > After: | ns/op | op/s | err% | total | benchmark |--------------------:|--------------------:|--------:|----------:|:---------- | 1,738,683.04 | 575.15 | 0.2% | 11.04 | `ReadBlockBench` | 3,052,658.88 | 327.58 | 1.0% | 10.91 | `WriteBlockBench` </details> <details> <summary>Ubuntu 24 - GNU 13.3.0</summary> > Before: | ns/op | op/s | err% | ins/op | cyc/op | IPC | bra/op | miss% | total | benchmark |--------------------:|--------------------:|--------:|----------------:|----------------:|-------:|---------------:|--------:|----------:|:---------- | 6,895,987.11 | 145.01 | 0.0% | 71,055,269.86 | 23,977,374.37 | 2.963 | 5,074,828.78 | 0.4% | 22.00 | `ReadBlockBench` | 5,152,973.58 | 194.06 | 2.2% | 19,350,886.41 | 8,784,539.75 | 2.203 | 3,079,335.21 | 0.4% | 23.18 | `WriteBlockBench` > After: | ns/op | op/s | err% | ins/op | cyc/op | IPC | bra/op | miss% | total | benchmark |--------------------:|--------------------:|--------:|----------------:|----------------:|-------:|---------------:|--------:|----------:|:---------- | 5,771,882.71 | 173.25 | 0.0% | 65,741,889.82 | 20,453,232.33 | 3.214 | 3,971,321.75 | 0.3% | 22.01 | `ReadBlockBench` | 4,145,681.13 | 241.21 | 4.0% | 15,337,596.85 | 5,732,186.47 | 2.676 | 2,239,662.64 | 0.1% | 23.94 | `WriteBlockBench` </details> 2 full IBD runs against master (compiled with GCC where the gains seem more modest) for **888888** blocks (seeded from real nodes) indicates a ~**7%** total speedup. <details> <summary>Details</summary> ```bash COMMITS="d2b72b13699cf460ffbcb1028bcf5f3b07d3b73a 652b4e3de5c5e09fb812abe265f4a8946fa96b54"; \ STOP_HEIGHT=888888; DBCACHE=1000; \ C_COMPILER=gcc; CXX_COMPILER=g++; \ BASE_DIR="/mnt/my_storage"; DATA_DIR="$BASE_DIR/BitcoinData"; LOG_DIR="$BASE_DIR/logs"; \ (for c in $COMMITS; do git fetch origin $c -q && git log -1 --pretty=format:'%h %s' $c || exit 1; done) && \ hyperfine \ --sort 'command' \ --runs 2 \ --export-json "$BASE_DIR/ibd-${COMMITS// /-}-$STOP_HEIGHT-$DBCACHE-$C_COMPILER.json" \ --parameter-list COMMIT ${COMMITS// /,} \ --prepare "killall bitcoind; rm -rf $DATA_DIR/*; git checkout {COMMIT}; git clean -fxd; git reset --hard; \ cmake -B build -DCMAKE_BUILD_TYPE=Release -DENABLE_WALLET=OFF -DCMAKE_C_COMPILER=$C_COMPILER -DCMAKE_CXX_COMPILER=$CXX_COMPILER && \ cmake --build build -j$(nproc) --target bitcoind && \ ./build/bin/bitcoind -datadir=$DATA_DIR -stopatheight=1 -printtoconsole=0; sleep 100" \ --cleanup "cp $DATA_DIR/debug.log $LOG_DIR/debug-{COMMIT}-$(date +%s).log" \ "COMPILER=$C_COMPILER COMMIT=${COMMIT:0:10} ./build/bin/bitcoind -datadir=$DATA_DIR -stopatheight=$STOP_HEIGHT -dbcache=$DBCACHE -blocksonly -printtoconsole=0" d2b72b1369 refactor: rename leftover WriteBlockBench 652b4e3de5 optimization: Bulk serialization writes in `WriteBlockUndo` and `WriteBlock` Benchmark 1: COMPILER=gcc ./build/bin/bitcoind -datadir=/mnt/my_storage/BitcoinData -stopatheight=888888 -dbcache=1000 -blocksonly -printtoconsole=0 (COMMIT = d2b72b13699cf460ffbcb1028bcf5f3b07d3b73a) Time (mean ± σ): 41528.104 s ± 354.003 s [User: 44324.407 s, System: 3074.829 s] Range (min … max): 41277.786 s … 41778.421 s 2 runs Benchmark 2: COMPILER=gcc ./build/bin/bitcoind -datadir=/mnt/my_storage/BitcoinData -stopatheight=888888 -dbcache=1000 -blocksonly -printtoconsole=0 (COMMIT = 652b4e3de5c5e09fb812abe265f4a8946fa96b54) Time (mean ± σ): 38771.457 s ± 441.941 s [User: 41930.651 s, System: 3222.664 s] Range (min … max): 38458.957 s … 39083.957 s 2 runs Relative speed comparison 1.07 ± 0.02 COMPILER=gcc ./build/bin/bitcoind -datadir=/mnt/my_storage/BitcoinData -stopatheight=888888 -dbcache=1000 -blocksonly -printtoconsole=0 (COMMIT = d2b72b13699cf460ffbcb1028bcf5f3b07d3b73a) 1.00 COMPILER=gcc ./build/bin/bitcoind -datadir=/mnt/my_storage/BitcoinData -stopatheight=888888 -dbcache=1000 -blocksonly -printtoconsole=0 (COMMIT = 652b4e3de5c5e09fb812abe265f4a8946fa96b54) ``` </details> ACKs for top commit: maflcko: re-ACK 8d801e3 🐦 achow101: ACK 8d801e3 ryanofsky: Code review ACK 8d801e3. Most notable change is switching from BufferedReader to ReadRawBlock for block reads, which makes sense, and there are also various cleanups in blockstorage and test code. hodlinator: re-ACK 8d801e3 Tree-SHA512: 24e1dee653b927b760c0ba3c69d1aba15fa5d9c4536ad11cfc2d70196ae16b9228ecc3056eef70923364257d72dc929882e73e69c6c426e28139d31299d08adc
2 parents 679bb2a + 8d801e3 commit 33df4ae

File tree

7 files changed

+330
-86
lines changed

7 files changed

+330
-86
lines changed

‎src/bench/readwriteblock.cpp

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -27,7 +27,7 @@ static CBlock CreateTestBlock()
2727
return block;
2828
}
2929

30-
static void SaveBlockBench(benchmark::Bench& bench)
30+
static void WriteBlockBench(benchmark::Bench& bench)
3131
{
3232
const auto testing_setup{MakeNoLogFileContext<const TestingSetup>(ChainType::MAIN)};
3333
auto& blockman{testing_setup->m_node.chainman->m_blockman};
@@ -63,6 +63,6 @@ static void ReadRawBlockBench(benchmark::Bench& bench)
6363
});
6464
}
6565

66-
BENCHMARK(SaveBlockBench, benchmark::PriorityLevel::HIGH);
66+
BENCHMARK(WriteBlockBench, benchmark::PriorityLevel::HIGH);
6767
BENCHMARK(ReadBlockBench, benchmark::PriorityLevel::HIGH);
6868
BENCHMARK(ReadRawBlockBench, benchmark::PriorityLevel::HIGH);

‎src/node/blockstorage.cpp

Lines changed: 68 additions & 62 deletions
Original file line numberDiff line numberDiff line change
@@ -529,7 +529,7 @@ bool BlockManager::LoadBlockIndexDB(const std::optional<uint256>& snapshot_block
529529
}
530530
for (std::set<int>::iterator it = setBlkDataFiles.begin(); it != setBlkDataFiles.end(); it++) {
531531
FlatFilePos pos(*it, 0);
532-
if (OpenBlockFile(pos, true).IsNull()) {
532+
if (OpenBlockFile(pos, /*fReadOnly=*/true).IsNull()) {
533533
return false;
534534
}
535535
}
@@ -660,27 +660,30 @@ bool BlockManager::ReadBlockUndo(CBlockUndo& blockundo, const CBlockIndex& index
660660
const FlatFilePos pos{WITH_LOCK(::cs_main, return index.GetUndoPos())};
661661

662662
// Open history file to read
663-
AutoFile filein{OpenUndoFile(pos, true)};
664-
if (filein.IsNull()) {
665-
LogError("OpenUndoFile failed for %s", pos.ToString());
663+
AutoFile file{OpenUndoFile(pos, true)};
664+
if (file.IsNull()) {
665+
LogError("OpenUndoFile failed for %s while reading block undo", pos.ToString());
666666
return false;
667667
}
668+
BufferedReader filein{std::move(file)};
668669

669-
// Read block
670-
uint256 hashChecksum;
671-
HashVerifier verifier{filein}; // Use HashVerifier as reserializing may lose data, c.f. commit d342424301013ec47dc146a4beb49d5c9319d80a
672670
try {
671+
// Read block
672+
HashVerifier verifier{filein}; // Use HashVerifier, as reserializing may lose data, c.f. commit d3424243
673+
673674
verifier << index.pprev->GetBlockHash();
674675
verifier >> blockundo;
676+
677+
uint256 hashChecksum;
675678
filein >> hashChecksum;
676-
} catch (const std::exception& e) {
677-
LogError("%s: Deserialize or I/O error - %s at %s\n", __func__, e.what(), pos.ToString());
678-
return false;
679-
}
680679

681-
// Verify checksum
682-
if (hashChecksum != verifier.GetHash()) {
683-
LogError("%s: Checksum mismatch at %s\n", __func__, pos.ToString());
680+
// Verify checksum
681+
if (hashChecksum != verifier.GetHash()) {
682+
LogError("Checksum mismatch at %s while reading block undo", pos.ToString());
683+
return false;
684+
}
685+
} catch (const std::exception& e) {
686+
LogError("Deserialize or I/O error - %s at %s while reading block undo", e.what(), pos.ToString());
684687
return false;
685688
}
686689

@@ -931,29 +934,34 @@ bool BlockManager::WriteBlockUndo(const CBlockUndo& blockundo, BlockValidationSt
931934
// Write undo information to disk
932935
if (block.GetUndoPos().IsNull()) {
933936
FlatFilePos pos;
934-
const unsigned int blockundo_size{static_cast<unsigned int>(GetSerializeSize(blockundo))};
937+
const auto blockundo_size{static_cast<uint32_t>(GetSerializeSize(blockundo))};
935938
if (!FindUndoPos(state, block.nFile, pos, blockundo_size + UNDO_DATA_DISK_OVERHEAD)) {
936-
LogError("FindUndoPos failed");
939+
LogError("FindUndoPos failed for %s while writing block undo", pos.ToString());
937940
return false;
938941
}
939-
// Open history file to append
940-
AutoFile fileout{OpenUndoFile(pos)};
941-
if (fileout.IsNull()) {
942-
LogError("OpenUndoFile failed");
943-
return FatalError(m_opts.notifications, state, _("Failed to write undo data."));
944-
}
945942

946-
// Write index header
947-
fileout << GetParams().MessageStart() << blockundo_size;
948-
// Write undo data
949-
pos.nPos += BLOCK_SERIALIZATION_HEADER_SIZE;
950-
fileout << blockundo;
943+
{
944+
// Open history file to append
945+
AutoFile file{OpenUndoFile(pos)};
946+
if (file.IsNull()) {
947+
LogError("OpenUndoFile failed for %s while writing block undo", pos.ToString());
948+
return FatalError(m_opts.notifications, state, _("Failed to write undo data."));
949+
}
950+
BufferedWriter fileout{file};
951+
952+
// Write index header
953+
fileout << GetParams().MessageStart() << blockundo_size;
954+
pos.nPos += STORAGE_HEADER_BYTES;
955+
{
956+
// Calculate checksum
957+
HashWriter hasher{};
958+
hasher << block.pprev->GetBlockHash() << blockundo;
959+
// Write undo data & checksum
960+
fileout << blockundo << hasher.GetHash();
961+
}
951962

952-
// Calculate & write checksum
953-
HashWriter hasher{};
954-
hasher << block.pprev->GetBlockHash();
955-
hasher << blockundo;
956-
fileout << hasher.GetHash();
963+
fileout.flush(); // Make sure `AutoFile`/`BufferedWriter` go out of scope before we call `FlushUndoFile`
964+
}
957965

958966
// rev files are written in block height order, whereas blk files are written as blocks come in (often out of order)
959967
// we want to flush the rev (undo) file once we've written the last block, which is indicated by the last height
@@ -986,29 +994,28 @@ bool BlockManager::ReadBlock(CBlock& block, const FlatFilePos& pos) const
986994
block.SetNull();
987995

988996
// Open history file to read
989-
AutoFile filein{OpenBlockFile(pos, true)};
990-
if (filein.IsNull()) {
991-
LogError("%s: OpenBlockFile failed for %s\n", __func__, pos.ToString());
997+
std::vector<uint8_t> block_data;
998+
if (!ReadRawBlock(block_data, pos)) {
992999
return false;
9931000
}
9941001

995-
// Read block
9961002
try {
997-
filein >> TX_WITH_WITNESS(block);
1003+
// Read block
1004+
SpanReader{block_data} >> TX_WITH_WITNESS(block);
9981005
} catch (const std::exception& e) {
999-
LogError("%s: Deserialize or I/O error - %s at %s\n", __func__, e.what(), pos.ToString());
1006+
LogError("Deserialize or I/O error - %s at %s while reading block", e.what(), pos.ToString());
10001007
return false;
10011008
}
10021009

10031010
// Check the header
10041011
if (!CheckProofOfWork(block.GetHash(), block.nBits, GetConsensus())) {
1005-
LogError("%s: Errors in block header at %s\n", __func__, pos.ToString());
1012+
LogError("Errors in block header at %s while reading block", pos.ToString());
10061013
return false;
10071014
}
10081015

10091016
// Signet only: check block solution
10101017
if (GetConsensus().signet_blocks && !CheckSignetBlockSolution(block, GetConsensus())) {
1011-
LogError("%s: Errors in block solution at %s\n", __func__, pos.ToString());
1018+
LogError("Errors in block solution at %s while reading block", pos.ToString());
10121019
return false;
10131020
}
10141021

@@ -1023,25 +1030,24 @@ bool BlockManager::ReadBlock(CBlock& block, const CBlockIndex& index) const
10231030
return false;
10241031
}
10251032
if (block.GetHash() != index.GetBlockHash()) {
1026-
LogError("%s: GetHash() doesn't match index for %s at %s\n", __func__, index.ToString(), block_pos.ToString());
1033+
LogError("GetHash() doesn't match index for %s at %s while reading block", index.ToString(), block_pos.ToString());
10271034
return false;
10281035
}
10291036
return true;
10301037
}
10311038

10321039
bool BlockManager::ReadRawBlock(std::vector<uint8_t>& block, const FlatFilePos& pos) const
10331040
{
1034-
FlatFilePos hpos = pos;
1035-
// If nPos is less than 8 the pos is null and we don't have the block data
1036-
// Return early to prevent undefined behavior of unsigned int underflow
1037-
if (hpos.nPos < 8) {
1038-
LogError("%s: OpenBlockFile failed for %s\n", __func__, pos.ToString());
1041+
if (pos.nPos < STORAGE_HEADER_BYTES) {
1042+
// If nPos is less than STORAGE_HEADER_BYTES, we can't read the header that precedes the block data
1043+
// This would cause an unsigned integer underflow when trying to position the file cursor
1044+
// This can happen after pruning or default constructed positions
1045+
LogError("Failed for %s while reading raw block storage header", pos.ToString());
10391046
return false;
10401047
}
1041-
hpos.nPos -= 8; // Seek back 8 bytes for meta header
1042-
AutoFile filein{OpenBlockFile(hpos, true)};
1048+
AutoFile filein{OpenBlockFile({pos.nFile, pos.nPos - STORAGE_HEADER_BYTES}, /*fReadOnly=*/true)};
10431049
if (filein.IsNull()) {
1044-
LogError("%s: OpenBlockFile failed for %s\n", __func__, pos.ToString());
1050+
LogError("OpenBlockFile failed for %s while reading raw block", pos.ToString());
10451051
return false;
10461052
}
10471053

@@ -1052,22 +1058,21 @@ bool BlockManager::ReadRawBlock(std::vector<uint8_t>& block, const FlatFilePos&
10521058
filein >> blk_start >> blk_size;
10531059

10541060
if (blk_start != GetParams().MessageStart()) {
1055-
LogError("%s: Block magic mismatch for %s: %s versus expected %s\n", __func__, pos.ToString(),
1056-
HexStr(blk_start),
1057-
HexStr(GetParams().MessageStart()));
1061+
LogError("Block magic mismatch for %s: %s versus expected %s while reading raw block",
1062+
pos.ToString(), HexStr(blk_start), HexStr(GetParams().MessageStart()));
10581063
return false;
10591064
}
10601065

10611066
if (blk_size > MAX_SIZE) {
1062-
LogError("%s: Block data is larger than maximum deserialization size for %s: %s versus %s\n", __func__, pos.ToString(),
1063-
blk_size, MAX_SIZE);
1067+
LogError("Block data is larger than maximum deserialization size for %s: %s versus %s while reading raw block",
1068+
pos.ToString(), blk_size, MAX_SIZE);
10641069
return false;
10651070
}
10661071

10671072
block.resize(blk_size); // Zeroing of memory is intentional here
10681073
filein.read(MakeWritableByteSpan(block));
10691074
} catch (const std::exception& e) {
1070-
LogError("%s: Read from block file failed: %s for %s\n", __func__, e.what(), pos.ToString());
1075+
LogError("Read from block file failed: %s for %s while reading raw block", e.what(), pos.ToString());
10711076
return false;
10721077
}
10731078

@@ -1077,22 +1082,23 @@ bool BlockManager::ReadRawBlock(std::vector<uint8_t>& block, const FlatFilePos&
10771082
FlatFilePos BlockManager::WriteBlock(const CBlock& block, int nHeight)
10781083
{
10791084
const unsigned int block_size{static_cast<unsigned int>(GetSerializeSize(TX_WITH_WITNESS(block)))};
1080-
FlatFilePos pos{FindNextBlockPos(block_size + BLOCK_SERIALIZATION_HEADER_SIZE, nHeight, block.GetBlockTime())};
1085+
FlatFilePos pos{FindNextBlockPos(block_size + STORAGE_HEADER_BYTES, nHeight, block.GetBlockTime())};
10811086
if (pos.IsNull()) {
1082-
LogError("FindNextBlockPos failed");
1087+
LogError("FindNextBlockPos failed for %s while writing block", pos.ToString());
10831088
return FlatFilePos();
10841089
}
1085-
AutoFile fileout{OpenBlockFile(pos)};
1086-
if (fileout.IsNull()) {
1087-
LogError("OpenBlockFile failed");
1090+
AutoFile file{OpenBlockFile(pos, /*fReadOnly=*/false)};
1091+
if (file.IsNull()) {
1092+
LogError("OpenBlockFile failed for %s while writing block", pos.ToString());
10881093
m_opts.notifications.fatalError(_("Failed to write block."));
10891094
return FlatFilePos();
10901095
}
1096+
BufferedWriter fileout{file};
10911097

10921098
// Write index header
10931099
fileout << GetParams().MessageStart() << block_size;
1100+
pos.nPos += STORAGE_HEADER_BYTES;
10941101
// Write block
1095-
pos.nPos += BLOCK_SERIALIZATION_HEADER_SIZE;
10961102
fileout << TX_WITH_WITNESS(block);
10971103
return pos;
10981104
}
@@ -1201,7 +1207,7 @@ void ImportBlocks(ChainstateManager& chainman, std::span<const fs::path> import_
12011207
if (!fs::exists(chainman.m_blockman.GetBlockPosFilename(pos))) {
12021208
break; // No block files left to reindex
12031209
}
1204-
AutoFile file{chainman.m_blockman.OpenBlockFile(pos, true)};
1210+
AutoFile file{chainman.m_blockman.OpenBlockFile(pos, /*fReadOnly=*/true)};
12051211
if (file.IsNull()) {
12061212
break; // This error is logged in OpenBlockFile
12071213
}

‎src/node/blockstorage.h

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -75,10 +75,10 @@ static const unsigned int UNDOFILE_CHUNK_SIZE = 0x100000; // 1 MiB
7575
static const unsigned int MAX_BLOCKFILE_SIZE = 0x8000000; // 128 MiB
7676

7777
/** Size of header written by WriteBlock before a serialized CBlock (8 bytes) */
78-
static constexpr size_t BLOCK_SERIALIZATION_HEADER_SIZE{std::tuple_size_v<MessageStartChars> + sizeof(unsigned int)};
78+
static constexpr uint32_t STORAGE_HEADER_BYTES{std::tuple_size_v<MessageStartChars> + sizeof(unsigned int)};
7979

8080
/** Total overhead when writing undo data: header (8 bytes) plus checksum (32 bytes) */
81-
static constexpr size_t UNDO_DATA_DISK_OVERHEAD{BLOCK_SERIALIZATION_HEADER_SIZE + uint256::size()};
81+
static constexpr uint32_t UNDO_DATA_DISK_OVERHEAD{STORAGE_HEADER_BYTES + uint256::size()};
8282

8383
// Because validation code takes pointers to the map's CBlockIndex objects, if
8484
// we ever switch to another associative container, we need to either use a
@@ -164,7 +164,7 @@ class BlockManager
164164
* blockfile info, and checks if there is enough disk space to save the block.
165165
*
166166
* The nAddSize argument passed to this function should include not just the size of the serialized CBlock, but also the size of
167-
* separator fields (BLOCK_SERIALIZATION_HEADER_SIZE).
167+
* separator fields (STORAGE_HEADER_BYTES).
168168
*/
169169
[[nodiscard]] FlatFilePos FindNextBlockPos(unsigned int nAddSize, unsigned int nHeight, uint64_t nTime);
170170
[[nodiscard]] bool FlushChainstateBlockFile(int tip_height);
@@ -400,7 +400,7 @@ class BlockManager
400400
void UpdatePruneLock(const std::string& name, const PruneLockInfo& lock_info) EXCLUSIVE_LOCKS_REQUIRED(::cs_main);
401401

402402
/** Open a block file (blk?????.dat) */
403-
AutoFile OpenBlockFile(const FlatFilePos& pos, bool fReadOnly = false) const;
403+
AutoFile OpenBlockFile(const FlatFilePos& pos, bool fReadOnly) const;
404404

405405
/** Translation to a filesystem path */
406406
fs::path GetBlockPosFilename(const FlatFilePos& pos) const;

‎src/streams.cpp

Lines changed: 16 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -87,21 +87,29 @@ void AutoFile::write(std::span<const std::byte> src)
8787
}
8888
if (m_position.has_value()) *m_position += src.size();
8989
} else {
90-
if (!m_position.has_value()) throw std::ios_base::failure("AutoFile::write: position unknown");
9190
std::array<std::byte, 4096> buf;
92-
while (src.size() > 0) {
91+
while (src.size()) {
9392
auto buf_now{std::span{buf}.first(std::min<size_t>(src.size(), buf.size()))};
94-
std::copy(src.begin(), src.begin() + buf_now.size(), buf_now.begin());
95-
util::Xor(buf_now, m_xor, *m_position);
96-
if (std::fwrite(buf_now.data(), 1, buf_now.size(), m_file) != buf_now.size()) {
97-
throw std::ios_base::failure{"XorFile::write: failed"};
98-
}
93+
std::copy_n(src.begin(), buf_now.size(), buf_now.begin());
94+
write_buffer(buf_now);
9995
src = src.subspan(buf_now.size());
100-
*m_position += buf_now.size();
10196
}
10297
}
10398
}
10499

100+
void AutoFile::write_buffer(std::span<std::byte> src)
101+
{
102+
if (!m_file) throw std::ios_base::failure("AutoFile::write_buffer: file handle is nullptr");
103+
if (m_xor.size()) {
104+
if (!m_position) throw std::ios_base::failure("AutoFile::write_buffer: obfuscation position unknown");
105+
util::Xor(src, m_xor, *m_position); // obfuscate in-place
106+
}
107+
if (std::fwrite(src.data(), 1, src.size(), m_file) != src.size()) {
108+
throw std::ios_base::failure("AutoFile::write_buffer: write failed");
109+
}
110+
if (m_position) *m_position += src.size();
111+
}
112+
105113
bool AutoFile::Commit()
106114
{
107115
return ::FileCommit(m_file);

0 commit comments

Comments
 (0)