Skip to content

Commit 338bc2c

Browse files
committed
Merge bitcoin/bitcoin#30454: build: Introduce CMake-based build system
4105129 cmake: Ignore build subdirectories within source directory (Hennadii Stepanov) 6ce50fd doc: Update for CMake-based build system (Hennadii Stepanov) 9730288 ci: Migrate CI scripts to CMake (Hennadii Stepanov) c360837 cmake, lint: Adjust `lint_includes_build_config` (Hennadii Stepanov) 3885441 cmake: Add presets for native Windows builds (Hennadii Stepanov) 7681746 cmake: Add vcpkg manifest file (Hennadii Stepanov) 8b6f1c4 cmake: Add `Coverage` and `CoverageFuzz` scripts (Hennadii Stepanov) 65bdbc1 cmake: Add `docs` build target (Hennadii Stepanov) fb75ebb cmake: Add compiler diagnostic flags (Hennadii Stepanov) e821f0a cmake: Migrate Guix build scripts to CMake (Hennadii Stepanov) 747adb6 cmake: Add `Maintenance` module (Hennadii Stepanov) 1f60b30 cmake: Add `APPEND_{CPP,C,CXX,LD}FLAGS` cache variables (Hennadii Stepanov) 2b43c45 cmake: Add `AddWindowsResources` module (Hennadii Stepanov) 973a3b0 cmake: Implement `install` build target (Hennadii Stepanov) 84ac35c cmake: Add cross-compiling support (Hennadii Stepanov) 0d01c22 build: Generate `toolchain.cmake` in depends (Hennadii Stepanov) 91a7992 depends: Add host-specific `cmake_system_version` variables (Hennadii Stepanov) 9b31209 depends: Rename `cmake_system` -> `cmake_system_name` (Hennadii Stepanov) 4a5208a Revert "build, qt: Do not install *.prl files" (Hennadii Stepanov) 6522af6 depends: Amend handling flags environment variables (Hennadii Stepanov) 90cec4d cmake: Add `MULTIPROCESS` option (Hennadii Stepanov) bb1a450 cmake: Build `bitcoin-chainstate` executable (Hennadii Stepanov) aed38ea cmake: Build `bitcoinkernel` library (Hennadii Stepanov) 975d673 cmake: Build `test_bitcoin-qt` executable (Hennadii Stepanov) 10fcc66 cmake: Add `WITH_DBUS` option (Hennadii Stepanov) 5bb5a4b cmake: Add `libqrencode` optional package support (Hennadii Stepanov) 57a6e2e cmake: Build `bitcoin-qt` executable (Hennadii Stepanov) 30f6429 cmake: Add `WERROR` option (Hennadii Stepanov) c98d4a4 cmake: Add `REDUCE_EXPORTS` option (Hennadii Stepanov) a01cb6e cmake: Add `HARDENING` option (Hennadii Stepanov) a8a2e36 cmake: Add Python-based tests (Hennadii Stepanov) 3d85379 cmake: Add fuzzing options (Hennadii Stepanov) 908530e cmake: Add `SANITIZERS` option (Hennadii Stepanov) 8bb0e85 cmake: Build `bench_bitcoin` executable (Hennadii Stepanov) 8017351 cmake: Add external signer support (Hennadii Stepanov) 353e0c9 cmake: Add `systemtap-sdt` optional package support (Hennadii Stepanov) d2fda82 cmake: Add `libzmq` optional package support (Hennadii Stepanov) ae7b39a cmake: Add `libminiupnpc` optional package support (Hennadii Stepanov) 6480e1d cmake: Add `libnatpmp` optional package support (Hennadii Stepanov) e73e930 cmake: Build `bitcoin-util` executable (Hennadii Stepanov) 027c6d7 cmake: Build `bitcoin-tx` executable (Hennadii Stepanov) d10c5c3 cmake: Add wallet functionality (Hennadii Stepanov) ab2e99b cmake: Create test suite for `ctest` (Hennadii Stepanov) 959370b cmake: Build `test_bitcoin` executable (Hennadii Stepanov) b27bf97 cmake: Build `bitcoin-cli` executable (Hennadii Stepanov) a9813df cmake: Build `bitcoind` executable (Hennadii Stepanov) 97829ce cmake: Add `FindLibevent` module (Hennadii Stepanov) 3118e40 cmake: Build `bitcoin_consensus` library (Hennadii Stepanov) 809a2f1 cmake: Build `bitcoin_util` static library (Hennadii Stepanov) 0a9a521 cmake: Build `bitcoin_crypto` library (Hennadii Stepanov) 958971f cmake: Build `univalue` static library (Hennadii Stepanov) 752747f cmake: Generate `obj/build.h` header (Hennadii Stepanov) 1f0a78e cmake: Build `minisketch` static library (Hennadii Stepanov) 12bfbc8 cmake: Build `leveldb` static library (Hennadii Stepanov) 51985c5 cmake: Build `crc32c` static library (Hennadii Stepanov) db7a198 cmake: Build `secp256k1` subtree (Hennadii Stepanov) dbb7ed1 cmake: Add `ccache` support (Hennadii Stepanov) cedfdf6 cmake: Redefine/adjust per-configuration flags (Hennadii Stepanov) b6b5e73 cmake: Add global compiler and linker flags (Hennadii Stepanov) f983279 cmake: Add `TryAppendLinkerFlag` module (Hennadii Stepanov) 4a0af29 cmake: Add `TryAppendCXXFlags` module (Hennadii Stepanov) 35cffc4 cmake: Add POSIX threads support (Hennadii Stepanov) fd72d00 cmake: Add position independent code support (Hennadii Stepanov) 07069e2 cmake: Add introspection module (Hennadii Stepanov) 27d687f cmake: Add `config/bitcoin-config.h` support (Hennadii Stepanov) fe5cdac cmake: Print compiler and linker flags in summary (Hennadii Stepanov) 7068388 cmake: Introduce interface libraries to encapsulate common flags (Hennadii Stepanov) a2317e2 cmake: Add root `CMakeLists.txt` file (Hennadii Stepanov) Pull request description: This PR introduces a new CMake-based build system, which is a drop-in replacement for the current Autotools-based build system. ML announcement: https://groups.google.com/g/bitcoindev/c/hgKkfQWzrTo As discussed during the recent CoreDev meetup in April, the switch from Autotools to CMake is intended to happen as soon as possible after branching 28.x off, which means that 29.0 will be built using CMake. This PR branch is essentially the [staging branch](https://github.com/hebasto/bitcoin/tree/cmake-staging), with every change reviewed and tested by a group of contributors, including (in alphabetical order): - [**achow101**](https://github.com/achow101) - [**fanquake**](https://github.com/fanquake) - [**maflcko**](https://github.com/maflcko) - [**m3dwards**](https://github.com/m3dwards) - [**pablomartin4btc**](https://github.com/pablomartin4btc) - [**real-or-random**](https://github.com/real-or-random) - [**ryanofsky**](https://github.com/ryanofsky) - [**sipsorcery**](https://github.com/sipsorcery) - [**TheCharlatan**](https://github.com/TheCharlatan) - [**theStack**](https://github.com/theStack) - [**theuni**](https://github.com/theuni) - [**vasild**](https://github.com/vasild) Reviewing in a separate staging repo was suggested in bitcoin/bitcoin#27060 (comment). The accompanying changes to the OSS-Fuzz project are available in hebasto/oss-fuzz#8. Please refer to the [build options parity table](https://gist.github.com/hebasto/2ef97d3a726bfce08ded9df07f7dab5e). The "auto" value is no longer available; non-default values must be specified explicitly. Additionally, the new default values have been chosen to suit the everyday build experience for the majority of developers. System requirements for using the CMake-based build system: - CMake >= 3.22 (if not available in your system's repository, it can be downloaded from https://cmake.org/download/) - a build tool of your choice: - any Make (GNU Make is no longer a requirement); GNU Make is still required to build depends - Ninja (https://ninja-build.org/) - MSBuild - Xcode A note for Windows users: The default installation of the latest version of MSVC 17.10.4 includes both CMake 3.28.3 and the vcpkg package manager). --- We, the build system developers, kindly ask reviewers to refrain from making suggestions that are not directly related to the migration process or can be implemented separately. Bugs in the scripts and errors in the updated documentation should be the focus of this PR. Please be advised that comments not aligned with this PR's goal may be ignored. Thank you all for your understanding. ACKs for top commit: maflcko: review ACK 4105129 🐥 sipsorcery: ACK 4105129. vasild: ACK 4105129 TheCharlatan: ACK 4105129 pablomartin4btc: tACK 4105129 i-am-yuvi: tACK [`4105129`](bitcoin/bitcoin@4105129) theuni: ACK 4105129. fanquake: ACK 4105129 Tree-SHA512: 6c1445054436c6c00ad63bfa0f19d64091a2b25c9bd694f85bf2218ac358ffb774d6c000685b3ca1e9b50401babed989fa2a0694b774c211d226bfd1944c9b39
2 parents 80f00ca + 4105129 commit 338bc2c

File tree

113 files changed

+5872
-518
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

113 files changed

+5872
-518
lines changed

.github/workflows/ci.yml

Lines changed: 34 additions & 115 deletions
Original file line numberDiff line numberDiff line change
@@ -67,12 +67,12 @@ jobs:
6767
echo "TEST_BASE=$(git rev-list -n$((${{ env.MAX_COUNT }} + 1)) --reverse HEAD $EXCLUDE_MERGE_BASE_ANCESTORS | head -1)" >> "$GITHUB_ENV"
6868
- run: |
6969
sudo apt-get update
70-
sudo apt-get install clang ccache build-essential libtool autotools-dev automake pkg-config bsdmainutils python3-zmq libevent-dev libboost-dev libsqlite3-dev libdb++-dev systemtap-sdt-dev libminiupnpc-dev libnatpmp-dev qtbase5-dev qttools5-dev qttools5-dev-tools qtwayland5 libqrencode-dev -y
70+
sudo apt-get install clang ccache build-essential cmake pkg-config bsdmainutils python3-zmq libevent-dev libboost-dev libsqlite3-dev libdb++-dev systemtap-sdt-dev libminiupnpc-dev libnatpmp-dev qtbase5-dev qttools5-dev qttools5-dev-tools qtwayland5 libqrencode-dev -y
7171
- name: Compile and run tests
7272
run: |
7373
# Run tests on commits after the last merge commit and before the PR head commit
7474
# Use clang++, because it is a bit faster and uses less memory than g++
75-
git rebase --exec "echo Running test-one-commit on \$( git log -1 ) && ./autogen.sh && CC=clang CXX=clang++ ./configure --with-incompatible-bdb && make clean && make -j $(nproc) check && ./test/functional/test_runner.py -j $(( $(nproc) * 2 ))" ${{ env.TEST_BASE }}
75+
git rebase --exec "echo Running test-one-commit on \$( git log -1 ) && CC=clang CXX=clang++ cmake -B build -DBUILD_GUI=ON -DBUILD_BENCH=ON -DBUILD_FUZZ_BINARY=ON -DWITH_BDB=ON -DWITH_NATPMP=ON -DWITH_MINIUPNPC=ON -DWITH_USDT=ON && cmake --build build -j $(nproc) && ctest --test-dir build -j $(nproc) && ./build/test/functional/test_runner.py -j $(( $(nproc) * 2 ))" ${{ env.TEST_BASE }}
7676
7777
macos-native-x86_64:
7878
name: 'macOS 13 native, x86_64, no depends, sqlite only, gui'
@@ -139,18 +139,17 @@ jobs:
139139
if: github.repository != 'bitcoin-core/gui' || github.event_name == 'pull_request'
140140

141141
env:
142-
CCACHE_MAXSIZE: '200M'
143-
CI_CCACHE_VERSION: '4.7.5'
144-
CI_QT_CONF: '-release -silent -opensource -confirm-license -opengl desktop -static -static-runtime -mp -qt-zlib -qt-pcre -qt-libpng -nomake examples -nomake tests -nomake tools -no-angle -no-dbus -no-gif -no-gtk -no-ico -no-icu -no-libjpeg -no-libudev -no-sql-sqlite -no-sql-odbc -no-sqlite -no-vulkan -skip qt3d -skip qtactiveqt -skip qtandroidextras -skip qtcharts -skip qtconnectivity -skip qtdatavis3d -skip qtdeclarative -skip doc -skip qtdoc -skip qtgamepad -skip qtgraphicaleffects -skip qtimageformats -skip qtlocation -skip qtlottie -skip qtmacextras -skip qtmultimedia -skip qtnetworkauth -skip qtpurchasing -skip qtquick3d -skip qtquickcontrols -skip qtquickcontrols2 -skip qtquicktimeline -skip qtremoteobjects -skip qtscript -skip qtscxml -skip qtsensors -skip qtserialbus -skip qtserialport -skip qtspeech -skip qtsvg -skip qtvirtualkeyboard -skip qtwayland -skip qtwebchannel -skip qtwebengine -skip qtwebglplugin -skip qtwebsockets -skip qtwebview -skip qtx11extras -skip qtxmlpatterns -no-openssl -no-feature-bearermanagement -no-feature-printdialog -no-feature-printer -no-feature-printpreviewdialog -no-feature-printpreviewwidget -no-feature-sql -no-feature-sqlmodel -no-feature-textbrowser -no-feature-textmarkdownwriter -no-feature-textodfwriter -no-feature-xml'
145-
CI_QT_DIR: 'qt-everywhere-src-5.15.11'
146-
CI_QT_URL: 'https://download.qt.io/official_releases/qt/5.15/5.15.11/single/qt-everywhere-opensource-src-5.15.11.zip'
147142
PYTHONUTF8: 1
148143
TEST_RUNNER_TIMEOUT_FACTOR: 40
149144

150145
steps:
151146
- name: Checkout
152147
uses: actions/checkout@v4
153148

149+
- name: Remove non-MSVC tool installations
150+
run: |
151+
Remove-Item -Path "$env:ProgramFiles/CMake" -Recurse -Force
152+
154153
- name: Configure Developer Command Prompt for Microsoft Visual C++
155154
# Using microsoft/setup-msbuild is not enough.
156155
uses: ilammy/msvc-dev-cmd@v1
@@ -159,141 +158,60 @@ jobs:
159158

160159
- name: Get tool information
161160
run: |
161+
cmake -version | Tee-Object -FilePath "cmake_version"
162+
Write-Output "---"
162163
msbuild -version | Tee-Object -FilePath "msbuild_version"
163164
$env:VCToolsVersion | Tee-Object -FilePath "toolset_version"
164-
$env:CI_QT_URL | Out-File -FilePath "qt_url"
165-
$env:CI_QT_CONF | Out-File -FilePath "qt_conf"
166165
py -3 --version
167166
Write-Host "PowerShell version $($PSVersionTable.PSVersion.ToString())"
168167
169-
- name: Restore static Qt cache
170-
id: static-qt-cache
171-
uses: actions/cache/restore@v4
172-
with:
173-
path: C:\Qt_static
174-
key: ${{ github.job }}-static-qt-${{ hashFiles('msbuild_version', 'qt_url', 'qt_conf') }}
175-
176-
- name: Build static Qt. Download
177-
if: steps.static-qt-cache.outputs.cache-hit != 'true'
178-
shell: cmd
179-
run: |
180-
curl --location --output C:\qt-src.zip %CI_QT_URL%
181-
choco install --yes --no-progress jom
182-
183-
- name: Build static Qt. Expand source archive
184-
if: steps.static-qt-cache.outputs.cache-hit != 'true'
185-
shell: cmd
186-
run: tar -xf C:\qt-src.zip -C C:\
187-
188-
- name: Build static Qt. Create build directory
189-
if: steps.static-qt-cache.outputs.cache-hit != 'true'
190-
run: |
191-
Rename-Item -Path "C:\$env:CI_QT_DIR" -NewName "C:\qt-src"
192-
New-Item -ItemType Directory -Path "C:\qt-src\build"
193-
194-
- name: Build static Qt. Configure
195-
if: steps.static-qt-cache.outputs.cache-hit != 'true'
196-
working-directory: C:\qt-src\build
197-
shell: cmd
198-
run: ..\configure %CI_QT_CONF% -prefix C:\Qt_static
199-
200-
- name: Build static Qt. Build
201-
if: steps.static-qt-cache.outputs.cache-hit != 'true'
202-
working-directory: C:\qt-src\build
203-
shell: cmd
204-
run: jom
205-
206-
- name: Build static Qt. Install
207-
if: steps.static-qt-cache.outputs.cache-hit != 'true'
208-
working-directory: C:\qt-src\build
209-
shell: cmd
210-
run: jom install
211-
212-
- name: Save static Qt cache
213-
if: steps.static-qt-cache.outputs.cache-hit != 'true'
214-
uses: actions/cache/save@v4
215-
with:
216-
path: C:\Qt_static
217-
key: ${{ github.job }}-static-qt-${{ hashFiles('msbuild_version', 'qt_url', 'qt_conf') }}
218-
219-
- name: Ccache installation cache
220-
id: ccache-installation-cache
221-
uses: actions/cache@v4
222-
with:
223-
path: |
224-
C:\ProgramData\chocolatey\lib\ccache
225-
C:\ProgramData\chocolatey\bin\ccache.exe
226-
C:\ccache\cl.exe
227-
key: ${{ github.job }}-ccache-installation-${{ env.CI_CCACHE_VERSION }}
228-
229-
- name: Install Ccache
230-
if: steps.ccache-installation-cache.outputs.cache-hit != 'true'
231-
run: |
232-
choco install --yes --no-progress ccache --version=$env:CI_CCACHE_VERSION
233-
New-Item -ItemType Directory -Path "C:\ccache"
234-
Copy-Item -Path "$env:ChocolateyInstall\lib\ccache\tools\ccache-$env:CI_CCACHE_VERSION-windows-x86_64\ccache.exe" -Destination "C:\ccache\cl.exe"
235-
236-
- name: Restore Ccache cache
237-
id: ccache-cache
238-
uses: actions/cache/restore@v4
239-
with:
240-
path: ~/AppData/Local/ccache
241-
key: ${{ github.job }}-ccache-${{ github.run_id }}
242-
restore-keys: ${{ github.job }}-ccache-
243-
244168
- name: Using vcpkg with MSBuild
245169
run: |
246170
Set-Location "$env:VCPKG_INSTALLATION_ROOT"
171+
Add-Content -Path "triplets\x64-windows.cmake" -Value "set(VCPKG_BUILD_TYPE release)"
247172
Add-Content -Path "triplets\x64-windows-static.cmake" -Value "set(VCPKG_BUILD_TYPE release)"
248-
.\vcpkg.exe --vcpkg-root "$env:VCPKG_INSTALLATION_ROOT" integrate install
249-
git rev-parse HEAD | Tee-Object -FilePath "$env:GITHUB_WORKSPACE\vcpkg_commit"
250173
251174
- name: vcpkg tools cache
252175
uses: actions/cache@v4
253176
with:
254177
path: C:/vcpkg/downloads/tools
255178
key: ${{ github.job }}-vcpkg-tools
256179

257-
- name: vcpkg binary cache
258-
uses: actions/cache@v4
180+
- name: Restore vcpkg binary cache
181+
uses: actions/cache/restore@v4
182+
id: vcpkg-binary-cache
259183
with:
260184
path: ~/AppData/Local/vcpkg/archives
261-
key: ${{ github.job }}-vcpkg-binary-${{ hashFiles('vcpkg_commit', 'msbuild_version', 'toolset_version', 'build_msvc/vcpkg.json') }}
262-
263-
- name: Generate project files
264-
run: py -3 build_msvc\msvc-autogen.py
185+
key: ${{ github.job }}-vcpkg-binary-${{ hashFiles('cmake_version', 'msbuild_version', 'toolset_version', 'vcpkg.json') }}
265186

266-
- name: Build
267-
shell: cmd
187+
- name: Generate build system
268188
run: |
269-
ccache --zero-stats
270-
msbuild build_msvc\bitcoin.sln -property:CLToolPath=C:\ccache;CLToolExe=cl.exe;UseMultiToolTask=true;Configuration=Release -maxCpuCount -verbosity:minimal -noLogo
271-
272-
- name: Ccache stats
273-
run: ccache --show-stats
189+
cmake -B build --preset vs2022-static -DCMAKE_TOOLCHAIN_FILE="$env:VCPKG_INSTALLATION_ROOT\scripts\buildsystems\vcpkg.cmake" -DBUILD_BENCH=ON -DBUILD_FUZZ_BINARY=ON -DWERROR=ON
274190
275-
- name: Save Ccache cache
191+
- name: Save vcpkg binary cache
276192
uses: actions/cache/save@v4
277-
if: github.event_name != 'pull_request' && steps.ccache-cache.outputs.cache-hit != 'true'
193+
if: github.event_name != 'pull_request' && steps.vcpkg-binary-cache.outputs.cache-hit != 'true'
278194
with:
279-
path: ~/AppData/Local/ccache
280-
# https://github.com/actions/cache/blob/main/tips-and-workarounds.md#update-a-cache
281-
key: ${{ github.job }}-ccache-${{ github.run_id }}
282-
283-
- name: Run unit tests
284-
run: src\test_bitcoin.exe -l test_suite
285-
286-
- name: Run benchmarks
287-
run: src\bench_bitcoin.exe -sanity-check
195+
path: ~/AppData/Local/vcpkg/archives
196+
key: ${{ github.job }}-vcpkg-binary-${{ hashFiles('cmake_version', 'msbuild_version', 'toolset_version', 'vcpkg.json') }}
288197

289-
- name: Run util tests
290-
run: py -3 test\util\test_runner.py
198+
- name: Build
199+
working-directory: build
200+
run: |
201+
cmake --build . -j $env:NUMBER_OF_PROCESSORS --config Release
291202
292-
- name: Run rpcauth test
293-
run: py -3 test\util\rpcauth-test.py
203+
- name: Run test suite
204+
working-directory: build
205+
run: |
206+
ctest -j $env:NUMBER_OF_PROCESSORS -C Release
294207
295208
- name: Run functional tests
209+
working-directory: build
296210
env:
211+
BITCOIND: '${{ github.workspace }}\build\src\Release\bitcoind.exe'
212+
BITCOINCLI: '${{ github.workspace }}\build\src\Release\bitcoin-cli.exe'
213+
BITCOINUTIL: '${{ github.workspace }}\build\src\Release\bitcoin-util.exe'
214+
BITCOINWALLET: '${{ github.workspace }}\build\src\Release\bitcoin-wallet.exe'
297215
TEST_RUNNER_EXTRA: ${{ github.event_name != 'pull_request' && '--extended' || '' }}
298216
shell: cmd
299217
run: py -3 test\functional\test_runner.py --jobs %NUMBER_OF_PROCESSORS% --ci --quiet --tmpdirprefix=%RUNNER_TEMP% --combinedlogslen=99999999 --timeout-factor=%TEST_RUNNER_TIMEOUT_FACTOR% %TEST_RUNNER_EXTRA%
@@ -306,8 +224,9 @@ jobs:
306224
git log -1
307225
308226
- name: Run fuzz binaries
227+
working-directory: build
309228
env:
310-
BITCOINFUZZ: "${{ github.workspace}}\\src\\fuzz.exe"
229+
BITCOINFUZZ: '${{ github.workspace }}\build\src\test\fuzz\Release\fuzz.exe'
311230
shell: cmd
312231
run: py -3 test\fuzz\test_runner.py --par %NUMBER_OF_PROCESSORS% --loglevel DEBUG %RUNNER_TEMP%\qa-assets\fuzz_seed_corpus
313232

.gitignore

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,8 @@
1+
# Build subdirectories.
2+
/*build*
3+
!/build-aux
4+
!/build_msvc
5+
16
*.tar.gz
27

38
*.exe

0 commit comments

Comments
 (0)