update bitcoin to 0.18.1 #1
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Adding these checks necessitates increasing the length of the generated test chains so that we can properly exceed the REDOWNLOAD_BUFFER_SIZE during the test. One can check out this commit and locally revert the TARGET_BLOCKS value change to prove the need for tests being able to control the buffer size, as is done by the next commit. Beyond the current REDOWNLOAD_BUFFER_SIZE of 15'009 we need 3 extra - 15'012 TARGET_BLOCKS: * 1 for the genesis block. * 1 for the test wanting to check that we start receiving headers for permanent storage *before* the final header (first_chain.back()). * 1 to exceed REDOWNLOAD_BUFFER_SIZE in HeadersSyncState::PopHeadersReadyForAcceptance(). (The release process includes an occasional increase of the REDOWNLOAD_BUFFER_SIZE value, see release-process.md and history of headerssync.cpp).
Move calculated constants from the top of src/headerssync.cpp into src/kernel/chainparams.cpp. Instead of being hardcoded to mainnet parameters, HeadersSyncState can now vary depending on chain or test. (This means we can reset TARGET_BLOCKS back to the nice round number of 15'000). Signet and testnets got new HeadersSyncParams constants through temporarily altering headerssync-params.py with corresponding GENESIS_TIME and MINCHAINWORK_HEADERS (based off defaultAssumeValid block height comments, corresponding to nMinimumChainWork). Regtest doesn't have a default assume valid block height, so the values are copied from Testnet 4. Since the constants only affect memory usage, and have very low impact unless dealing with a largely malicious chain, it's not that critical to keep updating them for non-mainnet chains. GENESIS_TIMEs (UTC): Testnet3: 1296688602 = datetime(2011, 2, 2) Testnet4: 1714777860 = datetime(2024, 5, 3) Signet: 1598918400 = datetime(2020, 9, 1)
9193c3e cmake: Fix regression in `secp256k1.cmake` (Hennadii Stepanov) Pull request description: This PR fixes a regression introduced in #33101 (mea culpa). From the CMake [docs](https://cmake.org/cmake/help/latest/command/enable_language.html): > The following restrictions apply to where `enable_language()` may be called: > > - It must be called in file scope, not in a function call. Fixes #33153. ACKs for top commit: TheCharlatan: ACK 9193c3e furszy: ACK 9193c3e Tree-SHA512: 5f9ca2209af195a5eefefffdceae1acf650db29b371616f803b482a6b8acc4e87acc66714488520ef91d06ad4436438d9030be9979bd4c482cb5d1ce65b7f67e
Co-Authored-By: Sjors Provoost <[email protected]>
mzumsande pointed out #32345 (comment) that this test was causing a warning: Warning: Disk space for "/tmp/test_common bitcoin/node_init_tests/init_test/bf78678cb7723a3e84b5/blocks" may not accommodate the block files. Approximately 810 GB of data will be stored in this directory. Fix by setting regtest instead of mainnet network before running the test.
bdf01c6 test: Prevent disk space warning during node_init_tests (Ryan Ofsky) Pull request description: mzumsande pointed out #32345 (comment) that this test was print a warning: ``` Warning: Disk space for "/tmp/test_common bitcoin/node_init_tests/init_test/bf78678cb7723a3e84b5/blocks" may not accommodate the block files. Approximately 810 GB of data will be stored in this directory. ``` Fix by setting regtest instead of mainnet network before running the test. ACKs for top commit: achow101: ACK bdf01c6 Eunovo: Tested ACK bdf01c6: janb84: ACK bdf01c6 l0rinc: tested ACK bdf01c6 mzumsande: utACK bdf01c6 enirox001: utACK bdf01c6 Tree-SHA512: ac4e1e48246c84a4c4b80ccb25e962b0090359ab0e541ee4f1a9e18ac9da8ec35a78c9a55501d231423053e945ff785862f0db141d4b620d622327670c764f8c
28efd72 depends: systemtap 5.3 (fanquake) Pull request description: The diff in the copied header is: ```diff < #if __STDC_VERSION__ >= 199901L --- > #if defined(__STDC_VERSION__) && __STDC_VERSION__ >= 199901L ``` From https://sourceware.org/git/?p=systemtap.git;a=commit;h=b8345d8e07b725a943a97b19aa4866e74baadd98. ACKs for top commit: 0xB10C: ACK 28efd72 Tree-SHA512: 30eb4656d354e463937b68f8977d3c3bfe1e5a63be19a9e446a41cabcd1222290f4c7ecc64e516c4f5cc38b744229906fd92a9a56f4e1380dfb8db9ed518ebb4
These methods in the Sock class wrap corresponding syscalls, accepting void* arguments and casting to char* internally, which is needed for Windows support and ignored on other platforms because the syscall itself accepts void*: Send() Recv() GetSockOpt() SetSockOpt()
0a26731 test: Add submitblock test in interface_ipc (TheCharlatan) Pull request description: Expands the ipc mining test a bit with submitting a solved block and checking its validity. ACKs for top commit: Sjors: ACK 0a26731 marcofleon: code review ACK 0a26731 zaidmstrr: Tested ACK [0a26731](0a26731) Tree-SHA512: 35c87d88496eec469bddedf2ae82c494626abb47ae15d5a45d6ab0400199c86501199c3e569e83836549830042be76b197b470e1100a317bdfef2578a9d5a92f
Tor inbound connections do not reveal the peer's actual network address. Therefore do not apply whitelist permissions to them. Co-authored-by: Vasil Dimov <[email protected]>
The dynamically created signing context for libsecp256k1 calls is only needed for functions that involve generator point multiplication with a secret key, i.e. different variants of public key creation and signing. The API docs hint to this by stating "not secp256k1_context_static" for the context parameter. In our case that applies to the following calls: - `secp256k1_ec_pubkey_create` - `secp256k1_keypair_create` - `secp256k1_ellswift_create` - `secp256k1_ecdsa_sign` - `secp256k1_ecdsa_sign_recoverable` - `secp256k1_schnorrsig_sign32` - `ec_seckey_export_der` (not a direct secp256k1 function, but calls `secp256k1_ec_pubkey_create` inside) For all the other secp256k1 calls we can simply use the static context.
This change updates the vcpkg manifest baseline from the "2025.03.19 Release" to the "2025.08.27 Release", with the following package changes: - boost: 1.87.0 --> 1.88.0 - qtbase: 6.8.2#1 -> 6.9.1 - qttools: 6.8.2 -> 6.9.1 - sqlite3: 3.49.1 --> 3.50.4
f563ce9 net: Do not apply whitelist permission to onion inbounds (Martin Zumsande) Pull request description: Tor inbound connections do not reveal the peer's actual network address. Do not apply whitelist permissions to them since address-based matching is ineffective. ACKs for top commit: darosior: ACK f563ce9 furszy: ACK f563ce9 vasild: ACK f563ce9 Tree-SHA512: 49ae70e382fc2f78b7073553fe649a6843a41214b2986ea7f77e285d02b7bd00fe0320a1b71d1aaca08713808fb14af058f0b1f19f19adb3a77b97cb9d3449ce
…le to distinguish these in tests
7584a4f cmake: Install `bitcoin` manpage (Hennadii Stepanov) Pull request description: This PR is an amendment to #31375. ACKs for top commit: ryanofsky: Code review ACK 7584a4f. Tree-SHA512: 66810c1d65fa8ae469b8161a5f807aa7b43a7b18e88d40b05617c7110b2e03e07bcb8f310c1736fb2c3738e274fc524032ff5d34d5c644824a4edd64372f1e9f
Choose the right binary by default if an IPC option is specified
…b8a552 47d79db8a552 Merge bitcoin-core/libmultiprocess#201: bug: fix mptest hang, ProxyClient<Thread> deadlock in disconnect handler f15ae9c9b9fb Merge bitcoin-core/libmultiprocess#211: Add .gitignore 4a269b21b8c8 bug: fix ProxyClient<Thread> deadlock if disconnected as IPC call is returning 85df96482c49 Use try_emplace in SetThread instead of threads.find ca9b380ea91a Use std::optional in ConnThreads to allow shortening locks 9b0799113557 doc: describe ThreadContext struct and synchronization requirements d60db601ed9b proxy-io.h: add Waiter::m_mutex thread safety annotations 4e365b019a9f ci: Use -Wthread-safety not -Wthread-safety-analysis 15d7bafbb001 Add .gitignore fe1cd8c76131 Merge bitcoin-core/libmultiprocess#208: ci: Test minimum cmake version in olddeps job b713a0b7bfbc Merge bitcoin-core/libmultiprocess#207: ci: output CMake version in CI script 0f580397c913 ci: Test minimum cmake version in olddeps job d603dcc0eef0 ci: output CMake version in CI script git-subtree-dir: src/ipc/libmultiprocess git-subtree-split: 47d79db8a5528097b408e18f7b0bae11a6702d26
This change should fix issue #33417 reported by zaidmstrr. It's possible to reproduce the `mp/proxy.capnp:0: failed: Duplicate ID @0xcc316e3f71a040fb` error by installing libmultiprocess system-wide, or to one of the locations listed in the python test's `imports` list before the local libmultiprocess subtree, and then running the test.
Added a minimal system helper to query total physical RAM on [Linux/macOS/Windows](https://stackoverflow.com/a/2513561) (on other platforms we just return an empty optional). The added test checks if the value is roughly correct by checking if the CI platforms are returning any value and if the value is at least 1 GiB and not more than 10 TiB. The max value is only validated on 64 bits, since it's not unreasonable for 32 bits to have max memory, but on 64 bits it's likely an error. https://learn.microsoft.com/en-us/windows/win32/api/sysinfoapi/ns-sysinfoapi-memorystatusex > ullTotalPhys The amount of actual physical memory, in bytes. https://man7.org/linux/man-pages/man3/sysconf.3.html: > _SC_PHYS_PAGES The number of pages of physical memory. Note that it is possible for the product of this value and the value of _SC_PAGESIZE to overflow. > _SC_PAGESIZE Size of a page in bytes. Must not be less than 1. See https://godbolt.org/z/ec81Tjvrj for further details
Oversized allocations can cause out-of-memory errors or [heavy swapping](getumbrel/umbrel-os#64 (comment)), [grinding the system to a halt](https://x.com/murchandamus/status/1964432335849607224). `LogOversizedDbCache()` now emits a startup warning if the configured `-dbcache` exceeds a cap derived from system RAM, using the same parsing/clamping as cache sizing via CalculateDbCacheBytes(). This isn't meant as a recommended setting, rather a likely upper limit. Note that we're not modifying the set value, just issuing a warning. Also note that the 75% calculation is rounded for the last two numbers since we have to divide first before multiplying, otherwise we wouldn't stay inside size_t on 32-bit systems - and this was simpler than casting back and forth. We could have chosen the remaining free memory for the warning (e.g. warn if free memory is less than 1 GiB), but this is just a heuristic, we assumed that on systems with a lot of memory, other processes are also running, while memory constrained ones run only Core. If total RAM < 2 GiB, cap is `DEFAULT_DB_CACHE` (`450 MiB`), otherwise it's 75% of total RAM. The threshold is chosen to be close to values commonly used in [raspiblitz](https://github.com/raspiblitz/raspiblitz/blob/dev/home.admin/_provision.setup.sh#L98-L115) for common setups: | Total RAM | `dbcache` (MiB) | raspiblitz % | proposed cap (MiB) | |----------:|----------------:|-------------:|-------------------:| | 1 GiB | 512 | 50.0% | 450* | | 2 GiB | 1536 | 75.0% | 1536 | | 4 GiB | 2560 | 62.5% | 3072 | | 8 GiB | 4096 | 50.0% | 6144 | | 16 GiB | 4096 | 25.0% | 12288 | | 32 GiB | 4096 | 12.5% | 24576 | [Umbrel issues](getumbrel/umbrel-os#64 (comment)) also mention 75% being the upper limit. Starting `bitcoind` on an 8 GiB rpi4b with a dbcache of 7 GiB: > ./build/bin/bitcoind -dbcache=7000 warns now as follows: ``` 2025-09-07T17:24:29Z [warning] A 7000 MiB dbcache may be too large for a system memory of only 7800 MiB. 2025-09-07T17:24:29Z Cache configuration: 2025-09-07T17:24:29Z * Using 2.0 MiB for block index database 2025-09-07T17:24:29Z * Using 8.0 MiB for chain state database 2025-09-07T17:24:29Z * Using 6990.0 MiB for in-memory UTXO set (plus up to 286.1 MiB of unused mempool space) ``` Besides the [godbolt](https://godbolt.org/z/EPsaE3xTj) reproducers for the new total memory method, we also tested the warnings manually on: - [x] Apple M4 Max, macOS 15.6.1 - [x] Intel Core i9-9900K, Ubuntu 24.04.2 LTS - [x] Raspberry Pi 4 Model B, Armbian Linux 6.12.22-current-bcm2711 - [x] Intel Xeon x64, Windows 11 Home Version 24H2, OS Build 26100.4351 Co-authored-by: stickies-v <[email protected]> Co-authored-by: Hodlinator <[email protected]> Co-authored-by: w0xlt <[email protected]>
e9c5227 test: Avoid interface_ipc.py Duplicate ID errors (Ryan Ofsky) Pull request description: This change should fix issue #33417 reported by zaidmstrr. It's possible to reproduce the `mp/proxy.capnp:0: failed: Duplicate ID @0xcc316e3f71a040fb` error by installing libmultiprocess system-wide, or to one of the locations listed in the python test's `imports` list before the local libmultiprocess subtree, and then running the test. ACKs for top commit: zaidmstrr: Tested ACK [e9c5227](e9c5227) Tree-SHA512: 5df7fe767989b91245ce96f7c43b6767b7af49ec6c7007175e462341ffd69e161f21632697804060ce286b3e102a8d141a57a53f7e0e32299ef9a3a69ca8794a
Throw RPC_CLIENT_INVALID_IP_OR_SUBNET when LookupHost(addr, false) fails
in addpeeraddress. This aligns with setban/addconnection and avoids the
opaque {"success": false} result for input errors. The JSON {success,
error?} object remains for addrman outcomes only. Update test to match.
168360f coins: warn on oversized -dbcache (Lőrinc) 6c72045 system: add helper for fetching total system memory (Lőrinc) Pull request description: ### Summary Oversized allocations can cause out-of-memory errors or [heavy swapping](getumbrel/umbrel-os#64 (comment)), [grinding the system to a halt](https://x.com/murchandamus/status/1964432335849607224). ### Fix Added a minimal system helper to query total physical RAM on [Linux/macOS/Windows](https://stackoverflow.com/a/2513561) (on unsupported platforms we just disable this warning completely). The added test checks if the value is roughly correct by checking if the CI platforms are returning any value and if the value is at least 1 GB (as a simple property test checking if the unit size is correct, e.g. doesn't return megabytes or bits). ### Details `LogOversizedDbCache()` now emits a startup warning if the configured `-dbcache` exceeds a cap derived from system RAM, using the same parsing/clamping as cache sizing via `CalculateDbCacheBytes()`. This isn't meant as a recommended setting, rather a likely upper limit. Note that we're not modifying the set value, just issuing a warning. Also note that the 75% calculation is rounded for the last two numbers since we have to divide first before multiplying, otherwise we wouldn't stay inside `size_t` on 32-bit systems - and this was simpler than casting back and forth. We could have chosen the remaining free memory for the warning (e.g. warn if free memory is less than 1 GiB), but this is just a heuristic, we assumed that on systems with a lot of memory, other processes are also running, while memory constrained ones run only Core. ### Cap If total RAM < 2 GiB, cap is `DEFAULT_DB_CACHE` (`450 MiB`), otherwise it's 75% of total RAM. The threshold is chosen to be close to values commonly used in [raspiblitz](https://github.com/raspiblitz/raspiblitz/blob/dev/home.admin/_provision.setup.sh#L98-L115) for common setups: | Total RAM | `dbcache` (MiB) | raspiblitz % | proposed cap (MiB) | |----------:|----------------:|-------------:|-------------------:| | 1 GiB | 512 | 50.0% | 450* | | 2 GiB | 1536 | 75.0% | 1536 | | 4 GiB | 2560 | 62.5% | 3072 | | 8 GiB | 4096 | 50.0% | 6144 | | 16 GiB | 4096 | 25.0% | 12288 | | 32 GiB | 4096 | 12.5% | 24576 | [Umbrel issues](getumbrel/umbrel-os#64 (comment)) also mention 75% being the upper limit. ### Reproducer Starting `bitcoind` on an 8 GiB rpi4b with a dbcache of 7 GiB: > ./build/bin/bitcoind -dbcache=7000 warns now as follows: ``` 2025-09-07T17:24:29Z [warning] A 7000 MiB dbcache may be too large for a system memory of only 7800 MiB. Warning: A 7000 MiB dbcache may be too large for a system memory of only 7800 MiB. 2025-09-07T17:24:29Z Cache configuration: 2025-09-07T17:24:29Z * Using 2.0 MiB for block index database 2025-09-07T17:24:29Z * Using 8.0 MiB for chain state database 2025-09-07T17:24:29Z * Using 6990.0 MiB for in-memory UTXO set (plus up to 286.1 MiB of unused mempool space) ``` ### Manual testing Besides the [godbolt](https://godbolt.org/z/ec81Tjvrj) reproducers for the new total memory method, we also tested the warnings manually on: - [x] Apple M4 Max, macOS 15.6.1 - [x] Intel Core i9-9900K, Ubuntu 24.04.2 LTS - [x] Raspberry Pi 4 Model B, Armbian Linux 6.12.22-current-bcm2711 - [x] Intel Xeon x64, Windows 11 Home Version 24H2, OS Build 26100.4351 ACKs for top commit: achow101: ACK 168360f w0xlt: reACK 168360f hodlinator: re-ACK 168360f danielabrozzoni: reACK 168360f Tree-SHA512: aa0c9b1034d55a6a4212685a19715d8cd89668ab7c33c688711a15559e6ad81aa65f3cd8b488c91385306e1e16cd9eeefa8f659ba90ef19ce9c7a2e64f8b561a
67f632b net: remove unnecessary casts in socket operations (Matthew Zipkin) Pull request description: During review of #32747 several casting operations were questioned in existing code that had been copied or moved. That lead me to find a few other similar casts in the codebase. It turns out that since the `Sock` class wraps syscalls with its own internal casting (see #24357 and #20788 written in 2020-2022) we no longer need to cast the arguments when calling these functions. The original argument-casts are old and were cleaned up a bit in #12855 written in 2018. The casting is only needed for windows compatibility, where those syscalls require a data argument to be of type `char*` specifically: https://learn.microsoft.com/en-us/windows/win32/api/winsock/nf-winsock-getsockopt ``` int getsockopt( [in] SOCKET s, [in] int level, [in] int optname, [out] char *optval, [in, out] int *optlen ); ``` but on POSIX the argument is `void*`: https://www.man7.org/linux/man-pages/man2/getsockopt.2.html ``` int getsockopt(socklen *restrict optlen; int sockfd, int level, int optname, void optval[_Nullable restrict *optlen], socklen_t *restrict optlen); ``` ACKs for top commit: Raimo33: ACK 67f632b achow101: ACK 67f632b hodlinator: ACK 67f632b vasild: ACK 67f632b davidgumberg: ACK 67f632b Tree-SHA512: c326d7242698b8d4d019f630fb6281398da2773c4e5aad1e3bba093a012c2119ad8815f42bd009e61a9a90db9b8e6ed5c75174aac059c9df83dd3aa5618a9ba6
fa75ef4 test: Move export_env_build_path to util.py (MarcoFalke) fa9f495 test: Move get_binary_paths and Binaries to util.py (MarcoFalke) Pull request description: Having the binary related utils sit in the test_framework.py is fine. However, they are mostly stand-alone utils, which may be used externally. So move them to utils.py, to allow easier external use. The diff is trivial and can be reviewed via the git options `--color-moved=dimmed-zebra --color-moved-ws=ignore-all-space`. ACKs for top commit: kevkevinpal: ACK [fa75ef4](fa75ef4) Sjors: lgtm ACK fa75ef4 yuvicc: Code review ACK fa75ef4 janb84: ACK fa75ef4 musaHaruna: Code Review ACK [fa75ef4](fa75ef4) enirox001: ACK [fa75ef4](fa75ef4) Tree-SHA512: f382118484cb5495e8888214437e72c81727d54f97b3c09dfd996faab6cb6643c4c2d816b89ab82de73fc091c36ed7b8744c7d34a443b6adc415eb06697ef6ea
… script 4b41f99 build: Move CMAKE_SKIP_INSTALL_RPATH from CMake to Guix script (Henry Romp) Pull request description: Remove `CMAKE_SKIP_INSTALL_RPATH` from CMakeLists.txt and add `CMAKE_SKIP_RPATH` to the Guix build script. This keeps build-environment-specific settings in the build scripts rather than hardcoded in the CMake configuration. ACKs for top commit: purpleKarrot: ACK 4b41f99 janb84: re ACK 4b41f99 Tree-SHA512: 74d6af382476d731f10f9833978d670e9981c160ba306d0e9d4b1ad1e9b9960b8d03a3b9b608e234edb1c0c2c7a2b4f9f606a2a7887b7a153792159e71ae9b21
The option is currently unused. If it is used again in the future, it could trivially be added back. Also, the logic is just a single undocumented python command one-liner. So remove it for now.
This has a few benefits: * The shellcheck SC2086 warning is disabled for the whole command, but is only needed for the DOCKER_BUILD_CACHE_ARG env var. So in Python, only pass this one env var to shlex.split() for proper word splitting. * Future logic improvements can be implemented in Python. The comments are moved, which can be checked via the git options: --color-moved=dimmed-zebra --color-moved-ws=ignore-all-space
Other cache providers offer too little space for this to be useful.
…nds-usdt workflow f031536 ci: use Mold linker for asan-lsan-ubsan-integer-no-depends-usdt workflow (Brandon Odiwuor) Pull request description: Follow up to #32888 (review) and #32888 (comment) >>Can we use `mold` as a linker in other Linux based system workflows ? dependencies [we have](https://github.com/bitcoin/bitcoin/blob/master/doc/dependencies.md#compiler) seem to satisfy the deps here https://github.com/rui314/mold?tab=readme-ov-file#how-to-build > > Sure, happy to review a follow-up. Only place to avoid it would probably the ci tasks that mirror the guix build (win-cross, mac-cross) Updated the `ASan + LSan + UBSan + integer, no depends, USDT` workflow to use `mold` linker ACKs for top commit: maflcko: lgtm ACK f031536 Tree-SHA512: 35a4cb3eec732bee3f18a3ea70e49b1c99b8e88624a0bb28eca8f3d72ed0835af8773307a27c750b89fc6d969ff20dd87b840d755b7fd14d3cb6ab68d9f587b9
c864a4c Simplify fs::path by dropping filename() and make_preferred() overloads (Ryan Ofsky) b0113af Fix windows libc++ fs::path fstream compile errors (Ryan Ofsky) Pull request description: Drop support for passing `fs::path` directly to `std::ifstream` and `std::ofstream` constructors and `open()` functions, because as reported by hebasto in #33545, after https://wg21.link/lwg3430 there is no way this can continue to work in windows builds, and there are already compile errors compiling for windows with newer versions of libc++. Instead, add an `fs::path::std_path()` method that returns `std::filesystem::path` references and use it where needed. ACKs for top commit: hebasto: ACK c864a4c. l0rinc: Code review ACK c864a4c maflcko: re-ACK c864a4c 🌥 Tree-SHA512: d22372692ab86244e2b2caf4c5e9c9acbd9ba38df5411606b75e428474eabead152fc7ca1afe0bb0df6b818351211a70487e94b40a17b68db5aa757604a0ddf6
…0 in p2p_i2p_ports.py fa20275 test: Use unassigned p2p_port instead of hardcoded 60000 in p2p_i2p_ports.py (MarcoFalke) Pull request description: The goal is to fix #30030. The root cause it unclear. However, hard-coding the port to 60000 does not seem ideal anyway. This could break in an unlikely setting where so many functional tests are run, such that the port is occupied. Also, it could fail when `TEST_RUNNER_PORT_MIN` is set sufficiently high. (This is purely theoretical, as I don't think anyone would run a command like this, but on current master it fails, and on this pull it passes: `TEST_RUNNER_PORT_MIN=60000 ./bld-cmake/test/functional/p2p_i2p_ports.py --portseed=0`) So fix those issues (and hopefully also 30030) by using an unoccupied p2p_port. The logic is similar to the `extra_port()` logic in the `feature_bind_extra.py` test. ACKs for top commit: laanwj: Code review ACK fa20275 mzumsande: ACK fa20275 Tree-SHA512: ac5487ca195db9ca746b78b8add91d0b9ef59cc3be0cdb7fbd9f76d42549eea68a61c32b4f5a162e01f3777959110f9f8d56ff05af6a13a9f61ea5be5b7d8631
9610b0d randomenv: Fix MinGW dllimport warning for `environ` (Lőrinc) Pull request description: Related to #33550 (comment) Extends 7703884 to guard environ declaration on all Windows builds, not just MSVC. In the `mingw-w64` headers (used by `llvm-mingw`), `environ` is defined as a macro which expands through [`_environ`](https://github.com/msys2-contrib/mingw-w64/blob/cdb052f1d4056cd510cb83197b55868427b87476/mingw-w64-headers/crt/stdlib.h#L262-L264) to `(* __p__environ())`, a call to a `dllimport` function, causing the same inconsistent linkage warning as MSVC. Use `WIN32` instead of `_MSC_VER` to match the platform-specific guards already used throughout the file. The warning occurs with `llvm-mingw` (both `UCRT` and `MSVCRT` variants as tested by Hebasto), but not with the `mingw-w64` toolchain currently used in CI (as mentioned by fanquake). ---- The error was reproduced by adding a temporary [nightly build](l0rinc/bitcoin-core-nightly#4) pointing to l0rinc#45. On `master` the failure can be seen in l0rinc/bitcoin-core-nightly#2 before: https://github.com/l0rinc/bitcoin-core-nightly/actions/runs/18327936488/job/52196728885?pr=2 <details> <summary>Details</summary> ``` /home/runner/work/bitcoin-core-nightly/bitcoin-core-nightly/src/randomenv.cpp:61:15: warning: '__p__environ' redeclared without 'dllimport' attribute: previous 'dllimport' ignored [-Winconsistent-dllimport] 61 | extern char** environ; // NOLINT(readability-redundant-declaration): Necessary on some platforms | ^ /home/runner/work/bitcoin-core-nightly/bitcoin-core-nightly/llvm_mingw_toolchain/aarch64-w64-mingw32/include/stdlib.h:656:17: note: expanded from macro 'environ' 656 | #define environ _environ | ^ /home/runner/work/bitcoin-core-nightly/bitcoin-core-nightly/llvm_mingw_toolchain/aarch64-w64-mingw32/include/stdlib.h:225:21: note: expanded from macro '_environ' 225 | #define _environ (* __p__environ()) | ^ /home/runner/work/bitcoin-core-nightly/bitcoin-core-nightly/llvm_mingw_toolchain/aarch64-w64-mingw32/include/stdlib.h:221:27: note: previous declaration is here 221 | _CRTIMP char ***__cdecl __p__environ(void); | ^ /home/runner/work/bitcoin-core-nightly/bitcoin-core-nightly/llvm_mingw_toolchain/aarch64-w64-mingw32/include/stdlib.h:221:3: note: previous attribute is here 221 | _CRTIMP char ***__cdecl __p__environ(void); | ^ /home/runner/work/bitcoin-core-nightly/bitcoin-core-nightly/llvm_mingw_toolchain/aarch64-w64-mingw32/include/_mingw.h:52:40: note: expanded from macro '_CRTIMP' 52 | # define _CRTIMP __attribute__ ((__dllimport__)) | ^ 1 warning generated. ``` </details> after: https://github.com/l0rinc/bitcoin-core-nightly/actions/runs/18329616268/job/52201940831?pr=4 <details> <summary>Details</summary> ``` [ 28%] Building CXX object src/util/CMakeFiles/bitcoin_util.dir/__/randomenv.cpp.obj ``` </details> Note that there are some other remaining warnings in the logs that will be fixed in separate PRs ACKs for top commit: sipa: utACK 9610b0d if this makes the compilers happy laanwj: Code review ACK 9610b0d hebasto: re-ACK 9610b0d. Tree-SHA512: a9e39d288b663ed24cbbbae228850e6f02d417d8781a3ac3d0b3db0b7ff734bbd62fddb9f57b8f77daab4e9c016ff66906ebc5fb2de7635ef539ef7f4dc2eaba
fabe0e0 ci: Only write docker build images to Cirrus cache (MarcoFalke) fab64a5 ci: Move buildx command to python script (MarcoFalke) fa72a2b ci: Remove unused MAYBE_CPUSET (MarcoFalke) Pull request description: The `DOCKER_BUILD_CACHE_ARG` env var holds the options on how to use cache providers. Storing the image layers is useful for the Cirrus cache provider, because it offers 10GB per runner (https://cirrus-runners.app/setup/#speeding-up-the-cache). The cached image layers can help to avoid issues when the upstream package manager infra (apt native, apt llvm, pip, apk, git clone, ...) has outages or network issues. However, on the GitHub Actions cache provider, a *total* cache of 10GB is offered for the whole repo. This cache must be shared with the depends cache, and the ccache, as well as the previous releases cache. So it is already full and trying to put the docker build layers into it will lead to an overflow. Fix it by only writing to the docker cache on Cirrus. Also, `DOCKER_BUILD_CACHE_ARG` requires a `shellcheck disable=SC2086` on the full build command. Fix that as well by using `shlex.split` from Python on just this variable. ACKs for top commit: m3dwards: ACK fabe0e0 cedwies: reACK fabe0e0 l0rinc: Code review ACK fabe0e0 willcl-ark: ACK fabe0e0 Tree-SHA512: 4f471f080007fdd0c3bc97b0cfe0e9c0457e5029a7ccde1d784d30eb4752e5eb309cd4b122b182bce31f1b986c8a9f3e9a49da1768bedbb2b1f64f70183680ba
The default `-rpcthreads` value spawns 16 HTTP server threads for each node. Running the functional test suite with default `rpcthreads` can exhaust file descriptors or hit other resource limits very easily. Moreover, having 16 threads is unnecessary since they are mostly idle. We run RPC calls on a single RPC connection and wait for it result synchronously. There is (almost) never two RPC calls occurring concurrently. Because of this, the threads are mostly idle, so we can safely limit the number of them to two.
…or `libevent` fe71a4b depends: Avoid `warning: "_FORTIFY_SOURCE" redefined` for `libevent` (Hennadii Stepanov) Pull request description: On Alpine Linux 3.12.3, compiling the `libevent` package produces multiple warnings: ``` $ gmake -C depends -j $(nproc) libevent <snip> <command-line>: warning: "_FORTIFY_SOURCE" redefined <built-in>: note: this is the location of the previous definition <snip> ``` This PR fixes these warnings. ACKs for top commit: shahsb: ACK fe71a4b maflcko: lgtm ACK fe71a4b theuni: utACK fe71a4b Tree-SHA512: 0a3ffb2a4cf811bce93addac8e5394cf6b3d79a46245cbdd8488771b9b51e56f66cd9222548138041e69183d52ad4b909d3d1441593f9d79d557d6c000fb324b
e9cd45e test: set number of RPC server threads to 2 (furszy) Pull request description: The default `-rpcthreads` value spawns 16 HTTP server threads for each node. Running the functional test suite with default `rpcthreads` can exhaust file descriptors or hit other resource limits very easily (more when tests are run in parallel). Furthermore, having 16 threads is unnecessary since they are mostly idle. We run RPC calls on a single RPC connection and wait for it result synchronously. There is (almost) never two RPC calls occurring concurrently. Because of this, the threads are mostly idle, so we can safely limit the number of them to two. Note for reviewers: I checked this does not introduce any timing regression but would be good to double-check it on your end too. We could add another thread if needed. Just the 16 threads default value is too high and unnecessary. ACKs for top commit: maflcko: lgtm ACK e9cd45e l0rinc: ACK e9cd45e kevkevinpal: ACK [e9cd45e](e9cd45e) andrewtoth: ACK e9cd45e Tree-SHA512: a777286f4a890fb87f5df72cd2ccfdc628657206a4b3e995044e5a0d12987b8c78a7cf7d684cc4e92605aa782aaeebc44e9f754752c3a524152fac94fa30f4b5
8f76732 miner: fix empty mempool case for waitNext() (Sjors Provoost) Pull request description: Block template fees are calculated by looping over `new_tmpl->vTxFees` and return (early) once the `fee_threshold` is exceeded. This left an edge case when the mempool is empty, which this commit fixes and adds a test for. Also update `test/functional/interface_ipc.py` to reflect the new behavior, Fixes Sjors/sv2-tp#9 ACKs for top commit: optout21: ACK 8f76732 cedwies: tACK 8f76732 sipa: utACK 8f76732 zaidmstrr: Concept ACK [8f76732](8f76732) Tree-SHA512: ef200fe95e96f810e425283bc37f945c4bf5efa16f4b74820b8a07968f30c5146bca213a372124be84b48beead5dfd35f2b5d10d188d0a465f847ebab61de10a
faa9d10 refactor: Construct g_verify_flag_names on first use (MarcoFalke) Pull request description: The current usage of the `g_verify_flag_names` map seems fine and I can not see a static initialization order fiasco here. However, it seems brittle to hope this remains the case in the future. Also, it triggers a msan false-positive in the fuzz CI task. (C.f https://github.com/bitcoin-core/qa-assets/actions/runs/18352815555/job/52413137315?pr=241#step:7:5245) So just apply the "Construct on first use" idiom. ACKs for top commit: kevkevinpal: ACK [faa9d10](faa9d10) ajtowns: ACK faa9d10 janb84: lgtm ACK faa9d10 stickies-v: ACK faa9d10 Tree-SHA512: 6685dfc91c99a8245722e07fac99a7a6d58586c30964be7ccd74a176dfbf00c6255c8594621e2909640763924f51d3efd4ce65ed65eaeeb1d05c2fd01fe63604
d0e1bba test: repeat block malleability test with relayable block over P2P (Musa Haruna) Pull request description: This PR adds a functional test to repeat the existing malleability check for oversized coinbase witness nonce size using a block that is small enough to be relayed over the P2P network. This addresses the TODO in test_block_malleability by ensuring behavior is consistent between submitblock RPC and P2P relay. ACKs for top commit: maflcko: lgtm ACK d0e1bba janb84: re ACK d0e1bba glozow: utACK d0e1bba Tree-SHA512: 05aec4fade5af8043f40274a8d2f3cf3f540acd038138975bdefbbbc81e105792d6d2588256a2ee5ddb1e05b37fe2d0b3d287160d2dbe86e1aac7cfa9cc02116
cc5dda1 headerssync: Make HeadersSyncState more flexible and move constants (Hodlinator) 8fd1c28 test(headerssync): Test returning of pow_validated_headers behavior (Hodlinator) 7b00643 test(headerssync): headers_sync_chainwork test improvements (Hodlinator) 04eeb95 doc(test): Improve comments (Hodlinator) fe896f8 refactor(test): Store HeadersSyncState on the stack (Hodlinator) f036868 refactor(test): Break up headers_sync_state (Hodlinator) e984618 refactor(headerssync): Process spans of headers (Hodlinator) a4ac991 refactor(headerssync): Extract test constants ahead of breakup into functions (Hodlinator) Pull request description: ### Background As part of the release process we often run *contrib/devtools/headerssync-params.py* and increase the values of the constants `HEADER_COMMITMENT_PERIOD` and `REDOWNLOAD_BUFFER_SIZE` in *src/headerssync.cpp* as per *doc/release-process.md* (example: 11a2d3a). This helps fine tune the memory consumption per `HeadersSyncState`-instance in the face of malicious peers. (The `REDOWNLOAD_BUFFER_SIZE`/`HEADER_COMMITMENT_PERIOD` ratio determines how many Headers Sync commitment bits must match between PRESYNC & REDOWNLOAD phases before we start permanently storing headers from a peer. For more details see comments in *src/headerssync.h* and *contrib/devtools/headerssync-params.py*). ### Problem: Not feeding back headers until completing sync During v30 release process #33274 made `REDOWNLOAD_BUFFER_SIZE` exceed the `target_blocks` constant used to control the length of chains generated for testing Headers Sync (`15000`, *headers_sync_chainwork_tests.cpp*). The `HeadersSyncState::m_redownloaded_headers`-buffer now does not reach the `REDOWNLOAD_BUFFER_SIZE`-threshold during those unit tests. As a consequence `HeadersSyncState::PopHeadersReadyForAcceptance()` will not start feeding back headers until the PoW threshold has been met. While this will not cause the unit test to start failing on master, it means we have gone from testing behavior that resembles mainnet (way more than `REDOWNLOAD_BUFFER_SIZE` headers to reach the PoW limit), to behavior that is not possible/expected there. ### Solution Avoid testing this unrealistic condition of completing Headers Sync before reaching `REDOWNLOAD_BUFFER_SIZE` by making tests able to define their own values through the new `HeadersSyncParams` instead of having them hard-coded for all chains & tests. ### Commits * First 6 commits refactor and improve the unit tests in order to clarify latter changes. * We then add checks for the behavior around the `REDOWNLOAD_BUFFER_SIZE` threshold. * The main change: we extract the section from *headerssync.cpp* containing the constants to *kernel/chainparams.cpp*, making `HeadersSyncState` no longer hard-coded to mainnet. ### Notes This PR used to be called "headerssync: Preempt unrealistic unit test behavior". ACKs for top commit: l0rinc: reACK cc5dda1 marcofleon: code review ACK cc5dda1 danielabrozzoni: reACK cc5dda1 Tree-SHA512: ccc824dcbbb8ad5ae98c3bf5808b38467aac0230739898a758c9b939eecd74f982df088fa0ba81cc1c1732f19a607b135a6e9577bb9fcf7f8570567ce92f66e6
fa37153 util: Abort on failing CHECK_NONFATAL in debug builds (MarcoFalke) fa0dc4b test: Allow testing of check failures (MarcoFalke) faeb58f refactor: Set G_ABORT_ON_FAILED_ASSUME when G_FUZZING_BUILD (MarcoFalke) Pull request description: A failing `CHECK_NONFATAL` will throw an exception. This is fine and even desired in production builds, because the program may catch the exception and give the user a way to easily report the bug upstream. However, in debug development builds, exceptions for internal bugs are problematic: * The exception could accidentally be caught and silently ignored * The exception does not include a full stacktrace, possibly making debugging harder Fix all issues by turning the exception into an abort in debug builds. This can be tested by reverting the hunks to `src/rpc/node.cpp` and `test/functional/rpc_misc.py` and then running the functional or fuzz tests. ACKs for top commit: achow101: ACK fa37153 ryanofsky: Code review ACK fa37153, just catching subprocess.CalledProcessError in test fixing up a comment since last review stickies-v: ACK fa37153 Tree-SHA512: 2d892b838ccef6f9b25a066e7c2f6cd6f5acc94aad1d91fce62308983bd3f5c5d724897a76de4e3cc5c3678ddadc87e2ee8c87362965373526038e598dfb0101
fa70e23 ci: Drop libFuzzer from msan fuzz task (MarcoFalke) Pull request description: libFuzzer is mostly unmaintained (https://llvm.org/docs/LibFuzzer.html#status), and it isn't really needed by the CI tasks. While it provides some additional stats like rss or the max input byte size, they are not essential. Dropping libFuzzer here would also drop the "60 seconds sanity check" for empty folders, but I think this is an acceptable price to pay to silence false-positives that were hit for years. Also, there seems to be a history of intermittent false-positive msan warnings (#33600 (comment)). It is unclear what exactly is causing the false-positives, so just disable libFuzzer in this task for now, to work around them. ACKs for top commit: kevkevinpal: ACK [fa70e23](fa70e23) dergoegge: ACK fa70e23 Tree-SHA512: c3e5958b8378ba30f51d923f97a84dec2ee60af8b9c2a4f13bc8de486a490031468371120e421384aa198ffec591db554e636935ab3c6d4de5e870238f5079f2
…t check 65a10fc p2p: add assertion for BlockTransactionsRequest indexes (frankomosh) 58be359 fuzz: add a target for DifferenceFormatter Class (frankomosh) Pull request description: Adds a fuzz test for the [`DifferenceFormatter`](https://github.com/bitcoin/bitcoin/blob/e3f416dbf7633b2fb19c933e5508bd231cc7e9cf/src/blockencodings.h#L22-L42) (used in [`BlockTransactionsRequest`](https://github.com/bitcoin/bitcoin/blob/master/src/blockencodings.h#L44-L54), [BIP 152](https://github.com/bitcoin/bips/blob/master/bip-0152.mediawiki)). The DifferenceFormatter class implements differential encoding for compact block transactions (BIP 152). This PR ensures that its strictly-monotonic property is maintained. It complements the tests in [`blocktransactionsrequest_deserialize`](https://github.com/bitcoin/bitcoin/blob/9703b7e6d563ea58f83a6eca819562365404f4ab/src/test/fuzz/deserialize.cpp#L314). Additionally, there's an added invariant check after GETBLOCKTXN deserialization in `net_processing.cpp`. ACKs for top commit: Crypt-iQ: tACK 65a10fc achow101: ACK 65a10fc dergoegge: Code review ACK 65a10fc Tree-SHA512: 70659cf045e99bb5f753763c7ddac094cb2883c202c899276cbe616889afa053b2d5e831f99d6386d4d1e4118cd35fa0b14b54667853fe067f6efe2eb77b4097
b63428a rpc: refactor: use more (Maybe)Arg<std::string_view> (stickies-v) 037830c refactor: increase string_view usage (stickies-v) b3bf18f rpc: refactor: use string_view in Arg/MaybeArg (stickies-v) Pull request description: The `RPCHelpMan::{Arg,MaybeArg}` helpers avoid copying (potentially) large strings by returning them as `const std::string*` (`MaybeArg`) or `const std::string&` (`Arg`). For `MaybeArg`, this has the not-so-nice effect that users need to deal with raw pointers, potentially also requiring new functions (e.g. [`EnsureUniqueWalletName` ](d127b25#diff-d8bfcfbdd5fa7d5c52d38c1fe5eeac9ce5c5a794cdfaf683585140fa70a32374R32)) with raw pointers being implemented. This PR aims to improve on this by returning a trivially copyable `std::string_view` (`Arg`) or `std::optional<std::string_view>` (`MaybeArg`), modernizing the interface without introducing any additional copying overhead. In doing so, it also generalizes whether we return by value or by pointer/reference using `std::is_trivially_copyable_v` instead of defining the types manually. In cases where functions currently take a `const std::string&` and it would be too much work / touching consensus logic to update them (`signmessage.cpp`), a `std::string` copy is made (which was already happening anyway). The last 2 commits increase usage of the `{Arg,MaybeArg}<std::string_view>` helpers, and could be dropped/pruned if anything turns out to be controversial - I just think it's a nice little cleanup. ACKs for top commit: maflcko: re-ACK b63428a 🎉 achow101: ACK b63428a pablomartin4btc: re-ACK [b63428a](b63428a) w0xlt: reACK b63428a Tree-SHA512: b4942c353a1658c22a88d8c9b402c288ad35265a3b88aa2072b1f9b6d921cd073194ed4b00b807cb48ca440f47c87ef3d8e0dd1a5d814be58fc7743f26288277
45bd891 log: split assumevalid ancestry-failure-reason message (Lőrinc) 6c13a38 log: separate script verification reasons (Lőrinc) f2ea6f0 refactor: untangle assumevalid decision branches (Lőrinc) 9bc2985 validation: log initial script verification state (Lőrinc) 4fad4e9 test: add assumevalid scenarios scaffold (Lőrinc) 91ac64b log: reword `signature validations` to `script verification` in `assumevalid` log (Lőrinc) Pull request description: ### Summary Users can encounter cases where script checks are unexpectedly enabled (e.g. after reindex, or when `assumevalid`/`minimumchainwork` gates fail). Without an explicit line, they must infer state from the absence of a message, which is incomplete and error-prone. The existing "Assuming ancestors of block …" line does not reliably indicate whether script checks are actually enabled, which makes debugging/benchmarking confusing. ### What this changes We make the initial **script-verification** state explicit and log **why** checks are enabled to avoid confusion. * Always log the first script-verification state on startup, **before** the first `UpdateTip`. * Flatten the nested `assumevalid` conditionals into a linear gating sequence for readability. * Extend the functional test to assert the old behavior with the new reason strings. This is a **logging-only** test change it shouldn't change any other behavior. ### Example output The state (with reason) is logged at startup and whenever the reason changes, e.g.: * `Disabling script verification at block #904336 (000000000000000000014106b2082b1a18aaf3091e8b337c6fed110db8c56620).` * `Enabling script verification at block #912527 (000000000000000000010bb6aa3ecabd7d41738463b6c6621776c2e40dbe738a): block too recent relative to best header.` * `Enabling script verification at block #912684 (00000000000000000001375cf7b90b2b86e559d05ed92ca764d376702ead3858): block height above assumevalid height.` ------ Follow-up to #32975 (comment) ACKs for top commit: Eunovo: re-ACK 45bd891 achow101: ACK 45bd891 hodlinator: re-ACK 45bd891 yuvicc: ACK 45bd891 andrewtoth: ACK 45bd891 ajtowns: ACK 45bd891 Tree-SHA512: 58328d7c418a6fe18f1c7fe1dd31955bb6fce8b928b0df693f6200807932eb5933146300af886a80a1d922228d93faf531145186dae55ad4ad1f691970732eca
5ded99a fuzz: MockMempoolMinFee in wallet_fees (brunoerg) c9a7a19 test: move MockMempoolMinFee to util/txmempool (brunoerg) adf67eb fuzz: create FeeEstimatorTestingSetup to set fee_estimator (brunoerg) ff10a37 fuzz: mock CBlockPolicyEstimator in wallet_fuzz (brunoerg) f591c3b fees: make estimateSmartFee/HighestTargetTracked virtual for mocking (brunoerg) 19273d0 fuzz: set mempool options in wallet_fees (brunoerg) Pull request description: Some functions in `wallet/fees.cpp` (fuzzed by the wallet_fees target) depends on some mempool stuff - e.g. relay current min fee, smart fee and max blocks estimation, relay dust fee and other ones. For better fuzzing of it, it would be great to have these values/interactions. That said, this PR enhances the `wallet_fees` target by: - Setting mempool options - `min_relay_feerate`, `dust_relay_feerate` and `incremental_relay_feerate` - when creating the `CTxMemPool`. - Creates a `ConsumeMempoolMinFee` function which is used to have a mempool min fee (similar approach from `MockMempoolMinFee` from unit test). - Mock `CBlockPolicyEstimator` - estimateSmartFee/HighestTagretTracket functions, especifically. It's better to mock it then trying to interact to CBlockPolicyEstimator in order to have some effective values due to performance. Note that I created `FeeEstimatorTestingSetup` because we cannot set `m_node.fee_estimator` in `ChainTestingSetup` since fae8c73. ACKs for top commit: maflcko: re-ACK 5ded99a 🎯 ismaelsadeeq: Code review ACK 5ded99a Tree-SHA512: 13d2af042098afd237ef349437021ea841069d93d4c3e3a32e1b562c027d00c727f375426709d34421092993398caf7ba8ff19077982cb6f470f8938a44e7754
…nt formatter versions 13f36c0 clang-format: regenerate configs (Lőrinc) Pull request description: Updates `.clang-format` file to reflect [latest supported Clang-Format standards](https://releases.llvm.org/16.0.0/tools/clang/docs/ClangFormatStyleOptions.html) while preserving most of the existing formatting behavior. Note that [`AfterStruct` brace placement](#32414 (comment)) was originally aligned here with `AfterClass`, but was reverted by reviewer demand. ACKs for top commit: maflcko: re-ACK 13f36c0 🖼 achow101: ACK 13f36c0 hodlinator: re-ACK 13f36c0 Tree-SHA512: 02bd9d8a22a9af268297aeddd1f2b2cce079fddd0e1f764d6e9650bb614cb7bcfbd20b38d6e4e5db1744b3dd1ba540380010c085f2cbc0be8aa936f21d27d8de
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
35 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.