This week’s newsletter describes a proposal for speeding up Bitcoin Core initial block download, with a proof-of-concept implementation that shows a roughly 5x speed up compared to Bitcoin Core’s defaults. Also included are our regular sections summarizing a Bitcoin Core PR Review Club meeting, announcing new releases and release candidates, and describing notable changes to popular Bitcoin infrastructure projects.

News

  • ● SwiftSync speedup for initial block download: Sebastian Falbesoner posted to Delving Bitcoin a sample implementation and performance results for SwiftSync, an idea proposed by Ruben Somsen during a recent Bitcoin Core developers meeting and later posted to the mailing list. As of writing, the most recent results posted in the thread show a 5.28x speedup of initial block download (IBD) over Bitcoin Core’s default IBD settings (which uses assumevalid but not assumeUTXO), reducing the initial sync time from about 41 hours to about 8 hours.

    Before using SwiftSync, someone who has already synced their node to a recent block creates a hints file that indicates which transaction outputs will be in the UTXO set at that block (i.e., which outputs will be unspent). This can be efficiently encoded to a few hundred megabytes for the current UTXO set size. The hints file also indicates what block it was generated at, which we’ll call the terminal SwiftSync block.

    The user performing SwiftSync downloads the hints file and uses it when processing each block prior to the terminal SwiftSync block to only store outputs in the UTXO database if the hints file indicates that the output will remain in the UTXO set when the terminal SwiftSync block is reached. This massively reduces the number of entries that are added, and then later removed, from the UTXO database during IBD.

    To ensure that the hints file is correct, every created output not stored in the UTXO database is added to a cryptographic accumulator. Every spent output is removed from the accumulator. When the node reaches the terminal SwiftSync block, it ensures the accumulator is empty, meaning every output seen was later spent. If that fails, it means that the hints file was incorrect and that IBD needs to be performed again from scratch without using SwiftSync. In this way, users do not need to trust the creator of the hints file—a malicious file cannot result in an incorrect UTXO state; it can only waste a few hours of the user’s computing resources.

    An additional property of SwiftSync that has not yet been implemented is allowing parallel validation of blocks during IBD. This is possible because assumevalid does not check the scripts of older blocks, entries are never removed from UTXO database prior to the terminal SwiftSync block, and the accumulator used only tracks the net effect of outputs being added (created) and removed (spent). That eliminates any dependencies between blocks before the terminal SwiftSync block. Parallel validation during IBD is also a feature of Utreexo for some of the same reasons.

    The discussion examined several aspects of the proposal. Falbesoner’s original implementation used the MuHash accumulator (see Newsletter #123), which has been shown to be resistant to the generalized birthday attack. Somsen described an alternative approach that may be faster. Falbesoner questioned whether the alternative approach was cryptographically secure but, since it was simple, implemented it anyway and found it further sped up SwiftSync.

    James O’Beirne asked whether SwiftSync is useful given that assumeUTXO provides an even larger speedup. Somsen replied that SwiftSync speeds up assumeUTXO’s background validation, making it a nice addition for users of assumeUTXO. He further notes that anyone who downloads assumeUTXO’s required data (the UTXO database at a particular block) doesn’t need a separate hints file if they use that same block as the terminal SwiftSync block.

    Vojtěch Strnad, 0xB10C, and Somsen discussed compressing the hint file data, with an expected savings of about 75%, bringing the test hints file (for block 850,900) down to about 88 MB.

    Discussion was ongoing at the time of writing.

Bitcoin Core PR Review Club

In this monthly section, we summarize a recent Bitcoin Core PR Review Club meeting, highlighting some of the important questions and answers. Click on a question below to see a summary of the answer from the meeting.

Add Fee rate Forecaster Manager is a PR by ismaelsadeeq that upgrades the transaction fee forecasting (also called estimation) logic. It introduces a new ForecasterManager class to which multiple Forecasters can be registered. The existing CBlockPolicyEstimator (which only considers confirmed transactions) is refactored to become one such forecaster, but notably a new MemPoolForecaster is introduced. MemPoolForecaster considers unconfirmed transactions that are in the mempool, and as such can react to feerate changes more quickly.

  • Why is the new system called a “Forecaster” and “ForecasterManager” rather than an “Estimator” and “Fee Estimation Manager”?

    The system predicts future outcomes based on current and past data. Unlike an estimator, which approximates present conditions with some randomization, a forecaster projects future events, which aligns with this system’s predictive nature and its output of uncertainty/risk levels. ➚

  • Why is CBlockPolicyEstimator not modified to hold the mempool reference, similar to the approach in PR #12966? What is the current approach and why is it better than holding a reference to mempool? (Hint: see PR #28368)

    CBlockPolicyEstimator inherits from CValidationInterface and implements its virtual methods TransactionAddedToMempool, TransactionRemovedFromMempool, and MempoolTransactionsRemovedForBlock. This gives CBlockPolicyEstimator all the mempool information it needs without being unnecessarily tightly coupled to the mempool via a reference. ➚

  • What are the trade-offs between the new architecture and a direct modification of CBlockPolicyEstimator?

    The new architecture with a FeeRateForecasterManager class to which multiple Forecasters can be registered is a more modular approach which allows for better testing, and enforces a better separation of concerns. It allows easily plugging in new forecasting strategies later on. This comes at the cost of a bit more code to maintain, and potentially confusing users about which estimation method to use. ➚

Releases and release candidates

New releases and release candidates for popular Bitcoin infrastructure projects. Please consider upgrading to new releases or helping to test release candidates.

  • ● Core Lightning 25.02.1 is a maintenance release for the current major version of this popular LN node that includes several bug fixes.

  • ● Core Lightning 24.11.2 is a maintenance release for a prior major version of this popular LN node. It includes several bug fixes, some of them the same bug fixes that were released in version 25.02.1.

  • ● BTCPay Server 2.1.0 is a major release of this self-hosted payment processing software. It includes breaking changes for users of some altcoins, improvements for RBF and CPFP fee bumping, and a better flow for multisig when all signers are using BTCPay Server.

  • ● Bitcoin Core 29.0rc3 is a release candidate for the next major version of the network’s predominate full node. Please see the version 29 testing guide.

  • ● LND 0.19.0-beta.rc2 is a release candidate for this popular LN node. One of the major improvements that could probably use testing is the new RBF-based fee bumping for cooperative closes.

Notable code and documentation changes

Notable recent changes in Bitcoin Core, Core Lightning, Eclair, LDK, LND, libsecp256k1, Hardware Wallet Interface (HWI), Rust Bitcoin, BTCPay Server, BDK, Bitcoin Improvement Proposals (BIPs), Lightning BOLTs, Lightning BLIPs, Bitcoin Inquisition, and BINANAs.

  • ● LDK #2256 and LDK #3709 improve attributable failures (see Newsletter #224) as specified in BOLTs #1044 by adding an optional attribution_data field to the UpdateFailHTLC struct and introducing the AttributionData struct. In this protocol, each forwarding node appends to the failure message a hop_payload flag, a duration field that records how long the node held the HTLC, and HMACs corresponding to different assumed positions in the route. If a node corrupts the failure message, the mismatch in the HMAC chain helps identify the pair of nodes between which this happened.

  • ● LND #9669 downgrades simple taproot channels to always use the legacy cooperative close flow, even if the RBF cooperative close flow (see Newsletter #347) is configured. Previously, a node that had both features configured would fail to start up.

  • ● Rust Bitcoin #4302 adds a new push_relative_lock_time() method to the script builder API, which takes a relative timelock parameter, and deprecates push_sequence() which takes a raw sequence number as a parameter. This change resolves a potential confusion where developers would mistakenly push a raw sequence number in scripts instead of a relative timelock value, which is then checked against an input’s sequence number using CHECKSEQUENCEVERIFY.

Want more?

For more discussion about the topics mentioned in this newsletter, join us for the weekly Bitcoin Optech Recap on Riverside.fm at 15:30 UTC on April 15. The discussion is also recorded and will be available from our podcasts page.