This week’s newsletter announces an upcoming change to the Bitcoin-Dev mailing list and briefly summarizes a proposal to allow aggregating multiple HTLCs together. Also included are our regular sections with the summary of a Bitcoin Core PR Review Club, announcements of new releases and release candidates, and descriptions of notable changes to popular Bitcoin infrastructure software.

News

  • ● Mailing list hosting: administrators for the Bitcoin-Dev mailing list announced that the organization hosting the list plans to cease hosting any mailing lists after the end of the year. The archives of previous emails are expected to continue being hosted at their current URLs for the foreseeable future. We assume that the end of email relay also affects the Lightning-Dev mailing list, which is hosted by the same organization.

    The administrators sought feedback from the community about options, including migrating the mailing list to Google Groups. If such a migration happens, Optech will begin using that as one of our news sources.

    We’re also aware that, in the months prior to the announcement, some well-established developers had begun experimenting with discussions on the DelvingBitcoin web forum. Optech will begin monitoring that forum for interesting or important discussions effective immediately.

  • ● HTLC aggregation with covenants: Johan Torås Halseth posted to the Lightning-Dev mailing list a suggestion for using a covenant to aggregate multiple HTLCs into a single output that could be spent all at once if a party knew all the preimages. If a party only knew some of the preimages, they could claim just those and then the remaining balance could be refunded to the other party. Halseth notes that this would be more efficient onchain and could make it more difficult to perform certain types of channel jamming attacks.

Bitcoin Core PR Review Club

In this monthly section, we summarize a recent Bitcoin Core PR Review Club meeting, highlighting some of the important questions and answers. Click on a question below to see a summary of the answer from the meeting.

Fee Estimator updates from Validation Interface/CScheduler thread is a PR by Abubakar Sadiq Ismail (ismaelsadeeq) that modifies the way the transaction fee estimator data is updated. (Fee estimation is used when the node’s owner initiates a transaction.) It moves fee estimator updates from occurring synchronously during mempool updates (transactions being added or removed) to instead occur asynchronously. While this adds more processing complexity overall, it improves critical-path performance (which the following discussion will make evident).

When a new block is found, its transactions that are in the mempool are removed along with any transactions that conflict with the block’s transactions. Since block processing and relay are performance-critical, it’s beneficial to reduce the required amount of work during the processing of a new block, such as updating the fee estimator.

  • Why is it beneficial to remove CTxMempool’s dependency on CBlockPolicyEstimator?

    Currently, upon receiving a new block, its processing is blocked while the fee estimator is updated. This delays the completion of new block processing, and also delays relaying the block to peers. Removing CTxMempool’s dependency on CBlockPolicyEstimator allows fee estimates to be updated asynchronously (in a different thread) so that validation and relay can complete more quickly. It may also make testing CTxMempool easier. Finally, it allows the future use of more complex fee estimation algorithms without affecting block validation and relay performance. ➚

  • Isn’t fee estimation currently updated synchronously when transactions are added to or removed from the mempool even without a new block arriving?

    Yes, but performance isn’t as critical at those times as during block validation and relay. ➚

  • Are there any benefits of the CBlockPolicyEstimator being a member of CTxMempool and updating it synchronously (the current arrangement)? Are there downsides to removing it?

    Synchronous code is simpler and easier to reason about. Also, the fee estimator has more visibility into the entire mempool; a downside is the need to encapsulate all the information needed for fee estimation into a new NewMempoolTransactionInfo structure. However, not much information is needed by the fee estimator. ➚

  • What do you think are the advantages and disadvantages of the approach taken in this PR, compared with the one taken in PR 11775 that splits CValidationInterface?

    While it seems nice to split them, they still had a shared backend (to keep the events well-ordered), so weren’t really very independent of each other. There doesn’t seem to be much practical benefit to splitting. The current PR is more narrow, and minimally scoped to making fee estimates update asynchronously. ➚

  • In a subclass, why is implementing a CValidationInterface method equivalent to subscribing to the event?

    All subclasses of CValidationInterface are clients. The subclass can implement some or all CValidationInterface methods (callbacks), for example, connecting or disconnecting a block, or a transaction added to or removed from the mempool. After being registered (by calling RegisterSharedValidationInterface()), any implemented CValidationInterface method will be executed each time the method callback is fired using CMainSignals. Callbacks are fired whenever the corresponding event occurs. ➚

  • BlockConnected and NewPoWValidBlock are different callbacks. Which one is asynchronous and which one is synchronous? How can you tell?

    BlockConnected is asynchronous; NewPoWValidBlock is synchronous. Asynchronous callbacks queue an “event” to be run later within the CScheduler thread. ➚

  • In commit 4986edb, why are we adding a new callback, MempoolTransactionsRemovedForConnectedBlock, instead of using BlockConnected (which also indicates a transaction being removed from the mempool)?

    The fee estimator needs to know when transactions are removed from the mempool for any reason, not just when a block is connected. Also, the fee estimator needs a transaction’s base fee, and this isn’t provided via BlockConnected (which provides a CBlock). We could add the base fee to the block.vtx (transaction list) entries, but it’s undesirable to change such an important and ubiquitous data structure just to support fee estimation. ➚

  • Why don’t we use a std::vector<CTxMempoolEntry> as a parameter of MempoolTransactionsRemovedForBlock callback? This would eliminate the requirement for a new struct type to hold the per-transaction information needed for fee estimation.

    The fee estimator doesn’t need all of the fields from CTxMempoolEntry. ➚

  • How is the base fee of a CTransactionRef computed?

    It’s the sum of the input values minus the sum of the output values. However, the callback can’t access the input values because they’re stored in the previous transaction outputs (which the callback doesn’t have access to). That’s why the base fee is included in the TransactionInfo structure. ➚

Releases and release candidates

New releases and release candidates for popular Bitcoin infrastructure projects. Please consider upgrading to new releases or helping to test release candidates.

Notable code and documentation changes

Notable recent changes in Bitcoin Core, Core Lightning, Eclair, LDK, LND, libsecp256k1, Hardware Wallet Interface (HWI), Rust Bitcoin, BTCPay Server, BDK, Bitcoin Improvement Proposals (BIPs), Lightning BOLTs, and Bitcoin Inquisition.