This week’s newsletter announces an upcoming change to the Bitcoin-Dev mailing list and briefly summarizes a proposal to allow aggregating multiple HTLCs together. Also included are our regular sections with the summary of a Bitcoin Core PR Review Club, announcements of new releases and release candidates, and descriptions of notable changes to popular Bitcoin infrastructure software.
● Mailing list hosting: administrators for the Bitcoin-Dev mailing list announced that the organization hosting the list plans to cease hosting any mailing lists after the end of the year. The archives of previous emails are expected to continue being hosted at their current URLs for the foreseeable future. We assume that the end of email relay also affects the Lightning-Dev mailing list, which is hosted by the same organization.
The administrators sought feedback from the community about options, including migrating the mailing list to Google Groups. If such a migration happens, Optech will begin using that as one of our news sources.
We’re also aware that, in the months prior to the announcement, some well-established developers had begun experimenting with discussions on the DelvingBitcoin web forum. Optech will begin monitoring that forum for interesting or important discussions effective immediately.
● HTLC aggregation with covenants: Johan Torås Halseth posted to the Lightning-Dev mailing list a suggestion for using a covenant to aggregate multiple HTLCs into a single output that could be spent all at once if a party knew all the preimages. If a party only knew some of the preimages, they could claim just those and then the remaining balance could be refunded to the other party. Halseth notes that this would be more efficient onchain and could make it more difficult to perform certain types of channel jamming attacks.
Bitcoin Core PR Review Club
In this monthly section, we summarize a recent Bitcoin Core PR Review Club meeting, highlighting some of the important questions and answers. Click on a question below to see a summary of the answer from the meeting.
Fee Estimator updates from Validation Interface/CScheduler thread is a PR by Abubakar Sadiq Ismail (ismaelsadeeq) that modifies the way the transaction fee estimator data is updated. (Fee estimation is used when the node’s owner initiates a transaction.) It moves fee estimator updates from occurring synchronously during mempool updates (transactions being added or removed) to instead occur asynchronously. While this adds more processing complexity overall, it improves critical-path performance (which the following discussion will make evident).
When a new block is found, its transactions that are in the mempool are removed along with any transactions that conflict with the block’s transactions. Since block processing and relay are performance-critical, it’s beneficial to reduce the required amount of work during the processing of a new block, such as updating the fee estimator.
Why is it beneficial to remove
CTxMempool’s dependency on
Currently, upon receiving a new block, its processing is blocked while the fee estimator is updated. This delays the completion of new block processing, and also delays relaying the block to peers. Removing
CTxMempool’s dependency on
CBlockPolicyEstimatorallows fee estimates to be updated asynchronously (in a different thread) so that validation and relay can complete more quickly. It may also make testing
CTxMempooleasier. Finally, it allows the future use of more complex fee estimation algorithms without affecting block validation and relay performance. ➚
Isn’t fee estimation currently updated synchronously when transactions are added to or removed from the mempool even without a new block arriving?
Yes, but performance isn’t as critical at those times as during block validation and relay. ➚
Are there any benefits of the
CBlockPolicyEstimatorbeing a member of
CTxMempooland updating it synchronously (the current arrangement)? Are there downsides to removing it?
Synchronous code is simpler and easier to reason about. Also, the fee estimator has more visibility into the entire mempool; a downside is the need to encapsulate all the information needed for fee estimation into a new
NewMempoolTransactionInfostructure. However, not much information is needed by the fee estimator. ➚
What do you think are the advantages and disadvantages of the approach taken in this PR, compared with the one taken in PR 11775 that splits
While it seems nice to split them, they still had a shared backend (to keep the events well-ordered), so weren’t really very independent of each other. There doesn’t seem to be much practical benefit to splitting. The current PR is more narrow, and minimally scoped to making fee estimates update asynchronously. ➚
In a subclass, why is implementing a
CValidationInterfacemethod equivalent to subscribing to the event?
All subclasses of
CValidationInterfaceare clients. The subclass can implement some or all
CValidationInterfacemethods (callbacks), for example, connecting or disconnecting a block, or a transaction added to or removed from the mempool. After being registered (by calling
RegisterSharedValidationInterface()), any implemented
CValidationInterfacemethod will be executed each time the method callback is fired using
CMainSignals. Callbacks are fired whenever the corresponding event occurs. ➚
In commit 4986edb, why are we adding a new callback,
MempoolTransactionsRemovedForConnectedBlock, instead of using
BlockConnected(which also indicates a transaction being removed from the mempool)?
The fee estimator needs to know when transactions are removed from the mempool for any reason, not just when a block is connected. Also, the fee estimator needs a transaction’s base fee, and this isn’t provided via
BlockConnected(which provides a
CBlock). We could add the base fee to the
block.vtx(transaction list) entries, but it’s undesirable to change such an important and ubiquitous data structure just to support fee estimation. ➚
Why don’t we use a
std::vector<CTxMempoolEntry>as a parameter of
MempoolTransactionsRemovedForBlockcallback? This would eliminate the requirement for a new struct type to hold the per-transaction information needed for fee estimation.
The fee estimator doesn’t need all of the fields from
How is the base fee of a
It’s the sum of the input values minus the sum of the output values. However, the callback can’t access the input values because they’re stored in the previous transaction outputs (which the callback doesn’t have access to). That’s why the base fee is included in the
Releases and release candidates
New releases and release candidates for popular Bitcoin infrastructure projects. Please consider upgrading to new releases or helping to test release candidates.
● Bitcoin Core 26.0rc2 is a release candidate for the next major version of the predominant full node implementation. There’s a brief overview to suggested testing topics and a scheduled meeting of the Bitcoin Core PR Review Club dedicated to testing on 15 November 2023.
Notable code and documentation changes
Notable recent changes in Bitcoin Core, Core Lightning, Eclair, LDK, LND, libsecp256k1, Hardware Wallet Interface (HWI), Rust Bitcoin, BTCPay Server, BDK, Bitcoin Improvement Proposals (BIPs), Lightning BOLTs, and Bitcoin Inquisition.
● Core Lightning #6824 updates the implementation of the interactive funding protocol to “store state when sending
commitment_signed, and [add] a
channel_reestablishto ask our peer to retransmit signatures that we haven’t received.” It is based on an update to the proposed dual funding PR.
● Eclair #2761 allows forwarding a limited number of HTLCs to a party even if they’re below their channel reserve requirement. This can help resolve a stuck funds problem that might occur after splicing or dual funding. See Newsletter #253 for another mitigation by Eclair for a stuck funds problem.