This week’s newsletter describes a proposal to remove details from the LN specification that are no longer relevant to modern nodes and includes the penultimate entry in our limited weekly series about mempool policy, plus our regular sections summarizing a Bitcoin Core PR Review Club meeting, announcing new releases and release candidates, and describing notable changes to popular Bitcoin infrastructure software.

News

  • ● LN specification clean up proposed: Rusty Russell posted to the Lightning-Dev mailing list a link to a PR where he proposes to remove some features that are no longer supported by modern LN implementations and to assume other features will always be supported. Related to the proposed changes, Russell also provides the results of a survey he ran of public node features according to their gossip messages. His results imply that nearly all nodes support the following features:

    • ● Variable-sized onion messages: made part of the specification in 2019 (see Newsletter #58) around the same time as the specification was updated to use Type-Length-Value (TLV) fields. This replaced the original format for encrypted onion routing that required each hop to use a fixed-length message and limited the number of hops to 20. The variable-sized format makes it much easier to relay arbitrary data to specific hops, with the only downside being that the overall message size remains constant, so any increase in the amount of data sent decreases the maximum number of hops.

    • ● Gossip queries: made part of the specification in 2018 (see BOLTs #392). This allows a node to request from its peers only a subset of gossip messages sent by other nodes on the network. For example, a node may request only recent gossip updates, ignoring older updates to save bandwidth and reduce processing time.

    • ● Data loss protection: made part of the specification in 2017 (see BOLTs #240). Nodes using this feature send information about their latest channel state when they reconnect. This may allow a node to detect that it has lost data, and it encourages a node that has not lost data to close the channel in its latest state. See Newsletter #31 for additional details.

    • ● Static remote-party keys: made part of the specification in 2019 (see Newsletter #67), this allows a node to request that every channel update commit to sending the node’s non-HTLC funds to the same address. Previously, a different address was used in every channel update. After this change, a node that opted into this protocol and lost data would almost always eventually receive at least some of their funds to their chosen address, such as an address in their HD wallet.

    Initial replies to the clean-up proposal PR were positive.

Waiting for confirmation #9: Policy Proposals

A limited weekly series about transaction relay, mempool inclusion, and mining transaction selection—including why Bitcoin Core has a more restrictive policy than allowed by consensus and how wallets can use that policy most effectively.

Last week’s post described anchor outputs and CPFP carve out, ensuring either channel party can fee-bump their shared commitment transactions without requiring collaboration. This approach still contains a few drawbacks: channel funds are tied up to create anchor outputs, commitment transaction feerates typically overpay to ensure they meet mempool minimum feerates, and CPFP carve out only allows one extra descendant. Anchor outputs cannot ensure the same ability to fee-bump for transactions shared between more than two parties, such as coinjoins or multi-party contracting protocols. This post explores current efforts to address these and other limitations.

Package relay includes P2P protocol and policy changes to enable the transport and validation of groups of transactions. It would allow a commitment transaction to be fee-bumped by a child even if the commitment transaction does not meet a mempool’s minimum feerate. Additionally, Package RBF would allow the fee-bumping child to pay for replacing transactions its parent conflicts with. Package relay is designed to remove a general limitation at the base protocol layer. However, due to its utility in fee-bumping of shared transactions, it has also spawned a number of efforts to eliminate pinning for specific use cases. For example, Package RBF would allow commitment transactions to replace each other when broadcast with their respective fee-bumping children, removing the need for multiple anchor outputs on each commitment transaction.

A caveat is that existing RBF rules require the replacement transaction to pay a higher absolute fee than the aggregate fees paid by all to-be-replaced transactions. This rule helps prevent DoS through repeated replacements, but allows a malicious user to increase the cost to replace their transaction by attaching a child that is high fee but low feerate. This hinders the transaction from being mined by unfairly preventing its replacement by a high-feerate package, and is often referred to as “Rule 3 pinning.”

Developers have also proposed entirely different ways of adding fees to presigned transactions. For example, signing inputs of the transaction using SIGHASH_ANYONECANPAY | SIGHASH_ALL could allow the transaction broadcaster to provide fees by appending additional inputs to the transaction without changing the outputs. However, as RBF does not have any rule requiring the replacement transaction to have a higher “mining score” (i.e. would be selected for a block faster), an attacker could pin these types of transactions by creating replacements encumbered by low-feerate ancestors. What complicates the accurate assessment of the mining score of transactions and transaction packages is that the existing ancestor and descendant limits are insufficient to bound the computational complexity of this calculation. Any connected transactions can influence the order in which transactions get picked into a block. A fully-connected component, called a cluster, can be of any size given current ancestor and descendant limits.

A long term solution to address some mempool deficiencies and RBF pinning attacks is to restructure the mempool data structure to track clusters instead of just ancestor and descendant sets. These clusters would be limited in size. A cluster limit would restrict the way users can spend unconfirmed UTXOs, but make it feasible to quickly linearize the entire mempool using the ancestor score-based mining algorithm, build block templates extremely quickly, and add a requirement that replacement transactions have a higher mining score than the to-be-replaced transaction(s).

Even so, it’s possible that no single set of policies can meet the wide range of needs and expectations for transaction relay. For example, while recipients of a batched payment transaction benefit from being able to spend their unconfirmed outputs, a relaxed descendant limit leaves room for pinning package RBF of a shared transaction through absolute fees. A proposal for v3 transaction relay policy was developed to allow contracting protocols to opt in to a more restrictive set of package limits. V3 transactions would only permit packages of size two (one parent and one child) and limit the weight of the child. These limits would mitigate RBF pinning through absolute fees, and offer some of the benefits of cluster mempool without requiring a mempool restructure.

Ephemeral Anchors builds upon the properties of v3 transactions and package relay to improve anchor outputs further. It exempts anchor outputs belonging to a zero-fee v3 transaction from the dust limit, provided the anchor output is spent by a fee-bumping child. Since the zero-fee transaction must be fee-bumped by exactly one child (otherwise a miner would not be incentivized to include it in a block), this anchor output is “ephemeral” and would not become a part of the UTXO set. The ephemeral anchor proposal implicitly prevents non-anchor outputs from being spent while unconfirmed without 1 OP_CSV timelocks, since the only allowed child must spend the anchor output. It would also make LN symmetry feasible with CPFP as the fee provisioning mechanism for channel closing transactions. It also makes this mechanism available for transactions shared between more than two participants. Developers have been using bitcoin-inquisition to deploy Ephemeral Anchors and proposed soft forks to build and test these multi-layer changes on a signet.

The pinning problems highlighted in this post, among others, spawned a wealth of discussions and proposals to improve RBF policy last year across mailing lists, pull requests, social media, and in-person meetings. Developers proposed and implemented solutions ranging from small amendments to a complete revamp. The -mempoolfullrbf option, intended to address pinning concerns and a discrepancy in BIP125 implementations, illuminated the difficulty and importance of collaboration in transaction relay policy. While a genuine effort had been made to engage the community using typical means, including starting the bitcoin-dev mailing list conversation a year in advance, it was clear that the existing communication and decision-making methods had not produced the intended result and needed refinement.

Decentralized decision-making is a challenging process, but necessary to support the diverse ecosystem of protocols and applications that use Bitcoin’s transaction relay network. Next week will be our final post in this series, in which we hope to encourage our readers to participate in and improve upon this process.

Bitcoin Core PR Review Club

In this monthly section, we summarize a recent Bitcoin Core PR Review Club meeting, highlighting some of the important questions and answers. Click on a question below to see a summary of the answer from the meeting.

Stop relaying non-mempool txs is a PR by Marco Falke (MarcoFalke) that simplifies the Bitcoin Core client by removing an in-memory data structure, mapRelay, that may cause high memory consumption and is no longer needed, or at least is of very marginal benefit. This map contains transactions that may or may not also be in the mempool, and is sometimes used to reply to getdata requests from peers.

  • What are the reasons to remove mapRelay?

    The memory consumption of this data structure is unbounded. Though typically it doesn’t use much memory, it’s concerning when the size of any data structure is determined by the behavior of outside entities (peers) and has no maximum, as this can create a DoS vulnerability. ➚

  • Why is the memory usage of mapRelay hard to determine?

    Each entry in mapRelay is a shared pointer to a transaction (CTransaction), with the mempool possibly holding another pointer. A second pointer to the same object uses very little additional space relative to a single pointer. If a shared transaction is removed from the mempool, all of its space becomes attributable to the mapRelay entry. So mapRelay’s memory usage doesn’t depend only on the number of transactions and their individual sizes, but also on how many of its transactions are no longer in the mempool, which is hard to predict. ➚

  • What problem is solved by introducing m_most_recent_block_txs? (This is a list of only the transactions in the most recently-received block.)

    Without it, since mapRelay is no longer available, we wouldn’t be able to serve just-mined transactions (in the most recent block) to peers requesting them, since we will have dropped them from our mempool. ➚

  • Do you think it is necessary to introduce m_most_recent_block_txs, as opposed to just removing mapRelay without any replacement?

    There was some uncertainty among review club attendees on this question. It was suggested that m_most_recent_block_txs might improve block propagation speed, because if our peer doesn’t yet have the block that we just received, our node’s ability to provide its transactions may help complete our peer’s compact block. Another suggestion was that it may help in the event of a chain split; if our peer is on a different tip than us, it may not have that transaction via a block. ➚

  • What are the memory requirements for m_most_recent_block_txs compared to mapRelay?

    The number of entries in m_most_recent_block_txs is bounded by the number of transactions in a block. But the memory requirement is even less than that many transactions, because m_most_recent_block_txs entries are shared pointers (to transactions), and they are also (already) pointed to by m_most_recent_block. ➚

  • Are there scenarios in which transactions would be made available for a shorter or longer time than before as a result of this change?

    Longer when the time since the last block is greater than 15 minutes (which is the time that entries remained in mapRelay), shorter otherwise. This seems acceptable since the 15-minute time choice was rather arbitrary. But the change may decrease the availability of transactions in case of chain splits greater than one block deep (which are extremely rare) because we’re not retaining transactions that are unique to non-best chains. ➚

Releases and release candidates

New releases and release candidates for popular Bitcoin infrastructure projects. Please consider upgrading to new releases or helping to test release candidates.

  • ● LND v0.16.4-beta is a maintenance release of this LN node software that fixes a memory leak that may affect some users.

Notable code and documentation changes

Notable changes this week in Bitcoin Core, Core Lightning, Eclair, LDK, LND, libsecp256k1, Hardware Wallet Interface (HWI), Rust Bitcoin, BTCPay Server, BDK, Bitcoin Improvement Proposals (BIPs), Lightning BOLTs, and Bitcoin Inquisition.