This week’s newsletter summarizes mailing list discussion about using the MATT proposal to manage joinpools and replicate functions of the OP_CHECKTEMPLATEVERIFY proposal. Also included is another entry in our limited weekly series about mempool policy, plus our regular sections for announcing new software releases and release candidates and describing notable changes to popular Bitcoin infrastructure software.

News

• ● Using MATT to replicate CTV and manage joinpools: Johan Torås Halseth posted to the Bitcoin-Dev mailing list about using the OP_CHECKOUTPUTCONTRACTVERIFY opcode (COCV) from the Merklize All The Things (MATT) proposal (see Newsletters #226 and #249) to replicate the functionality of the OP_CHECKTEMPLATEVERIFY proposal. For committing a transaction with multiple outputs, each output would require using a different COCV opcode. By comparison, a single CTV opcode could commit to all the outputs. That makes COCV less efficient but, as he notes, “simple enough to be interesting”.

  Beyond just describing the functionality, Halseth also provides a demo of the operation using Tapsim, a tool for “debugging Bitcoin Tapscript transactions […] aimed at developers wanting to play with Bitcoin script primitives, aid in script debugging, and visualize the VM state as scripts are executed.”

  In a separate thread, Halseth posted about using MATT plus OP_CAT to create a joinpool (also called a coinpool or a payment pool). Again, he provides an interactive demo using Tapsim. He also provided several suggested modifications to the opcodes in the MATT proposal based on the results of his experimental implementation. Salvatore Ingala, the originator of the MATT proposal, replied favorably.

Waiting for confirmation #4: Feerate estimation

A limited weekly series about transaction relay, mempool inclusion, and mining transaction selection—including why Bitcoin Core has a more restrictive policy than allowed by consensus and how wallets can use that policy most effectively.

Last week, we explored techniques for minimizing the fees paid on a transaction given a feerate. But what should that feerate be? Ideally, as low as possible to save money, but high enough to secure a spot in a block suitable for the user’s time preference.

The goal of fee(rate) estimation is to translate a target timeframe for confirmation to a minimal feerate the transaction should pay.

One complication of fee estimation is the irregularity of block space production. Let’s say a user needs to pay a merchant within one hour to receive their goods. The user may expect a block to be mined every 10 minutes, and thus aim for a spot within the next 6 blocks. However, it’s entirely possible for one block to take 45 minutes to be found. Fee estimators must translate between a user’s desired urgency or timeframe (something like “I want this to confirm by the end of the work day”) and a supply of block space (a number of blocks). Many fee estimators address this challenge by denominating confirmation targets in the number of blocks in addition to time.

With no information about transactions prior to their confirmation, one can build a naive fee estimator that uses historical data about what transaction feerates tend to land in blocks. As this estimator is blind to the transactions awaiting confirmation in mempools, it would become very inaccurate during unexpected fluctuations in block space demand and the occasional long block interval. Its other weakness is its reliance on information controlled wholly by miners, who would be able to drive feerates up by including fake high-feerate transactions in their blocks.

Fortunately, the market for block space is not a blind auction. We mentioned in our first post that keeping a mempool and participating in the peer-to-peer transaction relay network allows a node to see what users are bidding. The Bitcoin Core fee estimator also uses historical data to calculate the likelihood of a transaction at feerate f confirming within n blocks, but specifically tracks the height at which the node first sees a transaction and when it confirms. This method works around activity that happens outside the public fee market by ignoring it. If miners include artificially high-feerate transactions in their own blocks, this fee estimator isn’t skewed because it only uses data from transactions that were publicly relayed prior to confirmation.

We also have insights into the way transactions are selected for blocks. In a previous post, we mentioned that nodes emulate miner policies in order to keep incentive-compatible transactions in their mempools. Expanding on this idea, instead of looking only at past data, we could build a fee estimator that simulates what a miner would do. To find out what feerate a transaction would need to confirm in the next n blocks, the fee estimator could use the block assembly algorithm to project the next n block templates from its mempool and calculate the feerate that would beat the last transaction(s) that make it into block n.

Clearly, the efficacy of this fee estimator’s approach depends on the similarity between the contents of its mempool and the miners’, which can never be guaranteed. It is also blind to transactions a miner might include due to exterior motivations, e.g. transactions that belong to the miner or paid out-of-band fees to be confirmed. The projection must also account for additional transaction broadcasts between now and when the projected blocks are found. It can do so by decreasing the size of its projected blocks to account for other transactions – but by how much?

This question once again highlights the utility of historical data. An intelligent model may be able to incorporate patterns of activity and account for external events that influence feerates such as typical business hours, a company’s scheduled UTXO consolidation, and activity in response to changes in Bitcoin’s trading price. The problem of forecasting block space demand remains ripe for exploration, and is likely to always have room for innovation.

Fee estimation is a multi-faceted and difficult problem. Bad fee estimation can waste funds by overpaying fees, add friction to the use of Bitcoin for payments, and cause L2 users to lose money by missing a window within which a timelocked UTXO had an alternate spending path. Good fee estimation allows users to clearly and precisely communicate transaction urgency to miners, and CPFP and RBF allow users to update their bids if initial estimates undershoot. Incentive-compatible mempool policies, combined with well-designed fee estimation tools and wallets, enable users to participate in an efficient, public auction for block space.

Fee estimators typically never return anything below 1sat/vB, regardless of how long the time horizon is or how few transactions are pending confirmation. Many consider 1sat/vB as the de facto floor feerate in the Bitcoin network, due to the fact that most nodes on the network (including miners) never accept anything below that feerate, regardless of how empty their mempools are. Next week’s post will explore this node policy and another motivation for utilizing feerate in transaction relay: protection from resource exhaustion.

Releases and release candidates

New releases and release candidates for popular Bitcoin infrastructure projects. Please consider upgrading to new releases or helping to test release candidates.

• ● LND 0.16.3-beta is a maintenance release for this popular LN node implementation. Its release notes say, “this release contains only bug fixes and is intended to optimize the recently added mempool watching logic, and also fix several suspected inadvertent force close vectors”. For more information about the mempool-watching logic, see Newsletter #248.

Notable code and documentation changes

Notable changes this week in Bitcoin Core, Core Lightning, Eclair, LDK, LND, libsecp256k1, Hardware Wallet Interface (HWI), Rust Bitcoin, BTCPay Server, BDK, Bitcoin Improvement Proposals (BIPs), Lightning BOLTs, and Bitcoin Inquisition.

• ● Bitcoin Core #26485 allows RPC methods that accept an options object parameter to accept the same fields as named parameters. For example, the bumpfee RPC can now be called with src/bitcoin-cli -named bumpfee txid fee_rate=10 instead of src/bitcoin-cli -named bumpfee txid options='{"fee_rate": 10}'.

• ● Eclair #2642 adds a closedchannels RPC that provides data about the node’s closed channels. See also a similar PR from Core Lightning mentioned in Newsletter #245.

• ● LND #7645 makes sure that any user-provided fee rate in RPC calls to OpenChannel, CloseChannel, SendCoins, and SendMany is no less than a ‘relay fee rate’. The change notes that “‘Relay fee rate’ may mean slightly different things depending on the backend. For bitcoind, it is effectively max(relay fee, min mempool fee)”.

• ● LND #7726 will always spend all HTLCs paying the local node if a channel needs to be settled onchain. It will sweep those HTLCs even if it might cost more in transaction fees to sweep them than they are worth. Compare this to a PR from Eclair described in last week’s newsletter where that program now won’t attempt to claim an HTLC that is uneconomical. Comments in the PR thread mention that LND is working toward other changes that will enhance its ability to calculate the costs and gains related to settling HTLCs (both offchain and onchain), allowing it to make optimal decisions in the future.

• ● LDK #2293 disconnects and then reconnects to peers if they haven’t responded within a reasonable amount of time. This may mitigate a problem with other LN software that sometimes stops responding, leading to channels being forced closed.