This week’s newsletter describes research into zero-knowledge validity proofs for Bitcoin and related protocols. Also included is another entry in our limited weekly series about mempool policy, plus our regular sections describing updates to clients and services, new releases and release candidates, and changes to popular Bitcoin infrastructure projects.

News

  • ● State compression with zero-knowledge validity proofs: Robin Linus posted to the Bitcoin-Dev mailing list about a paper he co-authored with Lukas George about using validity proofs to reduce the amount of state that a client needs to download in order to trustlessly verify future operations in a system. They first apply their system to Bitcoin. They report having a prototype that proves the cumulative amount of proof of work in a chain of block headers and allows a client to verify that a particular block header is part of that chain. This allows a client that receives multiple proofs to determine which demonstrates the most proof of work.

    They also have a suboptimal prototype which proves that all transaction state changes for the block chain respected the currency rules (e.g., how many bitcoins can be created by a new block, that each non-coinbase transaction must not create UTXOs with more bitcoin value than those it destroys (spends), and that a miner may claim any difference between the UTXOs destroyed in a block and those created). A client receiving this proof and a copy of the current UTXO set would be able to verify that the set was accurate and complete. They call this an assumevalid proof after the feature in Bitcoin Core that optionally skips verification of scripts for older blocks when many contributors agree that their nodes all validated those blocks successfully.

    To minimize the complexity of their proof, they use a version of utreexo with a hash function optimized for their system. They separately suggest that combining their proof with a utreexo client will allow that client to begin operating as a full node almost immediately after downloading a very small amount of data.

    Regarding the usability of their prototypes, they write that “we have implemented the header chain proof and the assumevalid state proof as prototypes. The former is feasible to prove, while the latter still requires performance improvements to prove reasonable-sized blocks”. They are also working on validating complete blocks, including scripts, but say that they need at least a 40x speed up for that to be viable.

    In addition to state compression of the Bitcoin block chain, they also describe a protocol that can be used for a client-side-validation token protocol similar to that used for Lightning Labs’ Taproot Assets and some uses of RGB (see Newsletters #195 and #247). When Alice transfers to Bob some amount of a token, Bob needs to verify the history of every previous transfer of those specific tokens back to when they were created. In an ideal scenario, that history grows linearly with the number of transfers. But if Bob wants to pay Carol an amount higher than he received from Alice, he will need to combine some of the tokens he received from Alice with some tokens he received in a different transaction. Carol will then need to verify both the history going back through Alice and also the history of Bob’s other tokens. This is called a merge. If merges happen often, the size of the history that needs to be verified approaches the size of the history of every transfer of that token between any users. In comparative terms, in Bitcoin every full node verifies every transaction made by every user; in token protocols using client-side validation, that’s not strictly required but eventually becomes effectively the case if merges are common.

    That means a protocol that can compress the state of Bitcoin can also be adapted to compress the state of a token’s history, even one where merges are common. The authors describe how they would accomplish that. Their goal is to produce a proof that each previous transfer of the token followed the token’s rules, including using their proofs for Bitcoin to prove that each previous transfer was anchored in the block chain. Alice could then transfer the tokens to Bob and give him a short constant-sized proof of validity; Bob could verify the proof to know that the transfer happened at a certain block height and paid his token wallet, giving him exclusive control over the tokens.

    Although the paper frequently mentions additional research and development that can be performed, we find this to be encouraging progress towards a feature that Bitcoin developers have desired for over a decade.

Waiting for confirmation #2: Incentives

A limited weekly series about transaction relay, mempool inclusion, and mining transaction selection—including why Bitcoin Core has a more restrictive policy than allowed by consensus and how wallets can use that policy most effectively.

Last week’s post discussed mempool as a cache of unconfirmed transactions that provides a decentralized method for users to send transactions to miners. However, miners are not obligated to confirm those transactions; a block with just a coinbase transaction is valid by consensus rules. Users can incentivize miners to include their transactions by increasing total input value without changing total output value, allowing miners to claim the difference as transaction fees.

Although transaction fees are common in traditional financial systems, new Bitcoin users are often surprised to find that on-chain fees are paid not in proportion to the transaction amount but by the weight of the transaction. Block space, instead of liquidity, is the limiting factor. Feerate is typically denominated in satoshis per virtual byte.

Consensus rules limit the space used by transactions in each block. This limit keeps block propagation times low relative to the block interval, reducing the risk of stale blocks. It also helps restrict the growth of the block chain and UTXO set, both of which directly contribute to the cost of bootstrapping and maintaining a full node.

As such, as part of their role as a cache of unconfirmed transactions, mempools also facilitate a public auction for inelastic block space: when functioning properly, the auction operates on free-market principles, i.e., priority is based purely on fees rather than relationships with miners.

Maximizing fees when selecting transactions for a block (which has limits on total weight and signature operations) is an NP-hard problem. This problem is further complicated by transaction relationships: mining a high-feerate transaction may require mining its low-feerate parent. Put another way, mining a low-feerate transaction may open up the opportunity to mine its high-feerate child.

The Bitcoin Core mempool computes the feerate for each entry and its ancestors (called ancestor feerate), caches that result, and uses a greedy block template building algorithm. It sorts the mempool in ancestor score order (the minimum of ancestor feerate and individual feerate) and selects ancestor packages in that order, updating the remaining transactions’ ancestor fee and weight information as it goes. This algorithm offers a balance between performance and profitability, and does not necessarily produce optimal results. Its efficacy can be boosted by restricting the size of transactions and ancestor packages—Bitcoin Core sets those limits to 400,000 and 404,000 weight units, respectively.

Similarly, a descendant score is calculated that is used when selecting packages to evict from the mempool, as it would be disadvantageous to evict a low-feerate transaction that has a high-feerate child.

Mempool validation also uses fees and feerate when dealing with transactions that spend the same input(s), i.e. double-spends or conflicting transactions. Instead of always keeping the first transaction it sees, nodes use a set of rules to determine which transaction is the more incentive compatible to keep. This behavior is known as Replace by Fee.

It is intuitive that miners would maximize fees, but why should a non-mining node operator implement these policies? As mentioned in last week’s post, the utility of a non-mining node’s mempool is directly related to its similarity to miners’ mempools. As such, even if a node operator never intends to produce a block using the contents of its mempool, they have an interest in keeping the most incentive-compatible transactions.

While there are no consensus rules requiring transactions to pay fees, fees and feerate play an important role in the Bitcoin network as a “fair” way to resolve competition for block space. Miners use feerate to assess acceptance, eviction, and conflicts, while non-mining nodes mirror those behaviors in order to maximize the utility of their mempools.

The scarcity of block space exerts a downward pressure on the size of transactions and encourages developers to be more efficient in transaction construction. Next week’s post will explore practical strategies and techniques for minimizing fees in on-chain transactions.

Changes to services and client software

In this monthly feature, we highlight interesting updates to Bitcoin wallets and services.

  • ● Passport firmware 2.1.1 released: The latest firmware for the Passport hardware signing device supports sending to taproot addresses, BIP85 features, and improvements to handling PSBTs and multisig configurations.

  • ● MuSig wallet Munstr released: The beta Munstr software uses the nostr protocol in order to facilitate the rounds of communication required for signing MuSig multisignature transactions.

  • ● CLN plugin manager Coffee released: Coffee is a CLN plugin manager that improves aspects of installation, configuration, dependency management, and upgrading of CLN plugins.

  • ● Electrum 4.4.3 released: The latest Electrum versions contain coin control improvements, a UTXO privacy analysis tool, and support for Short Channel Identifiers (SCIDs), among other fixes and improvements.

  • ● Trezor Suite adds coinjoin support: The Trezor Suite software announced support for coinjoins that use the zkSNACKs coinjoin coordinator.

  • ● Lightning Loop defaults to MuSig2: Lightning Loop now uses MuSig2 as the default swap protocol resulting in lower fees and better privacy.

  • ● Mutinynet announces new signet for testing: Mutinynet is a custom signet with 30-second block times that provides testing infrastructure including a block explorer, faucet, as well as test LN nodes and LSPs running on the network.

  • ● Nunchuk adds coin control, BIP329 support: The latest Android and iOS versions of Nunchuk add coin control and BIP329 wallet label export features.

  • ● MyCitadel Wallet adds enhanced miniscript support: The v1.3.0 release adds more complicated miniscript capabilities including timelocks.

  • ● Edge Firmware for Coldcard announced: Coinkite announced experimental firmware for the Coldcard hardware signer that is designed for wallet developers and power users to experiment with newer features. The initial 6.0.0X release includes taproot keypath spends, tapscript multisig payments, and BIP129 support.

Releases and release candidates

New releases and release candidates for popular Bitcoin infrastructure projects. Please consider upgrading to new releases or helping to test release candidates.

Notable code and documentation changes

Notable changes this week in Bitcoin Core, Core Lightning, Eclair, LDK, LND, libsecp256k1, Hardware Wallet Interface (HWI), Rust Bitcoin, BTCPay Server, BDK, Bitcoin Improvement Proposals (BIPs), Lightning BOLTs, and Bitcoin Inquisition.

  • ● Bitcoin Core #27021 adds an interface for calculating how much it would cost to bring an output’s unconfirmed ancestor transactions up to a given feerate, a value known as their fee deficit. When coin selection is considering using a particular output at a particular feerate, its ancestors’ fee deficit for that feerate is calculated and the result is deducted from its effective value. That discourages the wallet from choosing highly deficient outputs for a new transaction when other spendable outputs are available. In a follow-up PR, the interface will also be used to allow the wallet to pay the extra fees (called bump fees) if it has to select deficient outputs anyway, ensuring the new transaction pays the effective feerate the user requested.

    The algorithm is capable of assessing bump fees for any ancestor constellation by evaluating the unconfirmed UTXO’s entire related cluster of unconfirmed transactions and pruning the transactions that will have been picked into a block at the target feerate. A second method provides an aggregate bump fee across multiple unconfirmed outputs to correct for potential overlapping ancestries.

  • ● LND #7668 adds the ability to associate up to 500 characters of private text with a channel when opening it and allows the operator to retrieve that information later, which may help them recall why they opened that particular channel.

  • ● LDK #2204 adds the ability to set custom feature bits for announcing to peers, or for use when attempting to parse a peer’s announcement.

  • ● LDK #1841 implements a version of a security recommendation previously added to the LN specification (see Newsletter #128) where a node using anchor outputs should not attempt to batch together inputs controlled by multiple parties when the transaction needs to be confirmed promptly. This prevents other parties from being able to delay confirmation.

  • ● BIPs #1412 updates BIP329 for wallet label export with a field to store key origin information. Additionally, the specification now suggests a label length limit of 255 characters.