This special edition of the Optech Newsletter summarizes notable developments in Bitcoin during all of 2023. It’s the sequel to our summaries from 2018, 2019, 2020, 2021, and 2022.

Contents

January

  • ● Swap-in-potentiam: ZmnSCPxj and Jesse Posner proposed swap-in-potentiam, a non-interactive method for opening Lightning Network channels, addressing challenges faced by often-offline wallets like those on mobile devices. A client can receive funds in an onchain transaction while it is offline. The transaction can receive enough confirmations to make it immediately safe to open a channel with a pre-selected peer when the client comes back online—without placing any trust in that peer. Within a few months of the proposal, at least one popular LN wallet was using an implementation of the idea.
  • ● BIP329 wallet label export format: A standard format for wallet label export and import was assigned the identifier BIP329. Not only does the standard make it easier to back up important wallet data that can’t be recovered from a BIP32 seed, but it also makes it much easier to copy transaction metadata into non-wallet programs, such as accounting software. By the end of the year, several wallets had implemented BIP329 export.

February

  • ● Ordinals and inscriptions: February saw the beginning of a discussion that would continue for the rest of the year about Ordinals and Inscriptions, two related protocols for associating meaning and data with transaction outputs. Andrew Poelstra summarized the position of many protocol developers: “there is no sensible way to prevent people from storing arbitrary data in witnesses without incentivizing even worse behavior and/or breaking legitimate use cases.” Given the method used by Inscriptions allows storing large amounts of data, Christopher Allen suggested increasing Bitcoin Core’s 83-byte limit for data storage in outputs prefixed by OP_RETURN; a proposal that was also advanced by Peter Todd later in the year.
  • ● Bitcoin Search, ChatBTC, and TL;DR: BitcoinSearch.xyz was launched early in the year, providing a search engine for Bitcoin technical documentation and discussions. By the end of the year, the site offered a chat interface and summaries of recent discussions.
  • ● Peer storage backups: Core Lightning added experimental support for peer storage backups, which allow a node to store a small encrypted backup file for its peers. If a peer later needs to reconnect, perhaps after losing data, it can request the backup file. The peer can use a key derived from its wallet seed to decrypt the file and use the contents to recover the latest state of all of its channels.

  • ● LN quality of service: Joost Jager proposed a “high availability” flag for LN channels, allowing the channel to signal it offered reliable payment forwarding. Christian Decker noted challenges in creating reputation systems, such as infrequent node encounters. A previously proposed alternative approach was also mentioned: overpayment with recovery (previously termed boomerang or refundable overpayments), where payments are split and sent via multiple routes, reducing reliance on highly available channels.

  • ● HTLC endorsement: Ideas from a paper posted last year became a particular focus of the 2023 effort to mitigate LN channel jamming. In February, Carla Kirk-Cohen and the paper’s co-author Clara Shikhelman began asking for feedback on the suggested parameters to use when implementing one of the ideas from the paper, HTLC endorsement. In April, they posted a draft specification for their testing plans. In July, the idea and the proposal were discussed at the LN development meeting, which led to some mailing list discussion about an alternative approach to make the costs paid by both attackers and honest users reflect the underlying costs paid by the node operators providing the service; that way, a node operator who is earning a reasonable return on providing services to honest users will continue earning reasonable returns if attackers begin using those services. In August, it was announced developers associated with Eclair, Core Lightning, and LND were all implementing parts of the HTLC endorsement protocol in order to begin collecting data related to it.

  • ● Codex32: Russell O’Connor and Andrew Poelstra proposed a new BIP for backing up and restoring BIP32 seeds called codex32. Similar to SLIP39, it allows creating several shares using Shamir’s Secret Sharing Scheme with configurable threshold requirements. An attacker who obtains less than the threshold number of shares will learn nothing about the seed. Unlike other recovery codes that use a wordlist, codex32 uses the same alphabet as bech32 addresses. The main advantage of codex32 over existing schemes is its ability to perform all operations manually using pen, paper, instructions, and paper cutouts, including generating an encoded seed (with dice), protecting it with a checksum, creating checksummed shares, verifying checksums, and recovering the seed. This allows users to periodically verify the integrity of individual shares without relying on trusted computing devices.

March

  • ● Hierarchical channels: In March, pseudonymous developer John Law published a paper describing a way to create a hierarchy of channels for multiple users from a single onchain transaction. The design can allow all online users to spend their funds even when some of their channel counterparties are offline, which is not currently possible in LN. This optimization would allow always-online users to use their capital more efficiently, likely reducing the costs for other users of LN. The proposal depends on Law’s tunable penalty protocol, which has not seen any public software development since it was proposed in 2022.

Tunable Penalty Protocol

Summary 2023: Soft fork proposals

A proposal for a new OP_VAULT opcode was posted in January by James O’Beirne, followed in February by a draft BIP an implementation for Bitcoin Inquisition. This was further followed a few weeks later by a proposal for an alternative design for OP_VAULT by Gregory Sanders.

The merklize all the things (MATT) proposal first described last year saw activity again this year. Salvatore Ingala showed how it could provide most of the features of the proposed OP_VAULT opcodes. Johan Torås Halseth further demonstrated how one of the opcodes from the MATT proposal could replicate key functions of the OP_CHECKTEMPLATEVERIFY proposed opcode, although the MATT version was less space efficient. Halseth also used the opportunity to introduce readers to a tool he’d built, tapsim, which allows debugging Bitcoin transactions and tapscripts.

In June, Robin Linus described how users could timelock funds today, use them on a sidechain for a long time, and then allow the receivers of the funds on the sidechain to effectively withdraw them on Bitcoin at a later point—but only if Bitcoin users eventually decide to change the consensus rules in a certain way. This could allow users willing to take a financial risk to immediately begin using their funds with new consensus features they desire, while providing a path for those funds to later return to Bitcoin’s mainnet.

In August, Brandon Black proposed a version of OP_TXHASH combined with OP_CHECKSIGFROMSTACK that would provide most of the features of OP_CHECKTEMPLATEVERIFY (CTV) and SIGHASH_ANYPREVOUT (APO) without much additional onchain cost over those individual proposals.

In September, John Law suggested enhancing LN scalability using covenants. He uses a construction similar to channel factories and the proposed Ark protocol to potentially fund millions of channels offchain, which can be reclaimed by the factory funder after an expiry, with users withdrawing their funds over LN beforehand. The model allows funds to be moved between factories without user interaction, reducing the risk of last-minute onchain congestion and transaction fees. Anthony Towns raised concerns about the forced expiration flood problem, where the failure of a large user could force many time-sensitive onchain transactions simultaneously. Law replied to note that he is working on a solution to delay expiry during high transaction fee periods.

October started with Steven Roose posting a draft BIP for a new OP_TXHASH opcode. The idea for the opcode has been discussed before but this is the first specification of the idea. In addition to describing exactly how the opcode would work, it examined some downsides, such as full nodes potentially needing to hash up to several megabytes of data every time the opcode is invoked. The BIP draft included a sample implementation of the opcode.

Also in October, Rusty Russell researched generic covenants with minimal changes to Bitcoin’s scripting language and Ethan Heilman published a draft BIP to add an OP_CAT opcode that would concatenate two elements on the stack. The discussion for both of those topics would continue into November.

Before the end of the year, Johan Torås Halseth would also suggest that covenant-style soft forks could allow aggregation of multiple HTLCs into a single output that could be spent all at once if a party knew all the preimages. If a party only knew some of the preimages, they could claim just those and then the remaining balance could be refunded to the other party. This would be more efficient onchain and could make it more difficult to perform certain types of channel jamming attacks.

April

  • ● Watchtower accountability proofs: Sergi Delgado Segura proposed an accountability mechanism for watchtowers for cases where they fail to respond to protocol breaches that they were capable of detecting. For example, Alice provides a watchtower with data for detecting and responding to the confirmation of an old LN channel state; later, that state is confirmed but the watchtower fails to respond. Alice would like to be able to hold the watchtower operator accountable by publicly proving it failed to respond appropriately. Delgado suggests a mechanism based on cryptographic accumulators that watchtowers can use for creating commitments, which users can later use to produce accountability proofs if a breach occurs.

  • ● Route blinding: Route blinding, first described three years earlier, was added to the LN specification in April. It allows a receiver to provide a spender with the identifier of a particular forwarding node and an onion-encrypted path from that node to the receiver’s own node. The spender forwards a payment and the encrypted path information to the selected forwarding node; the forwarding node decrypts information for the next hop; that next hop decrypts the hop after it; and so on, until the payment arrives at the receiver’s node with neither the spender nor any of the forwarding nodes ever learning (for sure) which node belonged to the receiver. It significantly improves the privacy of receiving money using LN.

  • ● MuSig2: BIP327 was assigned to the MuSig2 protocol for creating scriptless multisignatures in April. This protocol would be implemented in multiple programs and systems over the year, including LND’s signrpc RPC, Lightning Lab’s Loop service, BitGo’s multisignature service, LND’s experimental simple taproot channels, and a draft BIP for extending PSBTs.

  • ● RGB and Taproot Assets: Maxim Orlovsky announced the release of RGB v0.10 in April, a new version of this protocol for allowing the creation and transfer of tokens (among other things) using contracts that are defined and validated offchain. Changes to the contract state (e.g. transfers) are associated with onchain transactions in a way that uses no additional block space over a typical transaction and which can keep all information about the contract state (including its existence) completely private from third parties. Later in the year, the Taproot Assets protocol, which is partly derived from RGB, released specifications intended to become BIPs.
  • ● Channel splicing: April also saw significant discussion about the proposed protocol for splicing, which allows nodes to continue using a channel while funds are added to it or removed from it. This is especially useful for keeping funds in a channel while still allowing instant onchain payments to be made from that balance, allowing a wallet user interface to show users a single balance from which they can make either onchain or offchain payments. By the end of the year, both Core Lightning and Eclair would support splicing.

Splicing

May

  • ● LSP specifications: A set of draft specifications for Lightning Service Providers (LSPs) was released in May. Standards make it easy for a client to connect to multiple LSPs, which will prevent vendor lock-in and improve privacy. The first specification released describes an API for allowing a client to purchase a channel from an LSP, achieving a functionality similar to liquidity advertisements. The second describes an API for setting up and managing just-in-time (JIT) channels.

  • ● Payjoin: Dan Gould spent a significant part of the year working on enhancements to the payjoin protocol, a privacy-enhancing technique that makes it much more difficult for a third party to reliably associate inputs and outputs in a transaction with either the spender or receiver. In February, he proposed a serverless payjoin protocol that can be used even if the receiver doesn’t operate an always-on HTTPS server on a public network interface. In May, he discussed several advanced applications using payjoin, including variations of payment cut-through; for example, rather than Alice paying Bob, Alice instead pays Bob’s vendor (Carol), reducing a debt he owes her (or pre-paying for an expected future bill)—this saves block space and further improves privacy over standard payjoin. In August, he posted a draft BIP for serverless payjoin which doesn’t require the spender and receiver to be online at the same time (although each of them will need to come online at least once after the transaction is initiated before it can be broadcast). Throughout the year, he was a top contributor to the payjoin development kit (PDK) as well as the payjoin-cli project that provides an add-on for creating payjoins with Bitcoin Core.

  • ● Ark: Burak Keceli proposed a new joinpool-style protocol called Ark, where Bitcoin owners can opt-in to use a counterparty as a co-signer on all transactions within a certain time period. Owners can either withdraw their bitcoins onchain after the expiry of the timelock or instantly and trustlessly transfer them offchain to the counterparty before the timelock expires. The protocol provides a trustless single-hop, single-direction atomic transfer protocol from the owner to the counterparty for various uses such as mixing coins, making internal transfers, and paying LN invoices. Concerns were raised about the high onchain footprint and the need for the operator to keep a large amount of funds in a hot wallet compared to LN. However, several developers remained enthusiastic about the proposed protocol and its potential to provide a simple and trustless experience for its users.

June

  • ● Silent payments: Josie Baker and Ruben Somsen posted a draft BIP for silent payments, a type of reusable payment code that will produce a unique onchain address each time it is used, preventing output linking. Output linking can significantly reduce the privacy of users (including users not directly involved in a transaction). The draft goes into detail about the benefits of the proposal, its tradeoffs, and how software can effectively use it. Ongoing work implementing silent payments for Bitcoin Core was also discussed during a Bitcoin Core PR Review Club meeting.

Summary 2023: Security disclosures

Optech reported on three significant security vulnerabilities this year:

  • ● Milk Sad vulnerability in Libbitcoin bx: a widely undocumented lack of entropy in a command suggested for creating wallets ultimately led to the theft of a significant number of bitcoins across multiple wallets.

  • ● Fake funding denial-of-service against LN nodes: a denial-of-service attack was privately discovered and responsibly disclosed reported by Matt Morehouse. All affected nodes were able to update and, as of this writing, we are unaware of the vulnerability being exploited.

  • ● Replacement cycling against HTLCs: a fund-stealing attack against HTLCs used in LN and possibly other protocols was privately discovered and responsibly disclosed by Antoine Riard. All LN implementations tracked by Optech deployed mitigations, although the effectiveness of those mitigations was a subject of discussion and other mitigations have been proposed.

July

  • ● Validating Lightning Signer: The Validating Lightning Signer (VLS) project released their first beta version in July. The project allows the separation of an LN node from the keys that control its funds. An LN node running with VLS will route signing requests to a remote signing device instead of local keys. The beta release supports CLN and LDK, layer-1 and layer-2 validation rules, backup and recovery capabilities, and provides a reference implementation.

August

  • ● Onion messages: Support for onion messages was added to the LN specification in August. Onion messages allow sending one-way messages across the network. Like payments (HTLCs), the messages use onion encryption so that each forwarding node only knows what peer it received the message from and what peer should next receive the message. Message payloads are also encrypted so that only the ultimate receiver can read it. Onion messages use blinded paths, which were added to the LN specification in April, and onion messages are themselves used by the proposed offers protocol.
  • ● Outdated backup proofs: Thomas Voegtlin proposed a protocol that would allow penalizing providers for offering outdated backup states to users. This service involves a simple mechanism where a user, Alice, backs up data with a version number and signature to Bob. Bob adds a nonce and commits to the complete data with a time-stamped signature. If Bob provides outdated data, Alice can generate a fraud proof showing Bob previously signed a higher version number. This mechanism isn’t Bitcoin-specific, but incorporating certain Bitcoin opcodes could enable its use onchain. In a Lightning Network (LN) channel, that would allow Alice to claim all channel funds if Bob provided an outdated backup, thus reducing the risk of Bob deceiving Alice and stealing her balance. The proposal sparked significant discussion. Peter Todd pointed out its versatility beyond LN and suggested a simpler mechanism without fraud proofs, while Ghost43 highlighted the importance of such proofs when dealing with anonymous peers.
  • ● Simple taproot channels: LND added experimental support for “simple taproot channels”, allowing LN funding and commitment transactions to use P2TR with support for MuSig2-style scriptless multisignature signing when both parties are cooperating. This reduces transaction weight and improves privacy when channels are closed cooperatively. LND continues to exclusively use HTLCs, allowing payments starting in a taproot channel to continue to be forwarded through other LN nodes that don’t support taproot channels.

September

  • ● Compressed Bitcoin transactions: In September, Tom Briar published a draft specification and implementation of compressed Bitcoin transactions. The proposal addresses the challenge of compressing uniformly distributed data in Bitcoin transactions by replacing integer representations with variable-length integers, using block height and location to reference transactions instead of their outpoint txid, and omitting public keys from P2WPKH transactions. While the compressed format saves space, converting it back into a usable format requires more CPU, memory, and I/O than processing regular serialized transactions, which is an acceptable tradeoff in situations such as broadcast by satellite or steganographic transfer.

October

  • ● Payment switching and splitting: Gijs van Dam posted research results and code about payment splitting and switching (PSS). His code allows nodes to split incoming payments into multiple parts, which can take different routes before reaching the final recipient. For instance, a payment from Alice to Bob could be partly routed through Carol. This technique significantly hinders balance discovery attacks, where attackers probe channel balances to track payments across the network. Van Dam’s research showed a 62% decrease in information gained by attackers using PSS. Additionally, PSS offers increased Lightning Network throughput and may help mitigate channel jamming attacks.

  • ● Sidepools: Developer ZmnSCPxj proposed a concept called sidepools that aims to enhance LN’s liquidity management. Sidepools involve multiple forwarding nodes depositing funds into a multiparty offchain state contract similar to LN channels. This allows funds to be redistributed among participants offchain. For instance, if Alice, Bob, and Carol each start with 1 BTC, the state can be updated so that Alice has 2 BTC, Bob 0 BTC, and Carol 1 BTC. Participants would still use and advertise regular LN channels, and if those channels became imbalanced, a rebalancing can be done through an offchain peerswap within the state contract. This method is private to participants, requires less onchain space, and likely eliminates offchain rebalancing fees, thus improving revenue potential for forwarding nodes and reliability for LN payments. However, it requires a multiparty state contract, which is untested in production. ZmnSCPxj suggests building on LN-Symmetry or duplex payment channels, both of which have advantages and disadvantages.

  • ● AssumeUTXO: October saw the completion of the first phase of the assumeUTXO project, containing all the remaining changes necessary to both use an assumedvalid snapshot chainstate and do a full validation sync in the background. It makes UTXO snapshots loadable via an RPC. Although the feature set is not yet directly usable by inexperienced users, this merge marked the culmination of a multi-year effort. The project, proposed in 2018 and formalized in 2019, will significantly improve the user experience of new full nodes first coming onto the network.

  • ● Version 2 P2P transport: Also accomplished by the Bitcoin Core project in October was the addition of support for version 2 encrypted P2P transport as specified in BIP324. The feature is currently disabled by default but can be enabled using the -v2transport option. Encrypted transport helps improve the privacy of Bitcoin users by preventing passive observers (such as ISPs) from directly determining which transactions nodes relay to their peers. It’s also possible to use encrypted transport to detect active man-in-the-middle observers by comparing session identifiers. In the future, the addition of other features may make it convenient for a lightweight client to securely connect to a trusted node over a P2P encrypted connection.

  • ● Miniscript: Miniscript descriptor support saw several additional improvements in Bitcoin Core throughout the year. February saw the ability to create miniscript descriptors for P2WSH output scripts. October saw miniscript support updated to support taproot, including miniscript descriptors for tapscript.

  • ● State compression and BitVM: A method for state compression in Bitcoin using zero-knowledge validity proofs was described by Robin Linus and Lukas George in May. This massively reduces the amount of state that a client needs to download in order to trustlessly verify future operations in a system, for example starting a new full node with only a relatively small validity proof rather than validating every already confirmed transaction on the block chain. In October, Robin Linus introduced BitVM, a method enabling bitcoins to be paid contingent on the successful execution of an arbitrary program, without requiring a consensus change in Bitcoin. BitVM requires substantial offchain data exchange but needs only a single onchain transaction for agreement, or a small number of onchain transactions if there’s a dispute. BitVM makes complex trustless contracts possible even in adversarial scenarios, which has caught the attention of several developers.

November

  • ● Offers: With the final specification of blinded paths and onion messages, and their implementation in multiple popular LN nodes, a significant amount of progress was made through this year in the development of the offers protocol that depends on them. Offers allow a receiver’s wallet to generate a short offer that can be shared with a spender’s wallet. The spender’s wallet can use the offer to contact the receiver’s wallet over the LN protocol to request a specific invoice, which it can then pay in the usual way. This allows the creation of reusable offers that can each produce a different invoice, invoices that can be updated with current information (e.g. the exchange rate) just seconds before they’re paid, and offers that can be paid by the same wallet more than once (e.g. a subscription), among other features. Existing experimental implementations of offers in Core Lightning and Eclair were updated during the year, and support for offers was added to LDK. Additionally, November saw a discussion about creating an updated version of Lightning Addresses that is compatible with offers.
  • ● Liquidity advertisements: November also saw an update to the specification for liquidity advertisements that allow a node to announce that it’s willing to contribute some of its funds to a new dual-funded channel in exchange for a fee, allowing the requesting node to quickly begin receiving incoming LN payments. The updates were mostly minor, although there was discussion that continued into December about whether channels created from a liquidity advertisement should contain a time lock. A time lock could give an incentive-based assurance to the buyer that they would actually receive the liquidity they paid for, but the time lock could also be used by a malicious or inconsiderate buyer to lock up an excess amount of the provider’s capital.

Summary 2023: Bitcoin Optech

In Optech’s sixth year, we published 51 weekly newsletters, published a 10-part series about mempool policy, and added 15 new pages to our topics index. Altogether, Optech published over 86,000 English words about Bitcoin software research and development this year, the rough equivalent of a 250-page book.

In addition, every newsletter this year was accompanied by a podcast episode, totaling over 50 hours in audio form and 450,000 words in transcript form. Many of Bitcoin’s top contributors were guests on the show—some of them on more than one episode—with a total of 62 different unique guests in 2023:

  • Abubakar Ismail
  • Adam Gibson
  • Alekos Filini
  • Alex Myers
  • Anthony Towns
  • Antoine Poinsot
  • Armin Sabouri
  • Bastien Teinturier
  • Brandon Black
  • Burak Keceli
  • Calvin Kim
  • Carla Kirk-Cohen
  • Christian Decker
  • Clara Shikhelman
  • Dan Gould
  • Dave Harding
  • Dusty Daemon
  • Eduardo Quintana
  • Ethan Heilman
  • Fabian Jahr
  • Gijs van Dam
  • Gloria Zhao
  • Gregory Sanders
  • Henrik Skogstrøm
  • Jack Ronaldi
  • James O’Beirne
  • Jesse Posner
  • Johan Torås Halseth
  • Joost Jager
  • Josie Baker
  • Ken Sedgwick
  • Larry Ruane
  • Lisa Neigut
  • Lukas George
  • Martin Zumsande
  • Matt Corallo
  • Matthew Zipkin
  • Max Edwards
  • Maxim Orlovsky
  • Nick Farrow
  • Niklas Gögge
  • Olaoluwa Osuntokun
  • Pavlenex
  • Peter Todd
  • Pieter Wuille
  • Robin Linus
  • Ruben Somsen
  • Russell O’Connor
  • Salvatore Ingala
  • Sergi Delgado Segura
  • Severin Bühler
  • Steve Lee
  • Steven Roose
  • Thomas Hartman
  • Thomas Voegtlin
  • Tom Briar
  • Tom Trevethan
  • Valentine Wallace
  • Vivek Kasarabada
  • Will Clark
  • Yuval Kogman
  • ZmnSCPxj

Optech also published two contributed field reports from the business community: one from Brandon Black at BitGo about implementing MuSig2 for reducing fee costs and improving privacy, and a second report from Antoine Poinsot at Wizardsardine about building software with miniscript.

December

  • ● Cluster mempool: Several Bitcoin Core developers began working on a new cluster mempool design to simplify mempool operations while maintaining the necessary transaction ordering, where parent transactions must be confirmed before their children. Transactions are grouped into clusters, then split into feerate-sorted chunks, ensuring that high feerate chunks are confirmed first. This allows the creation of block templates by simply choosing the highest feerate chunks in the mempool, and eviction of transactions by dropping the lowest feerate chunks. This fixes some existing undesirable behavior (e.g., where miners might lose out on fee revenue due to suboptimal evictions) and may be able to improve other aspects of mempool management and transaction relay in the future. Archives of their discussions were published in early December.

  • ● Warnet: December also saw the public announcement of a new tool for launching a large number of Bitcoin nodes with a defined set of connections between them (usually on a test network). This can be used to test behavior that’s difficult to replicate using small numbers of nodes or which would cause problems on public networks, such as known attacks and propagation of gossiped information. One public example of using the tool was measuring the memory consumption of Bitcoin Core before and after a proposed change.

We thank all of the Bitcoin contributors named above, plus the many others whose work was just as important, for another incredible year of Bitcoin development. The Optech newsletter will return to its regular Wednesday publication schedule on January 3rd.