The Bitcoin Optech Scaling Book is a guide to effective practices and techniques that engineers interacting with the Bitcoin blockchain can adopt today. These practices are good for your business (they make your operations run more efficiently and save you money) and good for the ecosystem (they minimise your usage of common network resources and ensure that the Bitcoin network remains available for as many users and use cases as possible).

What is it that we mean by scaling when talking about the Bitcoin blockchain? In general, we can call a system ‘scalable’ if the resources consumed by the system scales no worse than linearly with the number of users or units of activity in the system. The Bitcoin base layer as a payment network does not appear to achieve this definition of scalability. Since the Bitcoin network makes use of a peer-to-peer gossip protocol in which every node validates every transaction, the global amount of computation, bandwidth and storage required scales (in at least some senses) quadratically with the number of users. n users validating transactions from n users results in the order of n2 validations. Even if we radically reduce the security by requiring new users to be ‘light clients’ and not validate the full state of the system, adding new users by increasing the transaction throughput is still at best linear in total global resource cost.

And there’s another problem. In many systems, the costs of the additional resources consumed is borne by the party benefiting from the additional activity. If a company needs to acquire additional computing resource to serve new customers, then that company would pay for the additional servers or cloud computing resources and benefit from the increased revenue generated by the customers. That isn’t the case with on-chain Bitcoin transactions. Each full node operator bears the resource costs of validating transactions, in the form of bandwidth, compute, memory and storage. Naively increasing the transaction throughput of the Bitcoin network would therefore benefit parties creating additional transactions at the expense of everyone running a full node. Many users recognise that a vital component of maintaining a decentralized network is keeping the cost of running a full node low and allowing as many users as possible to run a full node. An attempt in 2017 to double the block size (and hence increase the resource requirements of running a full node), failed to gain consensus and was withdrawn by its proponents.

So if scaling up the underlying resource requirements is not currently an option (and in any case could not deliver the global scale that Bitcoin proponents would like to see), scaling Bitcoin becomes a question of how to maximize the utility that can be delivered within the constraints of the system, or in other words, how we can achieve more with less?

The utility that the Bitcoin blockchain delivers is payments, since that’s what users want to use the Bitcoin system for. To confuse the logical concept of a Bitcoin transaction with the social concept of a payment is a conspicuous (and sadly very common) error. A payment is a transfer of value from one party to another. In Bitcoin, a transaction is a transformation on the global set of unspent transaction outputs, consuming some UTXOs and producing some new UTXOs. Transaction can also refer to a data structure that is passed over the P2P network that both describes the transformation on the UTXO set and carries a witness that the transformation is valid. In neither of these definitions does a transaction correspond to a payment - each transaction can realize one, many or even no payments.

To try to measure scalability with a metric like transactions per second (tps) is therefore flawed. Firstly, tps is not measuring what we’re interested in, and secondly, not all payments are equally useful.

To give a concrete Example of the first point, if high-frequency Bitcoin users switched from sending many single-recipient transactions per day to sending a single batched transaction per day, the average blockchain weight per payment would decrease, but the average blockchain weight per transaction would increase. We’d very likely see a Bitcoin network with fewer transactions per day, but a greater number of payments. Utility is up, but tps is down!

The second reason is that not all transactions are equally useful. Since each transaction is simply a transformation on the set of UTXOs, the Bitcoin consensus rules do not recognise any difference in importance between different transactions. The transaction produced by an institutional investor moving hundreds of Bitcoin into a cold storage multisig output as part of a long-term investment strategy could look identical to the transaction for a customer paying for a cup of coffee. Each counts as one ‘transaction’ but the first user probably gets much more utility from his transaction than the second user. The former may be prepared to pay a high fee to have his transaction recorded in a decentralized, trustless, immutable ledger, whereas the latter may be equally satisfied to have his payment made on a second-layer like lightning, a side-chain or even a trusted off-chain payment network.

The Bitcoin network has a mechanism for users to express the urgency and utility of their transactions through transaction fees, but at the time of writing, the fee market is underdeveloped, inelastic and immature. As the network develops, it is expected that the fee market will become more sophisticated.

When considered in this light, many practices which might not at first be considered as ‘scaling technologies’ can be considered as such. For example, a functioning fee market, and above all a reliable way to ‘bump’ fees on a transaction that has not yet included in a block, allows users to better express their time preference for a transaction to be confirmed. That means that the scarce block space can be allocated more efficiently to those users who value it more, and therefore the total utility delivered by the network is increased. This book takes this wider view of ‘scalability’. Any technique that can increase the usefulness of the network in aggregate can be thought of as contributing to scaling.

Take this book then as a guide to extracting the most possible value out of this scarce and precious resource called the Bitcoin blockchain. The protocol wizards continue to innovate on the base and second layers, and exciting developments like Schnorr signatures, taproot, signature aggregation, eltoo, atomic multi path payments and channel splicing are all in the works. But for now, we can all do our bit to minimize our impact on the base layer and allow as many users as possible to transact permissionlessly and trustlessly.

About Bitcoin Optech

Bitcoin Optech was formed in early 2018. Our aim is to help Bitcoin companies adopt the best scaling technologies and practices available to make efficient use of the blockchain, and thereby help Bitcoin to scale to more users and use cases.

During late 2017 and early 2018, we saw demand for Bitcoin usage explode and many users and companies were unready for the sudden spike in transaction volume. Block space suddenly became a competitive a rivalrous good, and transaction fees jumped as demand exceeded supply.

At the same time, many well-understood scaling technologies were being underutilized. Segwit usage remained below 40%, some high-frequency users of the chain were sending payments as individual transactions rather than batching, fee estimates were failing to react to fee market conditions and poor coin selection led to wallets accumulating low value UTXOs and paying very high fees.

The end result was a poor and costly experience for users, some use cases being priced out entirely, and Bitcoin failing to scale to meet demand. At Bitcoin Optech, we believe that seawalls are best built during low tide, and hope that Bitcoin companies will take advantage of conditions of low network usage and transaction fees to implement these scaling techniques. Those that do will be best positioned to continue offering their customers the best reliability and value when Bitcoin usage surges again.

Intended Audience

This book is intended for engineers who work in developer or operational roles at companies that interact with the Bitcoin blockchain. It is meant as a practical guide for people who are already familiar with standard Bitcoin concepts like ScriptPubKeys, ScriptSigs and witnesses; the SCRIPT language and standard output types; public and private keys; and transaction serialization. See Mastering Bitcoin or Bitcoin and Cryptocurrency Technologies for a beginners guide to those concepts or btcinformation.org for a comprehensive reference.

Likewise, this book is not overly concerned with the cryptographic underpinnings of Bitcoin, except where those are directly required for improving the efficiency and scalability of Bitcoin operations. Cryptography is a fascinating subject and Dan Boneh’s Applied Cryptography course is an excellent introduction for those wishing to begin their exploration.

Conventions

Most terms are used as readers would expect. The script in a transaction output which encumers the output is the scriptPubKey. The script in the transaction input which unlocks the funds is the scriptSig. The scriptCode is the actually executed script (which is the redeemscript in non-segwit P2SH, and the witnesscript in P2WSH/P2WPKH).

The smallest unit of account is referred to as a satoshi by convention, and 10^8 satoshis is named a bitcoin. No other units of account are used.

The party making a payment is called the spender (sender could be ambiguously refer to a party sending Bitcoin or a party sending data). There is no equivalent word in English to disambiguate the receiver of a payment and the receiver of data, so we use the word recipient to refer to one receiving a payment, and live with the ambiguity.

Transaction weight and virtual bytes

The consensus rule for block size is that the combined weight of transactions must be less than 4,000,000. The weight of a single transaction is calculated as follows:

  • The base size of a transaction is the number of bytes to serialize the block without witness
  • The size of a transaction is the number of bytes to serialize the block with witness
  • The weight is 3 x base size + size

This formula means that the bytes in the witness are ‘discounted’ by a factor of 4. They count less towards the weight of the transaction than bytes in the base transaction.

The virtual size of a transaction is weight / 4 and is measured in virtual bytes or vBytes. For transactions without any segwit inputs, the base size, size and virtual size are all equal.



Previous:
Table of contents
Next:
Fee Bumping