Bech32 Sending Support

On this page are copies of all published parts of our 24-part weekly series on bech32 sending support, from March 19th to August 28th, 2019.

1. Introduction to Bech32 Sending Support
   1. Sending to a legacy address
   2. Sending to a bech32 address
2. Bech32 usage statistics
3. Using the bech32 reference libraries
4. Locating typos in bech32 addresses
5. Fee savings with native segwit
6. Top bech32 questions from the Bitcoin Stack Exchange
7. Other addresses formats based on bech32
8. Automatic bech32 support for future soft forks
9. Creating more efficient QR codes with bech32 addresses
10. Bech32 support as a proxy for competence
11. Wallets that only support bech32 receiving
12. Bech32 trivia
13. Adoption speed
14. Address security
15. Reading and transcribing bech32 addresses
16. Quick testing checklist
17. Message signing support
18. Survey of support issues
19. Fee savings in dollar terms
20. Bech32 adoption rate
21. BRD field report
22. Same fee, faster confirmation
23. Insights from segwit compatibility matrix
24. Conclusion

Introduction to Bech32 Sending Support

Originally published in Newsletter #38.

Bech32 native segwit addresses were first publicly proposed almost exactly two years ago, becoming the BIP173 standard. This was followed by the segwit soft fork’s lock-in on 24 August 2017. Yet, seventeen months after lock-in, some popular wallets and services still don’t support sending bitcoins to bech32 addresses. Developers of other wallets and services are tired of waiting and want to default to receiving payments to bech32 addresses so that they can achieve additional fee savings and improved privacy. Bitcoin Optech would like to help this process along so, from now until the two-year anniversary of segwit lock-in, each of our newsletters will include a short section with resources to help get bech32 sending support fully deployed.

Note, we are only directly advocating bech32 sending support. This allows the people you pay to use segwit but doesn’t require you to implement segwit yourself. (If you want to use segwit yourself to save fees or access its other benefits, that’s great! We just encourage you to implement bech32 sending support first so that the people you pay can begin taking advantage of it immediately while you upgrade the rest of your code and infrastructure to fully support segwit.) To that end, this week’s section focuses on showing exactly how small the differences are between sending to a legacy address and sending to a bech32 address.

Sending to a legacy address

For a P2PKH legacy address that you already support such as 1B6FkNg199ZbPJWG5zjEiDekrCc2P7MVyC, your base58check library will decode that to a 20-byte commitment:

 6eafa604a503a0bb445ad1f6daa80f162b5605d6

This commitment is inserted into a scriptPubKey template:

OP_DUP OP_HASH160 OP_PUSH20 6eafa604a503a0bb445ad1f6daa80f162b5605d6 OP_EQUALVERIFY OP_CHECKSIG

Converting the opcodes to hex, this looks like:

76a9146eafa604a503a0bb445ad1f6daa80f162b5605d688ac

This is inserted into the scriptPubKey part of an output that also includes the length of the script (25 bytes) and the amount being paid:

     amount                           scriptPubKey
|--------------|  |------------------------------------------------|
00e1f505000000001976a9146eafa604a503a0bb445ad1f6daa80f162b5605d688ac
                |
    size: 0x19 -> 25 bytes

This output can then be added to the transaction, which is then signed and broadcast.

Sending to a bech32 address

For an equivalent bech32 P2WPKH address such as bc1qd6h6vp99qwstk3z668md42q0zc44vpwkk824zh, you can use one of the reference libraries to decode the address to a pair of values:

0 6eafa604a503a0bb445ad1f6daa80f162b5605d6

These two values are also inserted into a scriptPubKey template. The first value is the witness script version byte that’s used to add a value to the stack using one of the opcodes from OP_0 to OP_16. The second is the commitment that’s also pushed onto the stack:

OP_0 OP_PUSH20 6eafa604a503a0bb445ad1f6daa80f162b5605d6

Converting the opcodes to hex, this looks like:

00146eafa604a503a0bb445ad1f6daa80f162b5605d6

Then, just as before, this is inserted into the scriptPubKey part of an output:

     amount                        scriptPubKey
|--------------|  |------------------------------------------|
00e1f505000000001600146eafa604a503a0bb445ad1f6daa80f162b5605d6
                |
    size: 0x16 -> 22 bytes

The output is added to the transaction. The transaction is then signed and broadcast.

For bech32 P2WSH (the segwit equivalent of P2SH) or for future segwit witness versions, you don’t need to do anything special. The witness script version may be a different number, requiring you to use the corresponding OP_0 to OP_16 opcode, and the commitment may be a different length (from 2 to 40 bytes), but nothing else about the output changes. Because length variations are allowed, ensure your fee estimation software considers the actual size of the scriptPubKey rather than using a constant someone previously calculated based on P2PKH or P2SH sizes.

What you see above is the entire change you need to make on the backend of your software in order to enable sending to bech32 addresses. For most platforms, it should be a very easy change. See BIP173 and the reference implementations for a set of test vectors you can use to ensure your implementation works correctly.

Bech32 usage statistics

Originally published in Newsletter #39.

As described last week, implementing just segwit spending should be easy. Yet we suspect some managers might wonder whether there are enough people using segwit to justify their team spending development effort on it. This week, we look at sites that track various segwit adoption statistics so that you can decide whether it’s popular enough that your wallet or service might become an outlier by failing to support it soon.

Optech tracks statistics about segwit use on our dashboard; another site tracking related statistics is P2SH.info. We see an average of about 200 outputs per block are sent to native segwit addresses (bech32). Those outputs are then spent in about 10% of all Bitcoin transactions. That makes payments involving native segwit addresses more popular than almost all altcoins.

However, many wallets want to use segwit but still need to deal with services that don’t yet have bech32 sending support. These wallets can generate a P2SH address that references their segwit details, which is less efficient than using bech32 but more efficient than not using segwit at all. Because these are normal P2SH addresses, we can’t tell just by looking at transaction outputs which P2SH addresses are pre-segwit P2SH outputs and which contain a nested segwit commitment, and so we don’t know the actual number of payments to nested-segwit addresses. However, when one of these outputs is spent, the spender reveals whether the output was segwit. The above statistics sites report that currently about 37% of transactions contain at least one spend from a nested-segwit output. That corresponds to about 1,400 outputs per block on average.

Any wallet that supports P2SH nested segwit addresses also likely supports bech32 native addresses, so the number of transactions made by wallets that want to take advantage of bech32 sending support is currently over 45% and rising.

To further gauge segwit popularity, you might also want to know which notable Bitcoin wallets and services support it. For that, we recommend the community-maintained bech32 adoption page on the Bitcoin Wiki or the when segwit page maintained by BRD wallet.

The statistics and compatibility data show that segwit is already well supported and frequently used, but that there are a few notable holdouts that haven’t yet provided support. It’s our hope that our campaign and other community efforts will help convince the stragglers to catch up on bech32 sending support so that all wallets that want to take advantage of native segwit can do so in the next few months.

Using the bech32 reference libraries

Originally published in Newsletter #40.

In a previous week, we discussed how small the differences are between creating the output for a legacy address versus a native segwit address. In that section we simply pointed you towards the bech32 reference libraries and told you that you’d get two values back. In this week, we walkthrough the exact steps of using the Python reference library so you can see how little work this is. We start by importing the library:

>>> import segwit_addr

Bech32 addresses have a Human-Readable Part (HRP) that indicates what network the address is for. These are the first few characters of the address and are separated from the data part of the address by the delimiter 1. For example, Bitcoin testnet uses tb and an example testnet address is tb1q3w[…]g7a. We’ll set the Bitcoin mainnet HRP of bc in our code so that we can later ensure that the addresses we parse are for the network we expect.

>>> HRP='bc'

Finally, we have a few addresses we want to check—one that should work and two that should fail. (See BIP173 for a complete set of reference test vectors.)

>>> good_address='bc1qd6h6vp99qwstk3z668md42q0zc44vpwkk824zh'
>>> typo_address='bc1qd6h6vp99qwstk3z669md42q0zc44vpwkk824zh'
>>> wrong_network_address='tb1q3wrc5yq9c300jxlfeg7ae76tk9gsx044ucyg7a'

Now we can simply attempt to decode each of these addresses

>>> segwit_addr.decode(HRP, good_address)
(0, [110, 175, 166, 4, 165, 3, 160, 187, 68, 90, 209,
     246, 218, 168, 15, 22, 43, 86, 5, 214])

>>> segwit_addr.decode(HRP, typo_address)
(None, None)

>>> segwit_addr.decode(HRP, wrong_network_address)
(None, None)

If we get back a None for the first value (the witness version), the address is invalid on our chosen network. If that happens, you want to throw an exception up the stack so that whatever process is interfacing with the user can get them to provide you with a correct address. If you actually get a number and an array, the decode succeeded, the checksum was valid, and the length was within the allowed range.

The witness version must be a number between 0 and 16, so you’ll want to check that (e.g. 0 <= x <= 16) and then convert it into the corresponding opcodes OP_0 through OP_16. For OP_0, this is 0x00; for OP_1 through OP_16, this is 0x51 through 0x60. You then need to add a push byte for the data depending on its length (0x02 through 0x28 for 2 to 40 bytes), and then append the data as a series of bytes. Pieter Wuille’s code does this quite succinctly:

>>> witver, witprog = segwit_addr.decode(HRP, good_address)
>>> bytes([witver + 0x50 if witver else 0, len(witprog)] + witprog).hex()
'00146eafa604a503a0bb445ad1f6daa80f162b5605d6'

That’s your entire scriptPubKey. You can use that in the output of a transaction and send it. Note that bech32 scriptPubKeys can vary in size from 4 to 42 vbytes, so you need to consider the actual size of the scriptPubKey in your fee estimation code.

Your code doesn’t need to be written in Python. Reference libraries are also provided for C, C++, Go, Haskell, JavaScript, Ruby, and Rust. Further, BIP173 describes bech32 well enough that any decent programmer should be able to implement it from scratch in their preferred language without requiring anything beyond what most programming languages provide in their builtins and standard library.

Other bech32 sending support updates: BitGo announced that their API now supports sending to bech32 addresses; see their announcement for additional details about bech32 receiving support. The Gemini exchange also apparently added bech32 sending support this week, and users report that Gemini defaults to accepting deposits to bech32 addresses as well.

Locating typos in bech32 addresses

Originally published in Newsletter #41.

In last week’s newsletter, we used the Python reference library for bech32 to decode an address into a scriptPubKey that you could pay. However, sometimes the user provides an address containing a typo. The code we suggested would detect the typo and ensure you didn’t pay a wrong address, but bech32 is also able to help detect the location of typos for your users. This week, we’ll demonstrate this capability using the Javascript sample code.

The code is written using Node.js-style module inclusion syntax, so the first step is to compile it into code we can use in the browser. For that, we install a browserify tool:

sudo apt install node-browserify-lite

Then we compile it into a standalone file:

browserify-lite ./segwit_addr_ecc.js --outfile bech32-demo.js --standalone segwit_addr_ecc

Followed by including it in our HTML:

<script src="bech32-demo.js"></script>

For convenience, we’ve included that file on the web version of this newsletter, so you can follow along with the rest of this example by simply opening the developer console in your web browser. Let’s start by checking a valid address. Recall from last week that we provide the network identifier when checking an address (bc for Bitcoin mainnet):

>> segwit_addr_ecc.check('bc1qw508d6qejxtdg4y5r3zarvary0c5xw7kv8f3t4', 'bc')
error: null
program: Array(20) [ 117, 30, 118, … ]
version: 0

We see above that, just like last week, we get back the witness version and the witness program. The presence of the version field, plus the lack of an error, indicate that this program decoded without any checksum failure.

Now we replace one character in the above address with a typo and try checking that:

>> segwit_addr_ecc.check('bc1qw508d6qejxtdg4y5r4zarvary0c5xw7kv8f3t4', 'bc')
error: "Invalid"
pos: Array [ 21 ]

This time we get back the description of the error (the address is invalid because it doesn’t match its checksum) and a position. If we place the addresses above each other with each position marked, we see that this “21” identifies the location of the specific error:

                   1x        2x
         0123456789012345678901
>> good='bc1qw508d6qejxtdg4y5r3zarvary0c5xw7kv8f3t4'
>> typo='bc1qw508d6qejxtdg4y5r4zarvary0c5xw7kv8f3t4'
                              ^

What if we make an additional replacement to the typo address and try again?

>> segwit_addr_ecc.check('bc1qw508d6qejxtdg4y5r4zarvary0c5yw7kv8f3t4', 'bc')
error: "Invalid"
pos: Array [ 32, 21 ]

We get two locations. Once again, when we compare the addresses to each other, we see this has identified both incorrect characters:

                   1x        2x        3x
         012345678901234567890123456789012
>> good='bc1qw508d6qejxtdg4y5r3zarvary0c5xw7kv8f3t4'
>> typo='bc1qw508d6qejxtdg4y5r4zarvary0c5yw7kv8f3t4'
                              ^          ^

Pieter Wuille’s interactive demo of this Javascript code includes a few lines of additional code (view source on that page to see the function) that uses the position of the typo characters to emphasize them in red:

There’s a limit to how many errors the check() function can specifically identify. After that it can still tell that an address contains an error, but it can’t identify where to look in the address for the error. In that case, it’ll still return the address as invalid but it won’t return the position details:

>> segwit_addr_ecc.check('bc1qw508z6qejxtdg4y5r4zarvary0c5yw7kv8f3t4', 'bc')
error: "Invalid"
pos: null

In the case where there are other problems with the address, the error field will be set to a more descriptive message that may or may not include a position of the error. For example:

>> segwit_addr_ecc.check('bc1zw508d6qejxtdg4y5r3zarvary0c5xw7kv8f3t4yolo', 'bc')
error: "Invalid character"
pos: Array [ 43 ]

You can review the source for a complete list of errors.

Although we spent a lot of time looking at errors in this mini tutorial, we’ve hopefully shown how easy it is to provide nice interactive feedback to users entering bech32 addresses on a web-based platform. We encourage you to play around with the interactive demo to get an idea of what your users might see if you make use of this bech32 address feature.

Fee savings with native segwit

Originally published in Newsletter #42.

One reason your users and customers may want you to implement bech32 sending support is because it’ll allow the receivers of those payments to save on fees when they re-spend that money. This week, we’ll look at how much money they’ll save and discuss how their savings could also help you save money.

For the legacy P2PKH address format implemented in the first version of Bitcoin, the scriptSig that authorizes a spend is typically 107 vbytes. For P2SH-wrapped segwit P2WPKH, this same information is moved to a witness data field that only consumes 1/4 as many vbytes (27 vbytes) but whose P2SH overhead adds 23 vbytes for a total of 50 vbytes. For native segwit P2WPKH, there’s no P2SH overhead, so 27 vbytes is all that’s used.

This means you could argue that P2SH-P2WPKH saves over 50% compared to P2PKH, and that P2WPKH saves another almost 50% compared to P2SH-P2WPKH or 75% compared to P2PKH alone. However, spending transactions contain more than just scriptSigs and witness data, so the way we usually compare savings is by looking at prototype transactions. For example, we imagine a typical transaction containing a single input and two outputs (one to the receiver; one as change back to the spender). In that case:

• Spending P2PKH has a total transaction size of 220 vbytes
• Spending P2SH-P2WPKH has a size of 167 vbytes (24% savings)
• Spending P2WPKH output has a size of 141 vbytes (16% savings vs P2SH-P2WPKH or 35% vs P2PKH)

To compare simple multisig transactions (those that just use a single OP_CHECKMULTSIG opcode), things get more complex because k-of-n multisig inputs vary in size depending on the number of signatures (k) and the number of public keys (n). So, for simplicity’s sake, we’ll just plot the sizes of legacy P2SH-multisig compared to wrapped P2SH-P2WSH multisig (up to the maximum 15-of-15 supported by legacy P2SH). We can see that switching to P2SH-P2WSH can save from about 40% (1-of-2 multisig) to about 70% (15-of-15).

We can then compare P2SH-P2WSH to native P2WSH to see the additional constant-sized savings of about 35 bytes per transaction or about 5% to 15%.

The scripts described above account for almost all scripts being used with addresses that aren’t native segwit. (Users of more complex scripts, such as those used in LN, are mostly using native segwit today.) Those less efficient script types currently consume a majority fraction of block capacity (total block weight). Switching to native segwit in order to reduce a transaction’s weight allows you to reduce its fee by the same percentage without changing how long it’ll take to confirm—all other things being equal.

But all other things aren’t equal. Because the transactions use less block weight, there’s more weight available for other transactions. If the supply of available block weight increases and demand remains constant, we expect prices to go down (unless they’re already at the default minimum relay fee). This means more people spending native segwit inputs lowers the fee not just for those spenders but for everyone who creates transactions—including wallets and services that support sending to bech32 addresses.

Top bech32 questions from the Bitcoin Stack Exchange

Originally published in Newsletter #43.

This week we look at some of the top-voted bech32 questions and answers from the Bitcoin Stack Exchange. This includes everything since bech32 was first announced about two years ago.

• ● Will a Schnorr soft fork introduce a new address format? Although upgrading to bech32 sending support should be easy, you probably don’t want to repeat that work for Bitcoin’s next upgrade or the upgrade after that. Pieter Wuille answers this question by explaining how an upgrade to Schnorr-based public keys and signatures can still use bech32 addresses. (Optech will be covering this issue in greater detail in a future section.)

• ● Is it safe to translate a bech32 P2WPKH address into a legacy P2PKH address? If you read Newsletter #38, you’ll notice that the difference between a P2WPKH and P2PKH address for the same underlying public key is only a few characters in a scriptPubKey, making it possible to automatically convert one into the other. This answer by Andrew Chow and its accompanying comments explains why that’s a bad idea that could cause users to lose funds.

• ● Why does the bech32 decode function require specifying the address’s Human Readable Part (HRP) instead of extracting it automatically? The HRP is separated from the rest of the address by a 1, so it seems like the decoder could ignore that part all on its own. Pieter Wuille explains that calling the decoder with the expected HRP ensures that you don’t accidentally pay bitcoin to an address meant for testnet, litecoin, or some other network. Gregory Maxwell also corrects an additional assumption of the asker.

• ● What block explorers recognize bech32 addresses? More than two years after bech32 was first proposed and a year after this question was first asked, several popular block explorers don’t support search or display of bech32 addresses. The answer to this question suggests anyone who wants to learn the bech32 status of various block explorers should check the bech32 adoption Bitcoin Wiki page.

Other addresses formats based on bech32

Originally published in Newsletter #44.

It’s said that “imitation is the most sincere form of flattery.” In this week’s section, we take a quick look at a few other systems that are using variations on bech32. If you’re already going to need to implement something that’s basically bech32 for another project, it’s probably worth your time to implement it for Bitcoin too.

• ● LN invoices use the bech32 format with an extended Human-Readable Part (HRP) and without bech32’s normal 90-character limit. See BOLT11 for the full specification. Example: lnbc2500u1pvjluezpp5qqqsyqcyq5rqwzqfqqqsyqcyq5rqwzqfqqqsyqcyq5rqwzqfqypqdq5xysxxatsyp3k7enxv4jsxqzpuaztrnwngzn3kdzw5hydlzf03qdgm2hdq27cqv3agm2awhz5se903vruatfhq77w3ls4evs3ch9zw97j25emudupq63nyw24cg27h2rspfj9srp

• ● Bitcoin Cash new-style addresses use the bech32 format with the HRP bitcoincash and the separator :. Instead of the version byte encoding a segwit witness version, as in Bitcoin, it indicates whether the hash encoded by the address should be used with P2PKH or P2SH. See spec-cashaddr for the full specification. Example: bitcoincash:qpm2qsznhks23z7629mms6s4cwef74vcwvy22gdx6a

• ● Backup seeds: In June 2018, Jonas Schnelli proposed Bech32X, a scheme to encode Bitcoin private keys, extended private keys (xprivs), and extended public keys (xpubs) using bech32 for error correction. See the full draft specification. Example: pk1lmll7u25wppjn5ghyhgm7kndgjwgphae8lez0gra436mj7ygaptggl447a4xh7

• ● Elements-based sidechains: sidechains based on ElementsProject.org, such as Blockstream Liquid, use both bech32 address and a variation of them called “blech32” addresses. Blech32 addresses are intended for use with that platform’s confidential assets and will soon be supported by the Esplora block explorer for the Liquid sidechain. We’re unaware of a specification document for blech32, but this code is labeled as the reference implementation and is cited elsewhere in the project as, “See liquid_addr.py for compact difference from bech32.” Example of a blech32 address: lq1qqf8er278e6nyvuwtgf39e6ewvdcnjupn9a86rzpx655y5lhkt0walu3djf9cklkxd3ryld97hu8h3xepw7sh2rlu7q45dcew5

• ● Output script descriptors: although less directly related to bech32, checksums based on the same Bose-Chaudhuri-Hocquenghem (BCH) codes used in bech32 were added to the output script descriptors supported by Bitcoin Core. See Pieter Wuille’s detailed comment. Example: wpkh([f6bb4c63/0'/0'/28']02bf9d38386db60191f2f785cbf7ba90d01bed5958efb7b449a552b89da7550177)#efkksxw6

Automatic bech32 support for future soft forks

Originally published in Newsletter #45.

What do bip-taproot and bip-tapscript mean for people who have implemented bech32 sending support or who are planning to implement it? In particular, if you haven’t implemented segwit sending support yet, should you wait to implement it until the new features have been activated? In this weekly section, we’ll show why you shouldn’t wait and how implementing sending support now won’t cost you any extra effort in the future.

The designers of segwit and bech32 had a general idea what future protocol improvements would look like, so they engineered segwit scriptPubKeys and the bech32 address format to be forward compatible with those expected improvements. For example an address supporting Taproot might look like this:

bc1pqzkqvpm76ewe20lcacq740p054at9sv7vxs0jn2u0r90af0k63332hva8pt

You’ll notice that looks just like other bech32 addresses you’ve seen—because it is. You can use the exact same code we provided in Newsletter #40 (using the bech32 reference library for Python) to decode it.

>> import segwit_addr
>> address='bc1pqzkqvpm76ewe20lcacq740p054at9sv7vxs0jn2u0r90af0k63332hva8pt'
>> witver, witprog = segwit_addr.decode('bc', address)
>> witver
1
>> bytes(witprog).hex()
'00ac06077ed65d953ff8ee01eabc2fa57ab2c19e61a0f94d5c78cafea5f6d46315'

The differences here from the decoded bech32 addresses we’ve shown in previous newsletters are that this hypothetical Taproot address uses a witness version of 1 instead of 0 (meaning the scriptPubKey will start with OP_1 instead of OP_0) and the witness program is one byte longer than a P2WSH witness program. However, these don’t matter to your software if you’re just spending. We can use the exact same example code from Newsletter #40 to create the appropriate scriptPubKey for you to pay:

>> bytes([witver + 0x50 if witver else 0, len(witprog)] + witprog).hex()
'512100ac06077ed65d953ff8ee01eabc2fa57ab2c19e61a0f94d5c78cafea5f6d46315'

This means anyone who implements bech32 support in the generic way described in Newsletter #40 shouldn’t need to do anything special in order to support future script upgrades. In short, the work you invest into providing bech32 sending support now is something you won’t need to repeat when future expected changes to the Bitcoin protocol are deployed.

Creating more efficient QR codes with bech32 addresses

Originally published in Newsletter #46.

BIP173 forbids bech32 addresses from using mixed case. The preferred way to write a bech32 address is in all lowercase, but there’s one case where all uppercase makes sense: QR codes. Take a look at the following two QR codes for the same address with the only difference being lowercase versus uppercase:

This is a deliberate design feature of bech32. QR codes can be created in several modes that support different character sets. The binary mode character set is used for legacy addresses because they require mixed case. However, Bech32 addresses for Bitcoin¹ can be represented using only numbers and capital letters, so they can use the smaller uppercase alphanumeric character set. Because this set is smaller, it uses fewer bits to encode each character in a QR code, allowing the resultant code to be less complex.

Bitcoin addresses are often used in BIP21 URIs. BIP21 technically requires base58check formatting, but at least some wallets that support native segwit addresses (such as Bitcoin Core) allow them to be used with bech32. Although the preferred form of BIP21’s scheme identifier bitcoin: is lowercase, it can also be uppercased as allowed by both BIP21 and RFC3986. This also produces a less complex image (although in this case, our QR encoding library gave both images the same dimensions).

Unfortunately, the ? and & needed for passing additional parameters in a BIP21 URI are not part of the QR code uppercase character set, so only binary mode can be used for those characters. Additionally, BIP21 specifies that query parameter names such as amount and label are case sensitive, so uppercase versions of them aren’t expected to work anyway.

However, QR codes can support mixed character sets and doing so will always be at least slightly more efficient when used with a string that either begins or ends with an all-caps substring containing a bech32 address. This is because the minimum allowed size of a bech32 address (14 characters) combined with the efficiency gain from using uppercase mode (31.25%) exceeds the worst-case overhead of switching modes (20 extra bits). At least two QR code encoders we’re aware of, libqrencode (C) and node-qrcode (JS), automatically mix character sets by default as necessary to produce the least-complex QR code possible:

In summary, when using bech32 addresses in QR codes, consider uppercasing them and any other adjacent characters that can be uppercased in order to produce smaller and less complex QR codes. (However, for all other purposes, bech32 addresses should use all lowercase characters.)

Correction: an earlier version of this section claimed that QR codes which included BIP21 query parameters needed to use binary mode. Nadav Ivgi kindly informed us that it was possible to mix character modes, and we’ve updated the final two paragraphs of this section accordingly.

Bech32 support as a proxy for competence

Originally publised in Newsletter #47.

Up until this point in our series encouraging wallets and services to support sending to bech32 native segwit addresses, we’ve focused almost exclusively on technical information. Today, this section expresses an opinion: the longer you delay implementing bech32 sending support, the worse some of your users and potential users will think of your software or service.

“They can only pay legacy addresses.”
“Oh. Let’s look for another service that supports current technology.”

Services that only support legacy addresses are likely to become a cue to users that minimal development effort is being put into maintaining their Bitcoin integration. We expect that it’ll send the same signal to users as a website in 2019 that’s covered in Shockwave/Adobe Flash elements and that claims it’s best viewed in Internet Explorer 7 (or see an even more imaginative comparison written by Gregory Maxwell.)

Bech32 sending is not some experimental new technology that still needs testing—native segwit unspent outputs currently hold over 200,000 bitcoins. Bech32 sending is also something that’s easy to implement (see Newsletters #38 and #40). Most importantly, as more and more wallets and services upgrade to bech32 receiving by default, it’s going to become obvious which other services haven fallen behind by not providing sending support.

If you haven’t implemented bech32 sending support yet, we suggest you try to get it implemented by 24 August 2019 (the two-year anniversary of segwit activation). Not long after that, Bitcoin Core’s next release is expected to begin defaulting to bech32 receiving addresses in its GUI and perhaps also its API methods (see Newsletters #40 and #42). We expect other wallets to do the same—except for the ones that have already made bech32 their default (or even their only supported address format).

Wallets that only support bech32 receiving

Originally published in Newsletter #48.

Last week, we described one of the costs of not upgrading to bech32 sending support—users might think your service is out-of-date and so look for alternative services. This week, we’ll look at the stronger form of that argument: wallets which already can only receive to bech32 addresses. If the users of these wallets want to receive a payment or make a withdrawal from your service, and you don’t yet support sending to bech32 addresses, they’ll either have to use a second wallet or have to use one of your competitors.

• ● Wasabi wallet, known for its privacy-enhancing coinjoin mode and mandatory user coin control, only accepts payments to bech32 addresses. This relatively-new wallet was designed around compact block filters similar to those described in BIP158. However, since all of the filters are served by Wasabi’s infrastructure, the decision was made to minimize filter size by only including P2WPKH outputs and spends in the filter. This means the wallet can’t see payments to other output types, including P2SH for P2SH-wrapped segwit addresses.

• ● Trust wallet is a fairly new proprietary wallet owned by the Binance cryptocurrency exchange and compatible with Android and iOS. As a new wallet, they didn’t need to implement legacy address receiving support, so they only implemented segwit. That makes bech32 the only supported way to send bitcoins to this wallet.

• ● Electrum is a popular wallet for desktop and mobile. When creating a new wallet seed, you can choose between a legacy wallet and a segwit wallet, with segwit being the current default. Users choosing a segwit wallet seed will only be able to generate bech32 addresses for receiving. Electrum warns users about the compatibility issues this may create with software and services that haven’t upgraded to bech32 sending support yet:

  

  Please note that it’s neither required nor recommended for wallet authors to create a new seed in order to support a new address format. Other wallets, such as Bitcoin Core 0.16.0 and above, can produce legacy, p2sh-segwit, and bech32 addresses all from the same seed—the user just needs to specify which address type they want (if they don’t want the default).

As time goes on, we expect more new wallets to only implement receiving to the current best address format. Today that’s v0 segwit addresses for P2WPKH and P2WSH using bech32, but if Taproot is adopted, it will use v1 segwit addresses that will also use bech32. The longer your service delays implementing bech32 sending support, the more chance you’ll have of losing customers because they can’t request payments from you using their preferred wallet.

Bech32 trivia

Originally published in Newsletter #49.

This segment marks half-way through our series about bech32, so we decided to have some fun this week by describing some bech32-related trivia that’s interesting but not important enough for its own segment.

• ● How is bech32 pronounced? Pieter Wuille, co-author of the proposal, uses a soft “ch” so that the word sounds like “besh thirty two”. The name is a portmanteau that mixes the letters of the address’s error correction coding (BCH) into the name of its numeric base (base32). Pronouncing it with a soft “ch” allows the first syllable of bech32 to be similar to Bitcoin’s legacy address format, base58. We admit this extended explanation ruins the joke, but it’s a clever and amusing bit of wordplay.

• ● BCH has nothing to do with Bitcoin Cash’s ticker code: the name of the BCH codes bech32 is based upon are an abbreviation for Bose-Chaudhuri-Hocquenghem, with Hocquenghem inventing this type of cyclic codes in 1959 followed by Bose and Ray-Chaudhuri independently rediscovering them in 1960. Moreover, the bech32 address format was announced in March 2017, three months before the first plans for what would later be labeled Bitcoin Cash (which initially planned to use the ticker code BCC).

• ● Over ten CPU years consumed: using existing information about BCH codes, the authors of bech32 were able to find the set of codes that provided the minimum amount of error detection they desired for Bitcoin addresses. However, there were almost 160 thousand eligible codes in this set, and the authors expected some of them to be better than others. To find the best code among them, over 200 CPU cores and “more than 10 years of computation time” was used.

Adoption speed

Originally published in Newsletter #50.

Bech32 addresses aren’t the first time some Bitcoin users have changed address formats. In April 2012, P2SH addresses starting with a 3 were introduced and eventually came to be used in about 25% of all transaction outputs. This week, we’ll look at the relative speed of adoption of the two different address formats. For reasons we describe later, this can’t be an entirely fair comparison, but it may provide us with a rough guide to how well we’re doing so far with bech32 adoption.

We’ll first look at the percentage of outputs per block sent to P2SH or native segwit (bech32) addresses as measured from the day each proposal became active on mainnet. All plots in this section are averaged over 30 days using a simple moving average. We also limit the data points on P2SH plot to about two months before segwit activation so that almost no P2SH-wrapped segwit outputs are miscounted as legacy P2SH.

One particularly unfair aspect of the above plot is that P2SH is really only useful for advanced scripts (such as multisig). There was no need and no benefit for anyone using single-sig addresses (those starting with a 1) to upgrade to P2SH. By comparison, there are native segwit addresses for both single-sig users (P2WPKH) and advanced script users (P2WSH). To try to make that comparison more fair, the following plot over a smaller date range separates the two uses of native segwit so you can compare bech32 P2WSH use to its rough equivalent P2SH.

Notably, we see that almost all use of native segwit addresses to date is for single-sig P2WPKH. The P2SH activity prior to segwit activation, which peaked at 25% of all outputs, has not migrated to native P2WSH outputs. Indeed, when we consider that all LN deposit transactions (and at least some other onchain LN transaction) are using native P2WSH outputs, it appears as if almost none of the late-2017 P2SH activity has converted to P2WSH so far.

This points to another aspect that makes the different address data hard to compare: all the things that were possible using legacy P2SH are also possible using either P2SH-wrapped segwit addresses or native P2WSH addresses. P2SH-wrapped segwit addresses are backwards compatible and can significantly reduce transaction fees whereas bech32 addresses aren’t backwards compatible with older wallets and only save a small fixed amount of extra fees compared to P2SH-wrapped segwit. This may give users of advanced scripts less incentive than other users to switch from P2SH-wrapped segwit address to native segwit addresses in the short term.

Overall, the plot seems to show that it took about three years for P2SH addresses to really start taking off, but that bech32 address were already successful within just a few months of the segwit soft fork activation. With some wallets already defaulting to bech32 and several more planning to do so in the coming months, we expect to see increased adoption before the end of 2019.

Address security

Originally published in Newsletter #51.

There’s a class of multisig users who not only save on fees by using bech32 addresses but who also receive improved security against a potential type of attack called a hash collision. This class of users includes many exchanges and other business users.

To provide some background, all common single-sig addresses on Bitcoin today are the result of a pubkey being turned into a 160-bit RIPEMD160 hash digest. It’s theoretically possible for an attacker to generate a second pubkey that they control, hash it, and produce the same address. However, if we assume that the hash function produces perfectly unpredictable output, then the chance of this happening with a 160-bit hash function like RIPEMD160 is 1-in-2¹⁶⁰ for each pubkey the attacker tries.

For comparison, Bitcoin miners perform 2⁸⁰ hashing operations roughly every 5 hours as of this writing. The SHA256d hashing operation miners perform isn’t the same as used in this RIPEMD160 collision attack, so their equipment can’t be repurposed for that use, but we can use this as a reference rate for the number of brute force operations a real-world system can perform today (at great expense). At that rate, an attack that performed the 2¹⁵⁹ operations necessary to have a 50% chance of succeess would take about 25 million times the estimated age of the universe so far.

However, when multisig addresses are being used, the attacker may be one of the parties involved in generation of the address and so may be able to manipulate what address is finally chosen. For example, Bob sends his pubkey to Mallory expecting that Mallory will send her pubkey back. Then he expects they’ll each put the pubkeys into a multisig script template, hash it into an address, and someone will deposit money into that address.

Mallory instead takes the script template and Bob’s pubkey, inserts one of her pubkeys without telling Bob about it, and hashes it into an address. This allows her to see the address Bob will accept before Mallory has committed to using that pubkey. Mallory can then compare this address to a database of addresses generated from scripts that pay only her. If there’s a match (collision) between two of the addresses, she sends the pubkey back to Bob, waits for money to be deposited into the address, and then uses the script from her database to steal the money. If there’s not a match, Mallory can try again with a different pubkey over and over until she succeeds (if we assume she has unlimited time and resources).

Although this seems like the same brute force attack described earlier with a 1-in-2¹⁶⁰ chance of success per attempt, we have to consider the size of Mallory’s database. If we imagine the database has 100 addresses, then each different pubkey she tries has a 100-in-2¹⁶⁰ chance of success because it succeeds if it matches any one of the addresses in Mallory’s database.

This type of attack is called a collision attack. There are several algorithms with different CPU/memory tradeoffs for collision attacks, but the general rule security researchers follow is that a collision attack against a perfect hash function reduces its security to the square root of its number of combinations, i.e. it halves its size in bits. That means we can roughly assume that RIPEMD160’s security is reduced to 80 bits—which is the same number of operations we mentioned Bitcoin miners perform every 5 hours today using currently-existing technology. Again Bitcoin mining equipment can’t be used for this attack, and for an attacker to design and build enough custom equipment to find a collision in five hours might cost them billions of dollars—but it’s a theoretically possible attack that should concern those storing large values in P2SH, especially as custom hardware gets faster and cheaper. It’s also possible that there are variations on the collision attack that are easier and cheaper to execute because of weaknesses in the RIPEMD160 function.

It is possible to design multisig setup protocols so that they don’t have this problem and so that their collision resistance remains at 160 bits. However, the developers of segwit believed it was better to use a slightly longer hash function for segwit’s P2SH analog—P2WSH—so that users didn’t need to worry about these cryptographic particulars. For that reason segwit P2WSH uses the same SHA256d function used elsewhere in Bitcoin that provides 256-bit security for single-party cases and 128-bit worst-case security in multi-party use cases. To to continue our rough comparison, if we consider 80 bits to be equivalent to five hours of Bitcoin mining, 128 bits is equivalent to 160 billion years of mining.

Before we conclude this section, we do want to ensure a few things are clear:

1. We think it’s unlikely that anyone today has the ability to execute the attack described (but we can’t rule it out as a risk).

2. The attack can only be used at the time addresses are being created (although the actual theft could occur a long time afterwards).

3. The attack only applies to multi-party multisig addresses. If you’re a single party using P2SH multisig with only your own trusted devices or you’re using P2SH-P2WPKH (single-sig addresses), there’s no risk to you from this attack.

4. The attack applies to P2SH-wrapped segwit P2WSH addresses as well as regular P2SH addresses. To eliminate the risk, you have to use native segwit (bech32) addresses or a secure key exchange protocol.

To summarize, users of multi-party multisig who want the utmost in security should be migrating to bech32 P2WSH addresses to take advantage of their extra collision resistance. As users make that migration, it’ll become even more important for services to ensure they implement bech32 sending support so that they can send payments to those security-conscious users.

Reading and transcribing bech32 addresses

Originally published in Newsletter #52.

Before segwit was activated, developers discussed what format to use for native segwit addresses, with some developers suggesting that a new format was an opportunity to make addresses that were easier to read and transcribe. Developer Gregory Maxwell made this point rather effectively by asking other developers to call him up and try to successfully communicate a mixed-case legacy base58check address to him over the phone. If there was a communication error in just a single character—even just whether that character was uppercase or lowercase—both parties would need to go back and painstakingly try to locate the error.

BIP173 bech32 addresses were able to resolve both of these concerns. They use only a single case (lowercase preferred most of the time but uppercase can be used with QR codes for improved efficiency), and they use an error-correction code for a checksum so they can help users locate errors while ensuring typos will be caught an overwhelming percentage of the time.

However, as wallets and services consider upgrading to support both bech32 sending and receiving, we think it’s worth reminding any reluctant implementers about this key user-benefiting feature of bech32 addresses—so we’ve automated part of Maxwell’s old phone test to allow you to privately evaluate the relative difficulty of transcribing legacy and native segwit addresses.

If you click the following link (open it into a new tab), you’ll find a recording of two addresses paying the same hash value. You can type the addresses into the appropriate box below, which will turn red immediately if you enter any wrong character (case sensitive). Note: to help improve accuracy and eliminate problems with locale-specific letter pronunciations, we read each letter in the file using a phonetic alphabet, e.g. Alfa stands for A; bravo stands for B, etc.

Reading of base58check and bech32 addresses (1 minute, 33 seconds)

Legacy base58check address:

Native segwit bech32 address:

If you found the bech32 address much easier to transcribe accurately, then that means the designers of bech32 were successful at meeting one of their goals for the new address format. Users who discover this benefit of bech32 are more likely to want to use bech32 addresses in situations where they need to read or transcribe addresses, and so they’ll be more likely to use your software or service if it supports sending to bech32 addresses.

Quick testing checklist

Originally published in Newsletter #53.

Recently, an Optech contributor surveyed many popular wallets and Bitcoin exchanges to see what technical features they supported. For one of the exchanges, he initially recorded it as supporting sending to bech32 addresses—but later he discovered its support wasn’t entirely complete.

The problem was that the exchange supported P2WPKH bech32 addresses (single-sig addresses) but not P2WSH bech32 addresses (multisig and complex script addresses). Another problem was accepting all-lowercase bech32 addresses but not all-uppercase bech32 addresses. A different exchange limited the length of address form fields so that they couldn’t fit all valid bech32 addresses.

With these problems in mind, we’ve created a short checklist for testing basic bech32 sending support. Only perform these tests with small amounts of bitcoin that you can afford to lose if something goes wrong.

1. Generate two addresses of your own, one for P2WPKH and one for P2WSH. For example, using Bitcoin Core, the jq JSON parser, and the Bash shell, you can run the following commands:

     $ p2wpkh=$( bitcoin-cli getnewaddress "bech32 test" bech32 )
     $ p2wsh=$(
       bitcoin-cli addmultisigaddress 1 \[$(
         bitcoin-cli getaddressinfo $p2wpkh | jq .pubkey
       )\] "bech32 test" bech32 | jq -r .address
     )
     $ echo $p2wpkh $p2wsh
     $ echo $p2wpkh $p2wsh | tr '[a-z]' '[A-Z]'
   

2. Test sending bitcoin to each lowercase address using your software’s or service’s usual spending or withdrawal forms.

3. Test again with the uppercase form of each address (these are useful with QR codes).

4. Ensure that you received the funds by checking either the wallet you used to create the addresses or a block explorer. If that worked, your software fully supports current bech32 spending addresses.

   If you created addresses using a temporary Bitcoin Core wallet, you can wait for the transactions to confirm and then send all the money to your regular wallet using the following command: bitcoin-cli sendtoaddress YOUR_ADDRESS $( bitcoin-cli getbalance ) '' '' true

For unit tests where you don’t actually attempt to send money, or for integration tests where you send money on testnet or in regression testing mode, BIP173 provides a more comprehensive set of test vectors.

Message signing support

Originally published in Newsletter #54.

As we’ve shown in earlier parts of this series, bech32 addresses are better in almost every way than legacy addresses—they allow users to save fees, they’re easier to transcribe, address typos can be located, and they’re more efficient in QR codes. However, there is one feature that legacy P2PKH addresses support that is not widely supported by native segwit wallets—message signing support. In the spirit of full disclosure and the hopes of spurring wallet developers into action, we’ll take a look at this missing piece of bech32 address support.

For background, many wallets allow a user with a legacy P2PKH address to sign an arbitrary text message using the private key ultimately associated with that address:

$ bitcoin-cli getnewaddress "" legacy
125DTdGU5koq3YfAnA5GNqGfC8r1AZR2eh

$ bitcoin-cli signmessage 125DTdGU5koq3YfAnA5GNqGfC8r1AZR2eh Test
IJPKKyC/eFmYsUxaJx9yYfnZkm8aTjoN3iv19iZuWx7PUToF53pnQFP4CrMm0HtW1Nn0Jcm95Le/yJeTrxJwgxU=

Unfortunately, there’s no widely-implemented method for creating signed messages for legacy P2SH, P2SH-wrapped segwit, or native segwit addresses. In Bitcoin Core and many other wallets, trying to use anything besides a legacy P2PKH address will fail:

$ bitcoin-cli getnewaddress "" bech32
bc1qmhtn8x34yq9t7rvw9x6kqx73vutqq2wrxawjc8

$ bitcoin-cli signmessage bc1qmhtn8x34yq9t7rvw9x6kqx73vutqq2wrxawjc8 Test
error code: -3
error message:
Address does not refer to key

Some wallets do support message signing for segwit addresses—but in a non-standardized way. For example, the Trezor and Electrum wallets each provide message signing support for P2WPKH and P2SH-wrapped P2WPKH addresses. Yet both implementations were made independently and use slightly different protocols, so they’re unable to verify signatures produced by the other system. Additionally, the algorithms used by all wallets we’re aware of can’t be easily adapted to P2SH and P2WSH scripts used for multisig and other advanced encumbrances. That means message signing today is universally limited to users of single-sig addresses.

There is a proposed standard that should allow any address type or script to be used to create a signed message, BIP322. The protocol should even be forward compatible with future segwit versions, such as bip-taproot and bip-tapscript (with some unresolved limitations related to time locks). Unfortunately, even though the proposal was first made more than a year ago (see Newsletter #13), there’s no implementation for it—not even a proposed implementation under review.

This leaves users of segwit without the same level of message signing support available to users of legacy addresses, and it may represent a reason some users are unwilling to move to segwit addresses. The only solutions, besides wallets abandoning message signing support, are for wallet developers to agree on a standard and then widely implement it.

Survey of support issues

Originally published in Newsletter #55.

We have heard from wallet providers that a reason for their hesitation to default to receiving to bech32 addresses is concern that they’d see a significant increase in customer support requests. Despite this, some wallets already default to bech32 addresses and others plan to move to use them soon, such as Bitcoin Core.

We solicited input from a number of services including BitGo, BRD, Conio, Electrum, and Gemini regarding their customer support burden from use of bech32 addresses. Most services report minimal issues (“no support requests” and “there isn’t too much confusion”).

One service said, “So Bitcoin address-related customer support tickets increased 50%, but the absolute number of tickets is so small that not sure we can give too much significance. There have never been many tickets on this subject either before or after Bech32 so not sure this is an important point in making the argument for exchanges to make the switch. Instead, I might suggest focusing on fees, which really can add up if you are using an old wallet implementation.”

Electrum has seen some reports though, which are public, such as “strange addresses” and “Localbitcoins does not support sending to bech32”.

While not conclusive, it is reassuring that services opting to support receiving to bech32 addresses have not seen a negative impact on their customer support teams. The suggestion above to consider fee savings likely far outweighs concerns, and is consistent with Bitcoin Optech’s guidance. With few negative reports and significant potential fee savings for those wallets and services that support receiving to bech32 addresses, it may be time for more wallets to begin making bech32 their default address format. If that happens, it’ll be even more important for other wallets and services to support sending to bech32 addresses.

Fee savings in dollar terms

Originally published in Newsletter #56.

As of this writing, the price of bitcoin in national currencies has been rapidly increasing over the past few months. Significant changes to the bitcoin price are notable in the context of bech32 because transaction fees are paid in bitcoin terms rather than dollar terms. That means the real cost to send a transaction is expected to increase in correspondence with bitcoin prices even if feerates stay the same.

We’ve previously discussed how much users and services can save by switching to native segwit (bech32) addresses, but we’ve only described that in terms of vbyte and percentage savings. In this bech32 sending support section, we look at the savings in real terms.

The lowest practical fee is 0.00000001 BTC/vbyte. The highest typical fees seen to date were around 0.00001000 BTC/vbyte in December 2017 and January 2018. For that range, the following charts show the amount of money that may be saved for users of two common transaction templates, single-sig and 2-of-3 multisig:

For users of legacy transactions with other transaction templates, you may be able to learn roughly how much you will save in percentage terms by pasting in the txid of a typical transaction you sent into information sites such as an Esplora block explorer like Blockstream.info. You can multiply that percentage by the vbyte size of your transaction to see how many vbytes you’d save. Note that using a third-party service reveals to them that you have an interest in that transaction, possibly significantly reducing your privacy. You can privately get the approximate vbyte savings by examining a typical transaction of yours.² When you know how many vbytes you’ll save, you can calculate the amount of your savings in another currency by multiplying the saved vbytes by your expected feerate in BTC/vbyte and your expected price per bitcoin, e.g. saved_vbytes * feerate * price.

If users of native segwit begin to save tens or hundreds of dollars per transaction, we expect there to be increased competitive pressure for high-frequency spenders such as exchanges to migrate to only accepting deposits using bech32 addresses. Given that a very large percentage of daily Bitcoin transactions are deposits to exchanges, we would then expect wallets and services that don’t provide bech32 sending support to quickly fall out of favor with users.

Bech32 adoption rate

Originally published in Newsletter #57.

In percentage terms, the number of native segwit outputs—payments to bech32 addresses—has slightly declined over the past year (~50,000 blocks). The obvious explanation would be that people tried bech32 addresses, didn’t like them, and went back to using legacy addresses or P2SH-wrapped segwit addresses. If that’s the case, should we give up on bech32 addresses?

Note: all charts in this section use a simple moving average over 10,000 blocks.

Perhaps first we should investigate whether there’s an alternative explanation. Percentages are synthetic results—information derived from combining multiple other sources—so the first thing to look at is the raw data. The graph below shows the total number of outputs paid to various types of scripts over the past two years:

Contrary to the percentage data, we see the number of P2WPKH outputs slowly (but steadily) increasing. What we also see is that the number of other outputs is also increasing. This may become more clear if we stack the totals:

Now we can see that the average number of outputs is slightly higher today than it was at the previous peak of Bitcoin activity during late 2017 and early 2018. That makes sense—the more transactions that use segwit, the more block space there is available for other transactions. However, despite the overall increase, there are slightly fewer payment outputs (P2PKH, P2SH, and native segwit). The remaining outputs constitute a significant increase in the number of OP_RETURN (nulldata) scripts.

This growth in the overall number of outputs and the number of nulldata outputs explains why the percentage of bech32 outputs has been declining despite the absolute number increasing. It means that we shouldn’t give up on bech32. Indeed, the recent spike in overall outputs (and other recent data available elsewhere) may be a sign of upcoming feerate increases that will encourage more users and organizations to cut costs by switching to bech32.

BRD field report

Originally published in Newsletter #58.

The following case study contributed by Optech member company BRD describes what they learned implementing bech32 and other segwit technology for their wallet.

We began implementing bech32 support in BRD wallet in January 2018 with the addition of bech32 decoding and encoding support to breadwallet-core, an MIT-licensed cross-platform C library with no external dependencies. All of our software avoids third-party library dependencies as much as possible, with it currently using only Pieter Wuille’s libsecp256k1. Minimizing dependencies is typical for a high-security crypto project. For the bech32 implementation, we found that bech32’s BIP173 is pretty well documented, so there were no specific issues that were complex to deal with.

In March 2018, breadwallet-core was updated to automatically parse anything provided as a Bitcoin address to determine whether it was a legacy P2PKH, legacy P2SH, or segwit bech32 and to automatically generate the appropriate scriptPubKey in each case. This allowed BRD to begin sending to bech32 addresses. Finally in October 2018, we implemented full segwit support across the library backend and mobile app frontends, allowing users to begin receiving to bech32 addresses and making the default that all change addresses were now bech32.

One thing we never implemented was support for receiving to P2SH-wrapped segwit addresses, instead going straight to bech32. This was a deliberate design optimization to make the best use of the bloom filter mechanism BRD uses to scan for transactions affecting user wallets. To allow users to track when they’ve been paid, bloom filters are matched against each data element in a scriptPubKey. For a given public key, the data element in the scriptPubKey is identical for both legacy P2PKH and native segwit (bech32) P2WPKH. Here’s an example previously used by Optech:

• Legacy P2PKH scriptPubKey for address 1B6FkNg199ZbPJWG5zjEiDekrCc2P7MVyC:

  OP_DUP OP_HASH160 OP_PUSH20 6eafa604a503a0bb445ad1f6daa80f162b5605d6 OP_EQUALVERIFY OP_CHECKSIG

• Native segwit (bech32) P2WPKH scriptPubKey for address bc1qd6h6vp99qwstk3z668md42q0zc44vpwkk824zh:

  OP_0 OP_PUSH20 6eafa604a503a0bb445ad1f6daa80f162b5605d6

Because a bloom filter for a given element will match both P2PKH and P2WPKH addresses for the same public key, BRD is able to scan for either type of payment with zero additional overhead. It also makes the implementation much cleaner and doesn’t increase the resource use of public nodes that serve bloom filters. This may be a worthwhile optimization for wallets and services using other types of scanning as well, as it may produce less overhead than the separate HD derivation path recommended by BIP84.

ScriptPubKeys generated from bech32 addresses vary in length, affecting the amount of transaction fee that needs to be paid. Fee calculation on Bitcoin is hideous—feerates spike multiple orders of magnitude in a 24 hour period sometimes—but that was already true before segwit so we previously spent a lot of time on fee calculation and made it as flexible as possible. That means the variability in size of scriptPubKeys generated from bech32 addresses doesn’t change anything for BRD.

We want today’s app to be future-proof so the code supports sending to future segwit versions (see Optech’s description). That means, for example, BRD will support paying to taproot addresses automatically if Bitcoin users choose to make that soft fork change to the consensus rules.

Once real momentum is established and most other wallets and services support sending to bech32 addresses, BRD’s bech32 receiving support will be rolled out to our entire user base as the default setting. In preparation of this transition, it is important to get as many companies and services as possible to voluntarily start supporting bech32 sending capability. To help encourage adoption, we created the WhenSegwit website and became a member company of Optech. We hope that other wallets and services will make their own transitions to full segwit support soon while fees are still relatively low.

Same fee, faster confirmation

Originally published in Newsletter #59.

We’ve frequently mentioned the fee savings available to people spending segwit inputs, but we’ve never before mentioned that you don’t need to take advantage of the savings. If you want, you can pay the same fee you would’ve paid without segwit in order to possibly have your transaction confirm more quickly (all other things being equal). For some users, such as traders attempting arbitrage across exchanges, saving money may not be as important as faster confirmation for the same amount of money.

To examine this idea, let’s first generate a chart using Bitcoin Core’s fee estimates that illustrates the potential fee savings available to creators of typical single-sig, one-input, two-output transactions:

We see that, in general, it takes longer for a transaction to confirm the less you pay—but users of segwit can often pay less per transaction for the same amount of waiting. Now let’s simply re-arrange the axes for the same data:

For a given fee, it’s expected that users of segwit will sometimes wait less time for confirmation than legacy users, with native segwit users gaining the greatest advantage. In these estimates, the variation in confirmation speed for different transaction types all paying the same total fee can be more than 6 blocks (about an hour on average).

For users and organizations who have a fixed maximum price they’re willing to pay in fees per transaction, using segwit could significantly reduce confirmation time for their transactions during periods of high activity. Although this advantage only benefits people spending from bech32 and other segwit addresses, it’s another reason to expect people and organizations will increasingly request your software and services pay bech32 addresses in the near future.

Insights from segwit compatibility matrix

Originally published in Newsletter #60.

As of this writing, here are what we think are some of the most significant bech32-related insights we’ve gleaned from creating and reviewing the Compatibility Matrix.

• ● Most tools support paying bech32 addresses: 74% of the wallets and services surveyed support paying to segwit addresses. Although this isn’t the near-universal support we’d like to see, it may be enough support that we’ll soon see more wallets switching to bech32 receiving addresses by default.

• ● Supporting P2WPKH but not P2WSH: when we started testing various apps, we assumed “bech32 sending support” would be binary—either a tool supported it or not. However, one service we surveyed supports spending money to native segwit (bech32) P2WPKH addresses but not bech32 P2WSH addresses. This led us to tracking the two different segwit version 0 addresses separately. (If you’re a developer, please support sending to both address types.)

• ● Address input field length restrictions: some services might have supported sending to bech32 addresses, but when we attempted to enter a bech32 address, either it was rejected as being too long or the field simply refused to accept all the characters. (Reminder, BIP173 says this about the lengths of Bitcoin mainnet bech32 addresses: they “are always between 14 and 74 characters long [inclusive], and their length modulo 8 cannot be 0, 3, or 5.”)

• ● Screenshots of input fields: we documented the steps we took to try sending to bech32 addresses, which gives us a large collection of screenshots that UI designers can review for best practices when implementing their own bech32 sending support (or other features, such as RBF support).

• ● Lack of bech32 change address support: because sending to bech32 addresses is still not universally supported, it makes sense for segwit-compatible wallets to generate P2SH-wrapped segwit receiving addresses by default. However, many of these wallets also use P2SH-wrapped segwit addresses for receiving change sent from themselves to themselves. In some cases, this may have been done for privacy benefits (e.g. Bitcoin Core currently tries to match the type of change output to the type of payment output) but, in most cases, this seems like a missed opportunity for wallets to send change to their own bech32 addresses for increased fee savings.

Conclusion

Originally published in Newsletter #61.

After six months and over 10,000 words published, this is our last bech32 sending support section. Our goal was to gently persuade as many developers as possible to add support for paying bech32 addresses in their applications. We hoped that would make it easier for segwit-ready wallets to switch from using P2SH-wrapped segwit addresses by default to more efficient native segwit bech32 addresses.

By our efforts and by the efforts of many other Bitcoiners, we think we’re on the brink of success: 19 of the 23 popular wallets and services we’ve evaluated are ready to pay bech32 addresses and 4 already generate bech32 receiving addresses by default.

Every week, it’s looking more and more reasonable for wallets to switch to bech32 soon, and we expect to hear from an increasing number of developers that their next major release will default to bech32 receiving addresses. This will lower the transaction fees for the users of that software and make more block space available to all Bitcoin users, helping to keep fees lower for everyone a little bit longer.

Even though this series has ended, we’ll continue to update the segwit section of the compatibility matrix and report on notable bech32 developments in the other parts of the weekly newsletter. We thank all of you for reading this series and for helping to improve Bitcoin scalability one wallet and service at a time.

Footnotes

1. Bech32 addresses have three parts, a Human Readable Prefix (HRP) such as bc, a separator (always a 1), and a data part. The separator and the data part are guaranteed to be part of QR code’s uppercase alphanumeric set, but the range of characters allowed in the HRP according to BIP173 includes punctuation that isn’t part of that uppercase alphanumeric set. Specifically, the following characters are allowed in bech32 HRPs but are not part of the QR code uppercase alphanumeric set:

   !"#&'()';<=>?@[\]^_`{|}~
   

   None of the bech32 HRPs used in Bitcoin (bc, tb, bcrt) use any of these characters, and neither does any other application as far as we know. However, you may not want to make any assumptions in your code about uppercase always providing smaller QR codes for non-Bitcoin bech32 addresses. ↩

2. 1. Use a tool to parse the transaction. Bitcoin Core comes bundled with a tool called bitcoin-tx that can do this for you. Run bitcoin-tx -json <hex_serialized_tx>

   2. Add up the total size of all scriptSigs. bitcoin-tx outputs the hex of each scriptSig. Halve the length of this hex string to get the size of the scriptSig.

   3. Multiply the size of scriptSigs that value by 0.75 to get vbytes saved.

   ↩