This week’s newsletter includes the usual dashboard and action items, a
feature article by developer Anthony Towns about the consolidation of
four million UTXOs at Xapo, news about possible upgrades to Bitcoin’s
script system, links to a few highly-voted questions and answers on
the Bitcoin StackExchange, and some notable commits in the development
branches of the Bitcoin Core, Lightning Network Daemon (LND), and
Bitcoin Core 0.16.2 released: a minor release that brings bug
fixes and minor improvements. If you use the abandontransaction or verifytxoutproof RPCs,
you should particularly consider upgrading. Otherwise, we recommend
you review the release notes for other changes
that may affect your operation and upgrade when convenient.
Fees still low: hash rate increased difficulty by more than 14%
in the 2,016-block retarget period ending Sunday, giving an average
time between blocks of 8 minutes and 41 seconds. This helped to absorb the
increased transaction load from the past week’s price
speculation. Immediately following a difficulty retarget, the average
time between blocks is restored to 10 minutes.
As we transition away from the normal weekend transaction lull into the
new week, there’s the possibility for a rapid increase in estimated
transaction fees. We recommend being careful sending large low-fee
transactions such as consolidations until closer to the weekend when
transaction volume begins to taper off again.
Field Report: Consolidation of 4 Million UTXOs at Xapo
As mentioned in newsletter #3, the past few months of low
transaction fees makes it a great time to do UTXO consolidation!
Consolidation has been one of a variety of activities Xapo has been
undertaking to be prepared for the next time fees spike like they did in the
last few months of 2017.
Plot of total Bitcoin UXTOs, January - July 2018,
The idea behind UTXO consolidation is essentially this: when your average
outgoing payment is larger than your average incoming payment (or when they’re
the same, but you’re batching outgoing payments), you’ll often have to combine
many UTXOs in order to fund an outgoing transaction, which increases the size
of your transactions and hence the fees you pay. By consolidating UTXOs in
advance, you can combine inputs ahead of time, giving you more
control over when most of those costs are incurred. If you can do it when
fees are low, that lets you reduce those costs pretty substantially.
For example, if you would have spent a dozen 2-of-3 multisig inputs at 100 s/B
(satoshis per byte), that would cost around 360,000 satoshis; while if you
consolidated those inputs beforehand at 2 s/B, and then spent the single
consolidated input later at 100 s/B, your total cost for the two transactions
is only about 41,000 satoshis: i.e. 87% less paid in fees. And if fees don’t
rise the risk isn’t huge: if fees just sat at 2 s/B, you’d be spending 7,900
satoshis across two transactions if you consolidated, rather than spending
7,200 satoshis in a single transaction if you did nothing.
Consolidation also gives an opportunity to update the addresses you use for
your UTXOs, for example to roll keys over, switch to multisig, or switch to
segwit or bech32 addresses. And reducing the number of UTXOs makes it easier to
run a full node too, marginally improving Bitcoin’s decentralisation and
overall security, which is always nice.
Of course, one thing you really don’t want to have happen is for your
consolidation transactions to somehow fill up the blockchain and cause fees to
immediately start rising! There are two metrics to watch to avoid this risk:
one is whether the mempool is full (which causes the minimum
acceptable fee to rise), and the other is how much empty space there has been
in recent blocks (which gives an indication of whether miners will accept more
transactions at the minimum fee). Both these metrics have been very promising
most of the time over the past few months: the mempool has regularly been close
to empty, meaning the transactions paying as little as 1 s/B have been
propagated to miners; and many blocks have not been full, meaning cheap
consolidation transactions will get mined reasonably quickly rather than
creating a backlog that will cause fees to rise.
The approach we took to actually doing the consolidation was to have a script
that would select groups of small UTXOs and create a consolidation transaction
spending them to a single pool address at a fee rate of 1.01 satoshis per byte.
The script gradually feeds consolidation transactions into the network, so it
doesn’t cause too large a spike in the mempool, and perhaps more importantly so
we don’t risk having our transactions get dropped because they have low fees
and the mempool has filled up. We triggered this manually when we were
comfortable it wouldn’t interfere with our operations, and when there didn’t seem
to be much load on the Bitcoin network in general.
All in all, this has worked out pretty well; we’ve reduced our UTXO count by
something like 4 million UTXOs this year, and aside from
some concernedredditors, the cost to the network
as a whole has been minimal, as has the cost to us.
“Improvements in the Bitcoin Scripting Language” by Pieter
Wuille: a talk last week giving a high-level overview of several
possible near-term improvements to Bitcoin. We highly recommend
watching the video, viewing the slides,
or reading the transcript (with references) by
Bryan Bishop—but if you’re too busy, here’s Wuille’s conclusion: “my
initial focus here is Schnorr signatures and taproot. The reason for
this focus is that the ability to make any input and output in the
cooperative case look identical is an enormous win for how script
“Schnorr is necessary for this because without it we cannot encode
multiple parties into a single key. Having multiple branches in
there is a relatively simple change.
“If you look at the consensus changes necessary for these things,
it’s really remarkably small, dozens of lines of code. It looks like
a lot of the complexity is in explaining why these things are useful
and how to use them and not so much in the impact on the consensus
“Things like aggregation, I think, are something that can be
done after we have explored various options for structural
improvements to the scripting language, once it’s clear around what
the structuring should be, because we will probably learn from the
deployments how these things get used in practice. That’s what I’m
working on with a number of collaborators and we’ll hopefully be
proposing something soon, and that’s the end of my talk.”
Selected Q&A from Bitcoin StackExchange
Bitcoin StackExchange is one of the first places Optech
contributors look for answers to their questions—or when we have a
few spare moments of time to help answer other people’s questions. In
this new monthly feature, we highlight some of the top voted questions
and answers made there in the past month.
Schnorr versus ECDSA: in this answer, Bitcoin protocol
developer Pieter Wuille explains some of the chief advantages of the
Schnorr signature scheme over Bitcoin’s current ECDSA signature
The maximum size of a Bitcoin DER-encoded signature is…: in this answer, Pieter Wuille provides the math for calculating
the size of a Bitcoin signature. As mentioned in Newsletter #3,
the maximum size using a regular wallet is 72 bytes—but Wuille
explains how you can create a non-standard transaction with a 73 byte
signature and why you might think you saw a signature that was 74 or
even 75 bytes.
If you can use almost any opcode in P2SH, why can’t you use them in
scriptPubKeys?: in this answer, Bitcoin technical writer
David A. Harding explains why early versions of Bitcoin restricted the
types of transactions that could be sent to “standard transactions”
and why most of those restrictions are still in place even though
almost all opcodes are available for standard use now via P2SH and
A quick look at recent merges and commits made in various open source
Bitcoin Core #12257: if you start Bitcoin Core with the
optional flag -avoidpartialspends, the wallet will by default spend
all outputs received to the same address whenever any one of them
would be spent. This prevents two outputs to the same address from being spent
in separate transactions, which is a common way address reuse reduces
privacy. The downside is that it may make transactions larger than
the smallest they need to be. Bitcoin businesses using Bitcoin Core’s
built-in wallet who don’t need the extra privacy may still want to
toggle this flag on when fees are low for automatic consolidation of
LND #1617: updates estimates for the size of on-chain
transactions to prevent transactions from accidentally paying too low
of a fee and getting stuck. This commit may be interesting for
anyone wondering about the size of the transactions (and parts of
transactions) produced in the current protocol.
LND #1531: the daemon no longer looks for spends in the memory
pool—it waits for them to be a confirmed part of a block first.
This allows the same code to work on full nodes like Bitcoin Core and
btcd as well as on BIP157-based lightweight clients that don’t have
access to unconfirmed transactions. This is part of the ongoing effort
to help people without full nodes use LN.
In several commits, C-lightning developers have mostly completed
the transition from handling peer-related functions in gossipd to
handling them in channeld or connectd as appropriate.
C-lightning has improved its secret handling so that secrets and
signatures are always generated and stored by a separate daemon than the
parts of the system directly connected to the network.