This week’s newsletter includes an announcement of the 2019 Chaincode
Residency program, summarizes a few talks from the Stanford Blockchain
Conference, and provides the usual list of notable code changes in
popular Bitcoin infrastructure projects.
Apply to the Chaincode Residency: Bitcoin Optech encourages any
engineer who is interested in spending summer contributing to open source
Bitcoin and Lightning projects to apply to the Chaincode Residency. Full
details of the residency are in the News section below.
2019 Chaincode Residency: Chaincode Labs opened applications for its
fourth residency program to be held in New York over
summer 2019. The program combines a 3 week seminar and discussion series
covering Bitcoin and Lightning protocol development with a two month period for
working on an open source Bitcoin or Lightning project under the guidance of an
established protocol developer. The list of confirmed speakers and
mentors includes some of the most prolific contributors to
Bitcoin and Lightning. The previous Chaincode Residency (focused on Lightning
Network applications) was covered in Newsletter #21.
Chaincode is inviting developers who want to contribute to open source
Bitcoin and Lightning protocol projects to apply to the
residency. Applicants from all backgrounds are welcomed,
and Chaincode will cover travel and accommodation costs and provide a stipend
to support living expenses.
Notable talks from the Stanford Blockchain Conference
The third edition of this annual conference (formerly named BPASE)
included more than two dozen presentations across three days. We found
the following topics especially interesting from a Bitcoin perspective,
although we encourage anyone who wants to learn more to look at the
transcripts provided by Bryan Bishop or the videos provided by
the organizers (day 1, day 2, day 3).
Accumulators for blockchains presented by Benedikt Bünz
(transcript, video). Bitcoin
full nodes maintain a ledger (called the UTXO set) that stores the
ownership information for every currently-spendable grouping of
bitcoins. Currently, that ledger contains over 50 million entries
and uses about three gigabytes of disk space. This allows nodes who
receive a transaction to ensure the bitcoins being spent exist in
the UTXO set, retrieve the ownership information for those bitcoins,
and verify that information against the signature and other witness
data provided in the transaction.
But what if we also asked the spender to provide a copy of the
ownership information in their transaction along with a
cryptographic proof that the the information is actually in the UTXO
set? In that case, we wouldn’t need to store the whole set, we’d
only need to store a commitment to a set we knew was accurate and
that could be referenced by accurate proofs. RSA accumulators are
one idea (among several others) for how to create that commitment
and its related proofs. Using an accumulator, the size of the UTXO
commitment that nodes would store on disk would be tiny compared to
the full state. Transactions would increase in size due to needing
to provide ownership data and a proof that they were part of the
current UTXO set, but the size increase would be modest (assuming
current typical transactions, about 70 bytes of ownership
information per input plus a proof that could be aggregated down to
about 325 bytes per block).
Other considerations affect the suitability of accumulators to
the task, including being relatively new to cryptography,
requiring either a well-studied trusted setup or a
more novel untrusted setup, and potentially making blocks take
longer to verify than the current system given that accumulator
verification is about 100x slower than alternative systems using
merkle trees. In a new development compared to a previous
version of this talk given at Scaling Bitcoin 2018, the speaker
describes a potential major speedup to practical verification.
In summary, RSA accumulators remain an interesting area of investigation
into how to reduce full node requirements for storing and quickly
accessing the UTXO set. This may not be particularly important now,
when the UTXO set is relatively small and fast, but it could make it
easier to support initiatives that would change how people use the
UTXO set in the future. For example:
If accumulator-based proofs can indeed be verified quickly, they
could allow the UTXO set size to grow considerably (perhaps
because of a block size increase) while still ensuring that miners
can verify transaction inputs quickly enough to minimize the use of
spy mining or the production of stale blocks.
Software that uses trusted UTXO sets with newly-started nodes to
avoid the cost and delay of downloading and verifying the full
block chain (an option some software already
provides) could reduce those costs and delays even further by
replacing the multi-gigabyte UTXO set with an accumulator a
million times smaller.
be possible for eager experimenters to explore the use of
accumulators in Bitcoin without changing any consensus rules, such
as Tadge Dryja has been doing with his similar Utreexo system
based on merkle trees.
Miniscript presented by Pieter Wuille (transcript, video, slides).
Imagine you had a Bitcoin script that gave control over your bitcoins
to your lawyer a year after you last moved them, in case he needed to
distribute them to your heirs. That’s an easy script to write. But
what if someone then asked you to join a 3-of-4 multisig contract
where you’d be one of parties holding some funds. How hard would it
be for you to insert your existing policy into their generic multisig
contract and be sure you haven’t broken anything? That’s the question
asked by this speaker, and his answer is the composable policy
Miniscript is a subset of the full Bitcoin Script language that
focuses on just a few features such signatures, times, and
hashes plus operators for combining them, such as and, or, or
threshold. It’s compact, easy to read, and will compile down to
the most efficient script implementing a given policy. From an
existing script compatible with its operations, it will also reverse
it back into a policy for easy review. By design, it uses a
similar vocabulary to the output script descriptors Wuille has
been implementing in Bitcoin Core and it can help the updater or
finalizer in a BIP174 PSBT workflow figure out who needs to sign
next or whether all data for finalizing the script has been
Looking at the problem posed in the introduction paragraph, we can
define the example personal spending policy as either you providing a
signature for your compressed pubkey, pk(C), or your lawyer
waiting for a year, time(<seconds>), and then providing a
signature for his own compressed pubkey. We combine these branches
with an asymmetric “or,” aor, to reduce the witness data required
when following the first branch.
We can define the generic 3-of-4 multisig policy similarly using
compressed pubkeys (C):
A functionally equivalent policy that allows more flexibility would
use the threshold operation:
This allows you to just replace one of the public keys with your personal
A final benefit of miniscript is that it should allow a policy
written today to be automatically translated into a structure that
makes optimal use of MAST, taproot, or other possible Bitcoin
protocol upgrades. That means, as the Bitcoin protocol advances,
the user or developer who invested time into crafting a policy won’t
have to redo their work in order to continue using it with newer
miniscript compiler, and he and his collaborators
also have a Rust language version of the compiler they plan to
release as open source soon.
Probabilistic Bitcoin soft forks (sporks) presented by Jeremy
Rubin (transcript, video). By March 2017,
almost all Bitcoin nodes were ready to begin enforcing the segwit soft
fork but miners seemed unwilling to send the activation signal. This
created confusion: do miners get to veto protocol upgrades? If they
do, is segwit dead? If they don’t, what do users do to override them?
Ultimately the situation was resolved, but it was a mess that many
would prefer not to repeat.
The speaker identifies the root cause of the problem as miners being
able to delay activation at no cost to themselves. He proposes a
solution: use the randomness remaining in a block header hash to
determine whether or not a block signals for activation. For
example, we’d choose a target that only 1 header hash out of 26,000
would match. A block matching that target would be produced once every
six months on average, although nobody would know exactly when
(about 10% of the time, it’d be within 3 weeks; but another 10% of
the time, it’d take more than a year).
Miners would have no control over whether or not their block
signaled for activation, but they would have control over whether
they published that block. A miner who refused to publish his own
block if it signaled for activation would lose the income from that
block but would successfully prevent the fork from activating at
least until the next signaling block was produced (which could be
tomorrow or could be two years later). This gives miners a real
chance to hold back a change but only by sacrificing real income.
The end of the talk suggests some variations on the method with
The idea needs to be analyzed for possible problems, but it does
provide an interesting alternative to BIP9-style miner-activated
soft forks (MASFs) and BIP8-style user-activated soft forks (UASFs).
At the conclusion of his talk, this speaker also announced that the
next Scaling Bitcoin conference and EdgeDev++ training sessions will
be later in 2019 in Tel Aviv, Israel.
Bitcoin Core #14929 allows peers that your node automatically
disconnected for misbehavior (e.g. sending invalid data) to reconnect
to your node if you have unused incoming connection slots. If your
slots fill up, the misbehaving nodes are disconnected to make room for
nodes without a history of problems (unless the misbehaving node helps
your node in some other way, such as by connecting to a part of the
Internet from which you don’t have many other peers). Previously,
Bitcoin Core banned the IP addresses of misbehaving peers for a period
of time (default of 1 day); this was easily circumvented by attackers
with multiple IP addresses. This solution gives those peers a chance
to be useful but provides priority to potentially more helpful peers.
If you manually ban a peer, such as by using the setban RPC,
connections from that peer will still be rejected.
Bitcoin Core #13926 adds a new bitcoin-wallet tool to the
executables Bitcoin Core provides. Without using RPCs or any network access,
this tool can currently create a new wallet file or display some basic
information about an existing wallet, such as whether the wallet is
encrypted, whether it uses an HD seed, how many transactions it
contains, and how many address book entries it has. This helps people
who have a wallet file that hasn’t been synced to the most recent chain
tip; they can do a quick inspection on the wallet to see if it’s
interesting before they perform the lengthy rescan necessary to import
it. Developers plan to add more features to the tool in the future.
Bitcoin Core #15159 changes the getrawtransaction RPC so that it
will now only return transactions in the mempool by default. If you
have enabled the optional transaction index (txindex), it will also
return confirmed transactions as well. Prior to this change, even
if you didn’t have the txindex enabled, it would return confirmed
transactions if they hadn’t yet had all their outputs spent. This
previous behavior confused users: the call would work on some
confirmed transactions but not others, and sometimes transactions that
previously worked would suddenly stop working. This change makes the
RPC act more consistently (although, of course, mempools vary between
nodes and change over time).
LND #2538 increases the default time between sending updates about
what public nodes exist on the network from 30 seconds to 90 seconds.
This slows down propagation on the network, which has grown hugely in
size, in order to conserve bandwidth. Separately from this PR, LN
protocol devs are preparing changes to the gossip protocol to be more
bandwidth efficient, although a lower update frequency will still save
bandwidth there as well. (See also C-Lightning #2297 for the fix
this week to a bug some nodes were encountering because the volume of
gossip they received was so large.)
LND #2554 deprecates the IncorrectHtlcAmount onion error
in favor of the UnknownPaymentHash error that now includes the
amount of the failed payment. LND will still handle the old error
code but it will no longer generate it.
LND #2500 disconnects any peers that don’t support the Data Loss
Protection (DLP) protocol. This ensures that LND’s new backup format
will be usable. See the notable commits section and footnote from
last week’s newsletter for information about the new
backup protocol and the existing DLP protocol.