Compact block filters
Also covering BIP157, BIP158, and Neutrino protocol
Compact block filters are a condensed representation of the contents of a block that allow wallets to determine whether the block contains any transactions involving the user’s keys.
A full node uses BIP158 to create a Golomb-Rice Coded Set (GCS) of the data from each block in the block chain. The GCSs (called filters) are then distributed to wallets (such as via the P2P protocol as described in BIP157), allowing them to search for any matches to their scripts. If a match is found, the wallet can then download the corresponding block to access any relevant transactions.
The GCS mechanism guarantees that a wallet following the protocol will find any transactions matching its scripts, but it may also find some false-positive matches for which it will need to download and scan the block despite the block not containing any transactions relevant to the wallet.
The BIP157/158 protocol is sometimes incorrectly called “Neutrino” after the wallet library developed to use the protocol. It’s one of several methods that lightweight clients can use to acquire data about their wallet transactions. Compared to BIP37 bloom filters, it offers more privacy against spy nodes and less risk of attack against honest nodes. Compared to address-indexed servers (such as Electrum-style servers), it also provides more privacy and requires less server storage and CPU. However, the BIP157/158 does consume significantly more bandwidth in the normal case than either of those other protocols.
Primary code and documentation
Optech newsletter and website mentions
- Bitcoin Core #25957 improves the performance of wallet rescans using local block filters
- Bitcoin Core #23549 adds
scanblocksRPC to scan local compact block filters
- LDK #1706 adds support for using compact block filters for downloading confirmed transactions
- Bitcoin Core #17631 adds new REST endpoint for compact block filters
- Discussion of additional compact block filter verification
- Question about high bandwidth requirements for serving BIP157 filters
- Bitcoin Core #15946 allows retaining filters on a pruned node
- New Rust-Language light client released using compact block filters
- Question: how do clients using BIP157 receive unconfirmed transactions?
- Bitcoin Core 0.21.0 released with support for serving compact block filters
- 2020 year in review: compact block filters
- Bitcoin Core #19070 allows advertising support for serving BIP157 filters
- Bitcoin Core #19010 & #19044 add additional messages from BIP157
- Bitcoin Core #18877 adds
- Bitcoin Core 0.19 released with RPC support for BIP158 block filters
- Maximum number of block filters per request increased from 100 to 1,000
- BIP157 bandwidth higher than BIP37 bloom filters
- Basic BIP158 support merged into Bitcoin Core
- Functions for generating BIP158 filters added to Bitcoin Core
- Discussion of what data should be included in BIP158 filters
Coinswap Next Topic:
Compact block relay