Add scanblocks rpc, questions about RpcBlockchain

[chrisguida]

I'm adding a new RPC to the rust-bitcoincore-rpc crate, and I'd love to know how I can use this in BDK.

The reason for this is that I'm rewriting my plugin, smaug, to use an RPC backend instead of an Esplora backend, since (to my knowledge) there are no public mainnet Esplora instances that don't ratelimit the BDK wallet scanning, and presumably Esplora is also not great for privacy.

The idea with scanblocks is that you give it one or more wallet descriptors and it returns a list of relevant block hashes after a few minutes. It's much better than any other wallet rescanning method I'm aware of, but I need to be able to easily get the blockhashes that result from this call and turn around and ask bitcoind for a full block for each block hash, then scan each block against my BDK Wallet to see which transactions belong to me.

There don't seem to be any examples for how to use the RpcBlockchain struct in such a custom way, though perhaps I'm not looking in the right place. It seems to simply store the wallet inside bitcoind, is that correct?

I currently have this Esplora code; how can I adapt this to use RpcBlockchain instead of EsploraBlockchain? Does it work in an async context? (ie does it work with cln-plugin?) And can I use a custom rust-bitcoincore-rpc crate (ie my PR listed above) as an upstream to bdk?

If not, is this even the right approach?

I'd love some guidance here; I'm a bit lost as to where my efforts are best spent. I'm completely happy to contribute any missing pieces :)

[notmandatory]

One of the reasons BDK based wallets are getting rate limited is because they do a full scan for used addresses every time. The new BDK 1.0 API will give an option to do a "full_scan" or a simple "sync". The simple "sync" only checks addresses (SPKs, script pub keys) that the wallet has given out, and has the option for a super fast scan to only check SPKs that were given out but haven't been used yet. Check out bitcoindevkit/bdk#1194 see a bit more about how this will likely work.

[notmandatory]

As for the RPC client, I'm not sure yet if we're going to call what it does scanning or syncing, but I think it will only do a sync since it doesn't need to do the stopgap based full scan, but anyway it will be able to efficiently sync the transactions for wallets will a large number of SPKs to track.

[chrisguida]

Ok, thanks! I'm just looking for some example of the RPC client that works now. I can update the references once BDK 1.0 is released.

I'm thinking scanblocks to get all relevant blockhashes, then some kind of Wallet.scan_block_for_my_txs(block) for each block after retrieving them from the RPC?

[chrisguida]

Made a little example of how this would work: vladimirfomene/bdk#2

[notmandatory]

I put this in the beta.0 milestone to make sure any questions here about how to use the RPC blockchain client are resolved.