Local regtest fixup: make lnd containers sync to chain faster

[sutt]

Describe the problem you're trying to solve

Not a bug, but a worthy improvement.

Base on logs, there's wildly different sync times between the three lnd containers on regtest init:

- lnd:          0m:10s
- sn_lnd:       5m:40s
- router_lnd:   1m:30s

Given that they are all the same image (currently polarlightning/lnd:0.18.0-beta) ,connected to the same bitcoind, and configured relatively similar, why is this?

These are time to initiate the sync, then they all sync in 10 seconds. The laggards continue to throw the error before initiating the sync:
- [ERR] LNWL: Unable to synchronize wallet to chain, trying again in 5s: -8: Block height out of range

This appears to be an intermittent problem on LND reported here: lightningnetwork/lnd#8468. Most recent proposed fix is just upgrade to v19.

Note: this appears to occurs on regtest init, not start from already sync'd. I think this is because once sync is initialized it gets cached in a db, and the blocks are produced less frequently.

Describe the solution you'd like

Ideally all three lnd containers chain sync in under 30 seconds.

I think this would be helpful for writing tests on regtest functionality; where 30 second setup is not terrible but 6 minute setup is undesirable.

I'm going to try to monkey around and find what's making some containers so fast and others so slow.

Describe alternatives you've considered

No response

Additional context

This was also noted by ek in #2179:

I've had this happen before, but not sure how I fixed it.
update: The error fixes itself after a few minutes:

[sutt]

Ahh, I've found the problem, the minimum block_height gets cached in the wallet.db somewhere.

Before minecron mines enough blocks to exceed this minimum, the sync will not initiate.

Thus the trick is to create the wallet.db before minecron starts mining.

I'll send in a PR that fixes the startup time for router_lnd and sn_lnd.

[ekzyis]

Wow, nice find, this was really annoying