Skip to content

@rustyrussell: Protocol-triggered unilateral closure due to deadline hit on HTLC with our revoke_and_ack not known to have been received #8218

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
whitslack opened this issue Apr 7, 2025 · 3 comments

Comments

@whitslack
Copy link
Collaborator

whitslack commented Apr 7, 2025

@rustyrussell: What follows is a detailed analysis of the failure of a very active channel due to the peer's neglect to remove an HTLC before its deadline. The HTLC in question had reached the SENT_ADD_ACK_REVOCATION state just prior to the peer connection being lost, and we didn't receive any messages from the peer that would indicate that it had received our revoke_and_ack (or any message we sent subsequently), so it is possible that it did not. The peer is known to have been running LND 0.18.5.

Rusty Russell: This is supposed to be handled by the counters in the reestablish message: you can tell the peer didn't get your revoke_and_ack and thus retransmit.

I am sure that much of the following discussion will be superfluous, but I want to give context to show that the channel was working correctly and to highlight the overlapped nature of the HTLCs, which hopefully may lend a clue as to why the peer evidently didn't believe it needed to remove one of our HTLCs before its deadline.

Overview

  1. HTLC 35768: SENT_ADD_ACK_REVOCATIONRCVD_REMOVE_HTLC
  2. HTLC 35768: RCVD_REMOVE_HTLCRCVD_REMOVE_COMMIT
  3. HTLC 35768: RCVD_REMOVE_COMMITSENT_REMOVE_REVOCATION
  4. HTLC 35768: SENT_REMOVE_REVOCATIONSENT_REMOVE_ACK_COMMIT
  5. HTLC 35769: SENT_ADD_ACK_REVOCATIONRCVD_REMOVE_HTLC
  6. HTLC 35771: (new) SENT_ADD_HTLC
  7. HTLC 35769: RCVD_REMOVE_HTLCRCVD_REMOVE_COMMIT
  8. HTLC 35769: RCVD_REMOVE_COMMITSENT_REMOVE_REVOCATION
  9. HTLC 35772: (new) SENT_ADD_HTLC
  10. HTLC 35768: SENT_REMOVE_ACK_COMMITRCVD_REMOVE_ACK_REVOCATION
  11. HTLC 35769: SENT_REMOVE_REVOCATIONSENT_REMOVE_ACK_COMMIT
    HTLC 35771: SENT_ADD_HTLCSENT_ADD_COMMIT
    HTLC 35772: SENT_ADD_HTLCSENT_ADD_COMMIT
  12. HTLC 35769: SENT_REMOVE_ACK_COMMITRCVD_REMOVE_ACK_REVOCATION
  13. HTLC 35771: RCVD_ADD_REVOCATIONRCVD_ADD_ACK_COMMIT
    HTLC 35772: RCVD_ADD_REVOCATIONRCVD_ADD_ACK_COMMIT
  14. HTLC 35771: RCVD_ADD_ACK_COMMITSENT_ADD_ACK_REVOCATION
    HTLC 35772: RCVD_ADD_ACK_COMMITSENT_ADD_ACK_REVOCATION
  15. HTLC 35770: SENT_ADD_ACK_REVOCATIONRCVD_REMOVE_HTLC
  16. HTLC 35773: (new) SENT_ADD_HTLC
  17. HTLC 35773: SENT_ADD_HTLCSENT_ADD_COMMIT
  18. HTLC 35770: RCVD_REMOVE_HTLCRCVD_REMOVE_COMMIT
  19. HTLC 35770: RCVD_REMOVE_COMMITSENT_REMOVE_REVOCATION
  20. HTLC 35771: SENT_ADD_ACK_REVOCATIONRCVD_REMOVE_HTLC
  21. HTLC 35773: SENT_ADD_COMMITRCVD_ADD_REVOCATION
  22. HTLC 35770: SENT_REMOVE_REVOCATIONSENT_REMOVE_ACK_COMMIT
  23. HTLC 35771: RCVD_REMOVE_HTLCRCVD_REMOVE_COMMIT
    HTLC 35773: RCVD_ADD_REVOCATIONRCVD_ADD_ACK_COMMIT
  24. HTLC 35771: RCVD_REMOVE_COMMITSENT_REMOVE_REVOCATION
    HTLC 35773: RCVD_ADD_ACK_COMMITSENT_ADD_ACK_REVOCATION
  25. HTLC 35770: SENT_REMOVE_ACK_COMMITRCVD_REMOVE_ACK_REVOCATION
  26. HTLC 35772: SENT_ADD_ACK_REVOCATIONRCVD_REMOVE_HTLC
  27. HTLC 35771: SENT_REMOVE_REVOCATIONSENT_REMOVE_ACK_COMMIT
  28. HTLC 35774: (new) SENT_ADD_HTLC
  29. HTLC 35772: RCVD_REMOVE_HTLCSENT_ADD_ACK_REVOCATION (reverted)
    HTLC 35774: (dropped)
  30. HTLC 35774: (new) SENT_ADD_HTLC
  31. HTLC 35774: (dropped)
  32. HTLC 35774: (new) SENT_ADD_HTLC
  33. HTLC 35772: SENT_ADD_ACK_REVOCATIONRCVD_REMOVE_HTLC
  34. HTLC 35772: RCVD_REMOVE_HTLCSENT_ADD_ACK_REVOCATION (reverted)
    HTLC 35774: (dropped)
  35. HTLC 35772: SENT_ADD_ACK_REVOCATIONRCVD_REMOVE_HTLC
  36. HTLC 35772: RCVD_REMOVE_HTLCRCVD_REMOVE_COMMIT
  37. HTLC 35772: RCVD_REMOVE_COMMITSENT_REMOVE_REVOCATION
  38. HTLC 35771: SENT_REMOVE_ACK_COMMITRCVD_REMOVE_ACK_REVOCATION
  39. HTLC 35772: SENT_REMOVE_REVOCATIONSENT_REMOVE_ACK_COMMIT
  40. HTLC 35772: SENT_REMOVE_ACK_COMMITRCVD_REMOVE_ACK_REVOCATION
  41. HTLC 35773: (deadline)

(Detailed analysis of the above to follow in comments due to GitHub's post length limit.)

@whitslack
Copy link
Collaborator Author

Detailed Event Analysis

  1. The peer fails our HTLC 35768, moving it into the RCVD_REMOVE_HTLC state:
2025-03-21T04:11:48.884Z DEBUG   03…4c-channeld-chan#255714: peer_in WIRE_UPDATE_FAIL_HTLC
2025-03-21T04:11:48.884Z DEBUG   03…4c-channeld-chan#255714: FAIL:: HTLC LOCAL 35768 = RCVD_REMOVE_HTLC/SENT_REMOVE_HTLC 
2025-03-21T04:11:48.894Z DEBUG   03…4c-channeld-chan#255714: Trying commit
2025-03-21T04:11:48.894Z DEBUG   03…4c-channeld-chan#255714: Can't send commit: nothing to send, feechange not wanted ({ RCVD_ADD_ACK_REVOCATION:864 }) blockheight not wanted ({ RCVD_ADD_ACK_REVOCATION:0 })

Now we are expecting to receive a new commitment from the peer.


  1. We receive a new commitment from the peer, moving our HTLC 35768 into the RCVD_REMOVE_COMMIT state:
2025-03-21T04:11:58.592Z DEBUG   03…4c-channeld-chan#255714: peer_in WIRE_COMMITMENT_SIGNED
2025-03-21T04:11:58.592Z DEBUG   03…4c-channeld-chan#255714: handle_peer_commit_sig(splice: 0, remote_splice: 0, index: 97031)
2025-03-21T04:11:58.592Z DEBUG   03…4c-channeld-chan#255714: Received commit
2025-03-21T04:11:58.593Z DEBUG   03…4c-channeld-chan#255714: htlc 35768: RCVD_REMOVE_HTLC->RCVD_REMOVE_COMMIT
2025-03-21T04:11:58.593Z DEBUG   03…4c-channeld-chan#255714: htlc added LOCAL: local 1270596398 remote 229569507
2025-03-21T04:11:58.593Z DEBUG   03…4c-channeld-chan#255714: -> local 1470876853 remote 229569507
2025-03-21T04:11:58.593Z DEBUG   03…4c-channeld-chan#255714: rcvd_commit: HTLC LOCAL 35768 = RCVD_REMOVE_COMMIT/SENT_REMOVE_COMMIT FAILED
2025-03-21T04:11:58.593Z DEBUG   03…4c-channeld-chan#255714: Feerates are 864/864
2025-03-21T04:11:58.593Z DEBUG   03…4c-channeld-chan#255714: Committed 3, removing 0, adding 0
2025-03-21T04:11:58.593Z DEBUG   03…4c-channeld-chan#255714: We need 22076sat at feerate 864 for 3 untrimmed htlcs: we have 229569507msat/229569507msat (will have 229569507msat)
2025-03-21T04:11:58.593Z DEBUG   03…4c-channeld-chan#255714: Derived key 02…28 from basepoint 03…93, point 02…e2
2025-03-21T04:11:58.594Z DEBUG   03…4c-channeld-chan#255714: Received commit_sig with 3 htlc sigs
2025-03-21T04:11:58.595Z DEBUG   03…4c-hsmd: Got WIRE_HSMD_VALIDATE_COMMITMENT_TX

Now we are expected to send a revoke_and_ack and then our own new commitment.

  1. We send the expected revoke_and_ack, moving our HTLC 35768 into the SENT_REMOVE_REVOCATION state:
2025-03-21T04:11:58.595Z DEBUG   03…4c-channeld-chan#255714: Sending revoke_and_ack
2025-03-21T04:11:58.596Z DEBUG   03…4c-channeld-chan#255714: htlc 35768: RCVD_REMOVE_COMMIT->SENT_REMOVE_REVOCATION
2025-03-21T04:11:58.596Z DEBUG   03…4c-channeld-chan#255714: sending_revoke_and_ack: HTLC LOCAL 35768 = SENT_REMOVE_REVOCATION/RCVD_REMOVE_REVOCATION FAILED
2025-03-21T04:11:58.596Z DEBUG   03…4c-channeld-chan#255714: revoke_and_ack made pending: commit timer
2025-03-21T04:11:58.596Z DEBUG   03…4c-channeld-chan#255714: Sending master 1021
2025-03-21T04:11:58.596Z DEBUG   03…4c-chan#255714: got commitsig 97031: feerate 864, blockheight: 0, 0 added, 0 fulfilled, 1 failed, 0 changed. 0 splice commitments.
2025-03-21T04:11:58.596Z DEBUG   03…4c-chan#255714: HTLC out 35768 SENT_ADD_ACK_REVOCATION->RCVD_REMOVE_COMMIT
2025-03-21T04:11:58.597Z DEBUG   03…4c-chan#255714: Our HTLC 35768 failed (4294967295)
2025-03-21T04:11:58.597Z DEBUG   03…4c-chan#255714: HTLC out 35768 RCVD_REMOVE_COMMIT->SENT_REMOVE_REVOCATION
2025-03-21T04:11:58.604Z DEBUG   03…4c-channeld-chan#255714: ... , awaiting 1121
2025-03-21T04:11:58.605Z DEBUG   03…4c-channeld-chan#255714: Got it!
2025-03-21T04:11:58.605Z DEBUG   03…4c-hsmd: Got WIRE_HSMD_REVOKE_COMMITMENT_TX
2025-03-21T04:11:58.605Z DEBUG   03…4c-channeld-chan#255714: peer_out WIRE_REVOKE_AND_ACK

Now we are expected to send a new commitment.

  1. We send a new commitment, moving our HTLC 35768 into the SENT_REMOVE_ACK_COMMIT state:
2025-03-21T04:11:58.605Z DEBUG   03…4c-channeld-chan#255714: Trying commit
2025-03-21T04:11:58.605Z DEBUG   03…4c-channeld-chan#255714: htlc 35768: SENT_REMOVE_REVOCATION->SENT_REMOVE_ACK_COMMIT
2025-03-21T04:11:58.606Z DEBUG   03…4c-channeld-chan#255714: htlc added REMOTE: local 1270596398 remote 229569507
2025-03-21T04:11:58.606Z DEBUG   03…4c-channeld-chan#255714: -> local 1470876853 remote 229569507
2025-03-21T04:11:58.606Z DEBUG   03…4c-channeld-chan#255714: sending_commit: HTLC LOCAL 35768 = SENT_REMOVE_ACK_COMMIT/RCVD_REMOVE_ACK_COMMIT FAILED
2025-03-21T04:11:58.607Z DEBUG   03…4c-hsmd: Got WIRE_HSMD_SIGN_REMOTE_COMMITMENT_TX
2025-03-21T04:11:58.607Z DEBUG   03…4c-channeld-chan#255714: Creating commit_sig signature 97036 30…01 for tx 02…20 wscript 52…ae key 02…55
2025-03-21T04:11:58.608Z DEBUG   03…4c-hsmd: Got WIRE_HSMD_SIGN_REMOTE_HTLC_TX
2025-03-21T04:11:58.608Z DEBUG   03…4c-channeld-chan#255714: Creating HTLC signature 30…83 for tx 02…00 wscript 76…68 key 03…26
2025-03-21T04:11:58.608Z DEBUG   03…4c-hsmd: Got WIRE_HSMD_SIGN_REMOTE_HTLC_TX
2025-03-21T04:11:58.609Z DEBUG   03…4c-channeld-chan#255714: Creating HTLC signature 30…83 for tx 02…00 wscript 76…68 key 03…26
2025-03-21T04:11:58.609Z DEBUG   03…4c-hsmd: Got WIRE_HSMD_SIGN_REMOTE_HTLC_TX
2025-03-21T04:11:58.609Z DEBUG   03…4c-channeld-chan#255714: Creating HTLC signature 30…83 for tx 02…00 wscript 76…68 key 03…26
2025-03-21T04:11:58.609Z DEBUG   03…4c-channeld-chan#255714: Telling master we're about to commit...
2025-03-21T04:11:58.610Z DEBUG   03…4c-channeld-chan#255714: Sending master 1020
2025-03-21T04:11:58.610Z DEBUG   03…4c-chan#255714: HTLC out 35768 SENT_REMOVE_REVOCATION->SENT_REMOVE_ACK_COMMIT
2025-03-21T04:11:58.618Z DEBUG   03…4c-channeld-chan#255714: ... , awaiting 1120
2025-03-21T04:11:58.618Z DEBUG   03…4c-channeld-chan#255714: Got it!
2025-03-21T04:11:58.618Z DEBUG   03…4c-channeld-chan#255714: Sending commit_sig with 3 htlc sigs
2025-03-21T04:11:58.621Z DEBUG   03…4c-channeld-chan#255714: peer_out WIRE_COMMITMENT_SIGNED
2025-03-21T04:11:58.629Z DEBUG   03…4c-channeld-chan#255714: Can't send commit: waiting for revoke_and_ack

Now we are expecting to receive a revoke_and_ack from the peer.


  1. The peer fails our HTLC 35769, moving it into the RCVD_REMOVE_HTLC state:
2025-03-21T04:12:07.970Z DEBUG   03…4c-channeld-chan#255714: peer_in WIRE_UPDATE_FAIL_HTLC
2025-03-21T04:12:07.970Z DEBUG   03…4c-channeld-chan#255714: FAIL:: HTLC LOCAL 35769 = RCVD_REMOVE_HTLC/SENT_REMOVE_HTLC 
2025-03-21T04:12:07.980Z DEBUG   03…4c-channeld-chan#255714: Can't send commit: waiting for revoke_and_ack

Now we are expected to send a new commitment, but we can't because we are still waiting to receive a revoke_and_ack from the peer. We are also now expecting to receive a new commitment from the peer.


  1. We decide to forward a payment over the channel, so we add HTLC 35771 in the SENT_ADD_HTLC state:
2025-03-21T04:12:08.488Z DEBUG   02…76-chan#255553: Decided to forward htlc_id=6611 over channel with scid=…50x…58x0 with peer 03…4c
2025-03-21T04:12:08.499Z DEBUG   03…4c-channeld-chan#255714: NEW:: HTLC LOCAL 35771 = SENT_ADD_HTLC/RCVD_ADD_HTLC 
2025-03-21T04:12:08.499Z DEBUG   03…4c-channeld-chan#255714: Adding HTLC 35771 amount=6754437msat cltv=889173 gave CHANNEL_ERR_ADD_OK
2025-03-21T04:12:08.499Z DEBUG   03…4c-channeld-chan#255714: peer_out WIRE_UPDATE_ADD_HTLC
2025-03-21T04:12:08.499Z DEBUG   03…4c-channeld-chan#255714: REPLY WIRE_CHANNELD_OFFER_HTLC_REPLY with 0 fds
2025-03-21T04:12:08.509Z DEBUG   03…4c-channeld-chan#255714: Can't send commit: waiting for revoke_and_ack

Now we are expected to send a new commitment, but we still can't because we are still waiting to receive a revoke_and_ack from the peer. We are also still expecting to receive a new commitment from the peer.


  1. The peer sends a new commitment, moving our HTLC 35769 into the RCVD_REMOVE_COMMIT state:
2025-03-21T04:12:17.569Z DEBUG   03…4c-channeld-chan#255714: peer_in WIRE_COMMITMENT_SIGNED
2025-03-21T04:12:17.569Z DEBUG   03…4c-channeld-chan#255714: handle_peer_commit_sig(splice: 0, remote_splice: 0, index: 97032)
2025-03-21T04:12:17.569Z DEBUG   03…4c-channeld-chan#255714: Received commit
2025-03-21T04:12:17.569Z DEBUG   03…4c-channeld-chan#255714: htlc 35769: RCVD_REMOVE_HTLC->RCVD_REMOVE_COMMIT
2025-03-21T04:12:17.569Z DEBUG   03…4c-channeld-chan#255714: htlc added LOCAL: local 1470876853 remote 229569507
2025-03-21T04:12:17.569Z DEBUG   03…4c-channeld-chan#255714: -> local 1546075830 remote 229569507
2025-03-21T04:12:17.569Z DEBUG   03…4c-channeld-chan#255714: rcvd_commit: HTLC LOCAL 35769 = RCVD_REMOVE_COMMIT/SENT_REMOVE_COMMIT FAILED
2025-03-21T04:12:17.569Z DEBUG   03…4c-channeld-chan#255714: Feerates are 864/864
2025-03-21T04:12:17.569Z DEBUG   03…4c-channeld-chan#255714: Committed 2, removing 0, adding 0
2025-03-21T04:12:17.570Z DEBUG   03…4c-channeld-chan#255714: We need 21928sat at feerate 864 for 2 untrimmed htlcs: we have 229569507msat/229569507msat (will have 229569507msat)
2025-03-21T04:12:17.570Z DEBUG   03…4c-channeld-chan#255714: Derived key 02…25 from basepoint 03…93, point 03…3e
2025-03-21T04:12:17.570Z DEBUG   03…4c-channeld-chan#255714: Received commit_sig with 2 htlc sigs
2025-03-21T04:12:17.570Z DEBUG   03…4c-hsmd: Got WIRE_HSMD_VALIDATE_COMMITMENT_TX

Now we are expected to send a revoke_and_ack and then our own new commitment.

  1. We send the expected revoke_and_ack, moving our HTLC 35769 into the SENT_REMOVE_REVOCATION state:
2025-03-21T04:12:17.570Z DEBUG   03…4c-channeld-chan#255714: Sending revoke_and_ack
2025-03-21T04:12:17.570Z DEBUG   03…4c-channeld-chan#255714: htlc 35769: RCVD_REMOVE_COMMIT->SENT_REMOVE_REVOCATION
2025-03-21T04:12:17.570Z DEBUG   03…4c-channeld-chan#255714: sending_revoke_and_ack: HTLC LOCAL 35769 = SENT_REMOVE_REVOCATION/RCVD_REMOVE_REVOCATION FAILED
2025-03-21T04:12:17.570Z DEBUG   03…4c-channeld-chan#255714: revoke_and_ack made pending: commit timer
2025-03-21T04:12:17.571Z DEBUG   03…4c-channeld-chan#255714: Sending master 1021
2025-03-21T04:12:17.571Z DEBUG   03…4c-chan#255714: got commitsig 97032: feerate 864, blockheight: 0, 0 added, 0 fulfilled, 1 failed, 0 changed. 0 splice commitments.
2025-03-21T04:12:17.571Z DEBUG   03…4c-chan#255714: HTLC out 35769 SENT_ADD_ACK_REVOCATION->RCVD_REMOVE_COMMIT
2025-03-21T04:12:17.571Z DEBUG   03…4c-chan#255714: Our HTLC 35769 failed (4294967295)
2025-03-21T04:12:17.572Z DEBUG   03…4c-chan#255714: HTLC out 35769 RCVD_REMOVE_COMMIT->SENT_REMOVE_REVOCATION
2025-03-21T04:12:17.578Z DEBUG   03…4c-channeld-chan#255714: ... , awaiting 1121
2025-03-21T04:12:17.579Z DEBUG   03…4c-channeld-chan#255714: Got it!
2025-03-21T04:12:17.579Z DEBUG   03…4c-hsmd: Got WIRE_HSMD_REVOKE_COMMITMENT_TX
2025-03-21T04:12:17.580Z DEBUG   03…4c-channeld-chan#255714: peer_out WIRE_REVOKE_AND_ACK
2025-03-21T04:12:17.580Z DEBUG   03…4c-channeld-chan#255714: Can't send commit: waiting for revoke_and_ack

Now we are expected to send a new commitment, but we still can't because we are still waiting to receive a revoke_and_ack from the peer.


  1. We decide to forward a payment over the channel, so we add HTLC 35772 in the SENT_ADD_HTLC state:
2025-03-21T04:12:19.615Z DEBUG   03…3f-chan#249521: Decided to forward htlc_id=419109 over channel with scid=…50x…58x0 with peer 03…4c
2025-03-21T04:12:19.664Z DEBUG   03…4c-channeld-chan#255714: NEW:: HTLC LOCAL 35772 = SENT_ADD_HTLC/RCVD_ADD_HTLC 
2025-03-21T04:12:19.664Z DEBUG   03…4c-channeld-chan#255714: Adding HTLC 35772 amount=286627633msat cltv=889025 gave CHANNEL_ERR_ADD_OK
2025-03-21T04:12:19.664Z DEBUG   03…4c-channeld-chan#255714: peer_out WIRE_UPDATE_ADD_HTLC
2025-03-21T04:12:19.664Z DEBUG   03…4c-channeld-chan#255714: REPLY WIRE_CHANNELD_OFFER_HTLC_REPLY with 0 fds
2025-03-21T04:12:19.670Z DEBUG   03…4c-channeld-chan#255714: Can't send commit: waiting for revoke_and_ack

Now we are expected to send a new commitment, but we still can't because we are still waiting to receive a revoke_and_ack from the peer.


  1. We receive a revoke_and_ack from the peer, moving our HTLC 35768 into the RCVD_REMOVE_ACK_REVOCATION state:
2025-03-21T04:12:22.059Z DEBUG   03…4c-channeld-chan#255714: peer_in WIRE_REVOKE_AND_ACK
2025-03-21T04:12:22.059Z DEBUG   03…4c-channeld-chan#255714: Received revoke_and_ack
2025-03-21T04:12:22.059Z DEBUG   03…4c-channeld-chan#255714: htlc 35768: SENT_REMOVE_ACK_COMMIT->RCVD_REMOVE_ACK_REVOCATION
2025-03-21T04:12:22.059Z DEBUG   03…4c-channeld-chan#255714: rcvd_revoke_and_ack: HTLC LOCAL 35768 = RCVD_REMOVE_ACK_REVOCATION/SENT_REMOVE_ACK_REVOCATION FAILED
2025-03-21T04:12:22.059Z DEBUG   03…4c-channeld-chan#255714: No commits outstanding after recv revoke_and_ack
2025-03-21T04:12:22.059Z DEBUG   03…4c-channeld-chan#255714: HTLC 35768[LOCAL] => RCVD_REMOVE_ACK_REVOCATION
2025-03-21T04:12:22.059Z DEBUG   03…4c-hsmd: Got WIRE_HSMD_VALIDATE_REVOCATION
2025-03-21T04:12:22.061Z DEBUG   03…4c-hsmd: Got WIRE_HSMD_SIGN_PENALTY_TO_US
2025-03-21T04:12:22.061Z DEBUG   03…4c-channeld-chan#255714: Sending master 1022
2025-03-21T04:12:22.061Z DEBUG   03…4c-chan#255714: got revoke 97035: 1 changed
2025-03-21T04:12:22.061Z DEBUG   03…4c-chan#255714: HTLC out 35768 SENT_REMOVE_ACK_COMMIT->RCVD_REMOVE_ACK_REVOCATION
2025-03-21T04:12:22.062Z DEBUG   03…4c-chan#255714: Removing out HTLC 35768 state RCVD_REMOVE_ACK_REVOCATION REMOTEFAIL
2025-03-21T04:12:22.069Z DEBUG   03…4c-channeld-chan#255714: ... , awaiting 1122
2025-03-21T04:12:22.081Z DEBUG   03…4c-channeld-chan#255714: Got it!
2025-03-21T04:12:22.082Z DEBUG   03…4c-channeld-chan#255714: revoke_and_ack REMOTE: remote_per_commit = 03…b7, old_remote_per_commit = 03…d8

We are still expected to send a new commitment, and now we can.

  1. We send a new commitment, moving our HTLC 35769 into the SENT_REMOVE_ACK_COMMIT state and our HTLCs 35771 and 35772 into the SENT_ADD_COMMIT state:
2025-03-21T04:12:22.097Z DEBUG   03…4c-hsmd: Got WIRE_HSMD_SIGN_REMOTE_COMMITMENT_TX
2025-03-21T04:12:22.097Z DEBUG   03…4c-hsmd: Got WIRE_HSMD_SIGN_REMOTE_HTLC_TX
2025-03-21T04:12:22.097Z DEBUG   03…4c-hsmd: Got WIRE_HSMD_SIGN_REMOTE_HTLC_TX
2025-03-21T04:12:22.097Z DEBUG   03…4c-hsmd: Got WIRE_HSMD_SIGN_REMOTE_HTLC_TX
2025-03-21T04:12:22.097Z DEBUG   03…4c-hsmd: Got WIRE_HSMD_SIGN_REMOTE_HTLC_TX
2025-03-21T04:12:22.098Z DEBUG   03…4c-channeld-chan#255714: Trying commit
2025-03-21T04:12:22.098Z DEBUG   03…4c-channeld-chan#255714: htlc 35772: SENT_ADD_HTLC->SENT_ADD_COMMIT
2025-03-21T04:12:22.098Z DEBUG   03…4c-channeld-chan#255714: htlc added REMOTE: local 1470876853 remote 229569507
2025-03-21T04:12:22.098Z DEBUG   03…4c-channeld-chan#255714: -> local 1184249220 remote 229569507
2025-03-21T04:12:22.098Z DEBUG   03…4c-channeld-chan#255714: sending_commit: HTLC LOCAL 35772 = SENT_ADD_COMMIT/RCVD_ADD_COMMIT 
2025-03-21T04:12:22.098Z DEBUG   03…4c-channeld-chan#255714: htlc 35769: SENT_REMOVE_REVOCATION->SENT_REMOVE_ACK_COMMIT
2025-03-21T04:12:22.099Z DEBUG   03…4c-channeld-chan#255714: htlc added REMOTE: local 1184249220 remote 229569507
2025-03-21T04:12:22.099Z DEBUG   03…4c-channeld-chan#255714: -> local 1259448197 remote 229569507
2025-03-21T04:12:22.099Z DEBUG   03…4c-channeld-chan#255714: sending_commit: HTLC LOCAL 35769 = SENT_REMOVE_ACK_COMMIT/RCVD_REMOVE_ACK_COMMIT FAILED
2025-03-21T04:12:22.099Z DEBUG   03…4c-channeld-chan#255714: htlc 35771: SENT_ADD_HTLC->SENT_ADD_COMMIT
2025-03-21T04:12:22.099Z DEBUG   03…4c-channeld-chan#255714: htlc added REMOTE: local 1259448197 remote 229569507
2025-03-21T04:12:22.099Z DEBUG   03…4c-channeld-chan#255714: -> local 1252693760 remote 229569507
2025-03-21T04:12:22.099Z DEBUG   03…4c-channeld-chan#255714: sending_commit: HTLC LOCAL 35771 = SENT_ADD_COMMIT/RCVD_ADD_COMMIT 
2025-03-21T04:12:22.099Z DEBUG   03…4c-channeld-chan#255714: Creating commit_sig signature 97037 30…01 for tx 02…20 wscript 52…ae key 02…55
2025-03-21T04:12:22.099Z DEBUG   03…4c-channeld-chan#255714: Creating HTLC signature 30…83 for tx 02…00 wscript 76…68 key 02…05
2025-03-21T04:12:22.099Z DEBUG   03…4c-channeld-chan#255714: Creating HTLC signature 30…83 for tx 02…00 wscript 76…68 key 02…05
2025-03-21T04:12:22.099Z DEBUG   03…4c-channeld-chan#255714: Creating HTLC signature 30…83 for tx 02…00 wscript 76…68 key 02…05
2025-03-21T04:12:22.099Z DEBUG   03…4c-channeld-chan#255714: Creating HTLC signature 30…83 for tx 02…00 wscript 76…68 key 02…05
2025-03-21T04:12:22.099Z DEBUG   03…4c-channeld-chan#255714: Telling master we're about to commit...
2025-03-21T04:12:22.099Z DEBUG   03…4c-channeld-chan#255714: Sending master 1020
2025-03-21T04:12:22.101Z DEBUG   03…4c-chan#255714: HTLC out 35772 SENT_ADD_HTLC->SENT_ADD_COMMIT
2025-03-21T04:12:22.101Z DEBUG   03…4c-chan#255714: HTLC out 35769 SENT_REMOVE_REVOCATION->SENT_REMOVE_ACK_COMMIT
2025-03-21T04:12:22.103Z DEBUG   03…4c-chan#255714: HTLC out 35771 SENT_ADD_HTLC->SENT_ADD_COMMIT
2025-03-21T04:12:22.113Z DEBUG   03…4c-channeld-chan#255714: ... , awaiting 1120
2025-03-21T04:12:22.118Z DEBUG   03…4c-channeld-chan#255714: Got it!
2025-03-21T04:12:22.118Z DEBUG   03…4c-channeld-chan#255714: Sending commit_sig with 4 htlc sigs
2025-03-21T04:12:22.124Z DEBUG   03…4c-channeld-chan#255714: peer_out WIRE_COMMITMENT_SIGNED
2025-03-21T04:12:22.128Z DEBUG   03…4c-channeld-chan#255714: Can't send commit: waiting for revoke_and_ack

Now we are expecting to receive a revoke_and_ack from the peer and then their new commitment.


  1. We receive a revoke_and_ack from the peer, moving our HTLC 35769 into the RCVD_REMOVE_ACK_REVOCATION state and our HTLCs 35771 and 35772 into the RCVD_ADD_REVOCATION state:
2025-03-21T04:12:32.536Z DEBUG   03…4c-channeld-chan#255714: peer_in WIRE_REVOKE_AND_ACK
2025-03-21T04:12:32.536Z DEBUG   03…4c-channeld-chan#255714: Received revoke_and_ack
2025-03-21T04:12:32.536Z DEBUG   03…4c-channeld-chan#255714: htlc 35772: SENT_ADD_COMMIT->RCVD_ADD_REVOCATION
2025-03-21T04:12:32.536Z DEBUG   03…4c-channeld-chan#255714: rcvd_revoke_and_ack: HTLC LOCAL 35772 = RCVD_ADD_REVOCATION/SENT_ADD_REVOCATION 
2025-03-21T04:12:32.536Z DEBUG   03…4c-channeld-chan#255714: htlc 35769: SENT_REMOVE_ACK_COMMIT->RCVD_REMOVE_ACK_REVOCATION
2025-03-21T04:12:32.536Z DEBUG   03…4c-channeld-chan#255714: rcvd_revoke_and_ack: HTLC LOCAL 35769 = RCVD_REMOVE_ACK_REVOCATION/SENT_REMOVE_ACK_REVOCATION FAILED
2025-03-21T04:12:32.536Z DEBUG   03…4c-channeld-chan#255714: htlc 35771: SENT_ADD_COMMIT->RCVD_ADD_REVOCATION
2025-03-21T04:12:32.536Z DEBUG   03…4c-channeld-chan#255714: rcvd_revoke_and_ack: HTLC LOCAL 35771 = RCVD_ADD_REVOCATION/SENT_ADD_REVOCATION 
2025-03-21T04:12:32.536Z DEBUG   03…4c-channeld-chan#255714: Commits outstanding after recv revoke_and_ack
2025-03-21T04:12:32.536Z DEBUG   03…4c-channeld-chan#255714: HTLC 35772[LOCAL] => RCVD_ADD_REVOCATION
2025-03-21T04:12:32.536Z DEBUG   03…4c-channeld-chan#255714: HTLC 35769[LOCAL] => RCVD_REMOVE_ACK_REVOCATION
2025-03-21T04:12:32.536Z DEBUG   03…4c-channeld-chan#255714: HTLC 35771[LOCAL] => RCVD_ADD_REVOCATION
2025-03-21T04:12:32.537Z DEBUG   03…4c-channeld-chan#255714: Sending master 1022
2025-03-21T04:12:32.537Z DEBUG   03…4c-chan#255714: got revoke 97036: 3 changed
2025-03-21T04:12:32.537Z DEBUG   03…4c-chan#255714: HTLC out 35772 SENT_ADD_COMMIT->RCVD_ADD_REVOCATION
2025-03-21T04:12:32.537Z DEBUG   03…4c-chan#255714: HTLC out 35769 SENT_REMOVE_ACK_COMMIT->RCVD_REMOVE_ACK_REVOCATION
2025-03-21T04:12:32.538Z DEBUG   03…4c-chan#255714: Removing out HTLC 35769 state RCVD_REMOVE_ACK_REVOCATION REMOTEFAIL
2025-03-21T04:12:32.539Z DEBUG   03…4c-chan#255714: HTLC out 35771 SENT_ADD_COMMIT->RCVD_ADD_REVOCATION
2025-03-21T04:12:32.545Z DEBUG   03…4c-channeld-chan#255714: ... , awaiting 1122
2025-03-21T04:12:32.545Z DEBUG   03…4c-hsmd: Got WIRE_HSMD_VALIDATE_REVOCATION
2025-03-21T04:12:32.545Z DEBUG   03…4c-hsmd: Got WIRE_HSMD_SIGN_PENALTY_TO_US
2025-03-21T04:12:32.546Z DEBUG   03…4c-channeld-chan#255714: Got it!
2025-03-21T04:12:32.546Z DEBUG   03…4c-channeld-chan#255714: revoke_and_ack REMOTE: remote_per_commit = 02…91, old_remote_per_commit = 03…b7
2025-03-21T04:12:32.556Z DEBUG   03…4c-channeld-chan#255714: Trying commit
2025-03-21T04:12:32.557Z DEBUG   03…4c-channeld-chan#255714: Can't send commit: nothing to send, feechange not wanted ({ RCVD_ADD_ACK_REVOCATION:864 }) blockheight not wanted ({ RCVD_ADD_ACK_REVOCATION:0 })

Now we are expecting to receive a new commitment from the peer.


  1. We receive a new commitment from the peer, moving our HTLCs 35771 and 35772 into the RCVD_ADD_ACK_COMMIT state:
2025-03-21T04:12:38.540Z DEBUG   03…4c-channeld-chan#255714: peer_in WIRE_COMMITMENT_SIGNED
2025-03-21T04:12:38.540Z DEBUG   03…4c-channeld-chan#255714: handle_peer_commit_sig(splice: 0, remote_splice: 0, index: 97033)
2025-03-21T04:12:38.540Z DEBUG   03…4c-channeld-chan#255714: Received commit
2025-03-21T04:12:38.540Z DEBUG   03…4c-channeld-chan#255714: htlc 35772: RCVD_ADD_REVOCATION->RCVD_ADD_ACK_COMMIT
2025-03-21T04:12:38.540Z DEBUG   03…4c-channeld-chan#255714: htlc added LOCAL: local 1546075830 remote 229569507
2025-03-21T04:12:38.540Z DEBUG   03…4c-channeld-chan#255714: -> local 1259448197 remote 229569507
2025-03-21T04:12:38.540Z DEBUG   03…4c-channeld-chan#255714: rcvd_commit: HTLC LOCAL 35772 = RCVD_ADD_ACK_COMMIT/SENT_ADD_ACK_COMMIT 
2025-03-21T04:12:38.540Z DEBUG   03…4c-channeld-chan#255714: htlc 35771: RCVD_ADD_REVOCATION->RCVD_ADD_ACK_COMMIT
2025-03-21T04:12:38.540Z DEBUG   03…4c-channeld-chan#255714: htlc added LOCAL: local 1259448197 remote 229569507
2025-03-21T04:12:38.540Z DEBUG   03…4c-channeld-chan#255714: -> local 1252693760 remote 229569507
2025-03-21T04:12:38.541Z DEBUG   03…4c-channeld-chan#255714: rcvd_commit: HTLC LOCAL 35771 = RCVD_ADD_ACK_COMMIT/SENT_ADD_ACK_COMMIT 
2025-03-21T04:12:38.541Z DEBUG   03…4c-channeld-chan#255714: Feerates are 864/864
2025-03-21T04:12:38.541Z DEBUG   03…4c-channeld-chan#255714: Committed 4, removing 0, adding 0
2025-03-21T04:12:38.541Z DEBUG   03…4c-channeld-chan#255714: We need 22225sat at feerate 864 for 4 untrimmed htlcs: we have 229569507msat/229569507msat (will have 229569507msat)
2025-03-21T04:12:38.541Z DEBUG   03…4c-channeld-chan#255714: Derived key 02…5a from basepoint 03…93, point 03…77
2025-03-21T04:12:38.541Z DEBUG   03…4c-channeld-chan#255714: Received commit_sig with 4 htlc sigs
2025-03-21T04:12:38.542Z DEBUG   03…4c-hsmd: Got WIRE_HSMD_VALIDATE_COMMITMENT_TX

Now we are expected to send a revoke_and_ack.

  1. We send the expected revoke_and_ack, moving our HTLCs 35771 and 35772 into the SENT_ADD_ACK_REVOCATION state:
2025-03-21T04:12:38.542Z DEBUG   03…4c-channeld-chan#255714: Sending revoke_and_ack
2025-03-21T04:12:38.542Z DEBUG   03…4c-channeld-chan#255714: htlc 35772: RCVD_ADD_ACK_COMMIT->SENT_ADD_ACK_REVOCATION
2025-03-21T04:12:38.542Z DEBUG   03…4c-channeld-chan#255714: sending_revoke_and_ack: HTLC LOCAL 35772 = SENT_ADD_ACK_REVOCATION/RCVD_ADD_ACK_REVOCATION 
2025-03-21T04:12:38.543Z DEBUG   03…4c-channeld-chan#255714: htlc 35771: RCVD_ADD_ACK_COMMIT->SENT_ADD_ACK_REVOCATION
2025-03-21T04:12:38.543Z DEBUG   03…4c-channeld-chan#255714: sending_revoke_and_ack: HTLC LOCAL 35771 = SENT_ADD_ACK_REVOCATION/RCVD_ADD_ACK_REVOCATION 
2025-03-21T04:12:38.543Z DEBUG   03…4c-channeld-chan#255714: Sending master 1021
2025-03-21T04:12:38.543Z DEBUG   03…4c-chan#255714: got commitsig 97033: feerate 864, blockheight: 0, 0 added, 0 fulfilled, 0 failed, 2 changed. 0 splice commitments.
2025-03-21T04:12:38.543Z DEBUG   03…4c-chan#255714: HTLC out 35772 RCVD_ADD_REVOCATION->RCVD_ADD_ACK_COMMIT
2025-03-21T04:12:38.544Z DEBUG   03…4c-chan#255714: HTLC out 35771 RCVD_ADD_REVOCATION->RCVD_ADD_ACK_COMMIT
2025-03-21T04:12:38.544Z DEBUG   03…4c-chan#255714: HTLC out 35772 RCVD_ADD_ACK_COMMIT->SENT_ADD_ACK_REVOCATION
2025-03-21T04:12:38.544Z DEBUG   03…4c-chan#255714: HTLC out 35771 RCVD_ADD_ACK_COMMIT->SENT_ADD_ACK_REVOCATION
2025-03-21T04:12:38.551Z DEBUG   03…4c-channeld-chan#255714: ... , awaiting 1121
2025-03-21T04:12:38.552Z DEBUG   03…4c-channeld-chan#255714: Got it!
2025-03-21T04:12:38.552Z DEBUG   03…4c-hsmd: Got WIRE_HSMD_REVOKE_COMMITMENT_TX
2025-03-21T04:12:38.552Z DEBUG   03…4c-channeld-chan#255714: peer_out WIRE_REVOKE_AND_ACK

Now we're all sync'd up with our HTLCs 35771 and 35772 fully added to the channel. (The channel also has our HTLC 35770.)

@whitslack
Copy link
Collaborator Author

  1. The peer fails our HTLC 35770, moving it into the RCVD_REMOVE_HTLC state:
2025-03-21T04:12:38.552Z DEBUG   03…4c-channeld-chan#255714: peer_in WIRE_UPDATE_FAIL_HTLC
2025-03-21T04:12:38.552Z DEBUG   03…4c-channeld-chan#255714: FAIL:: HTLC LOCAL 35770 = RCVD_REMOVE_HTLC/SENT_REMOVE_HTLC 
2025-03-21T04:12:38.562Z DEBUG   03…4c-channeld-chan#255714: Trying commit
2025-03-21T04:12:38.562Z DEBUG   03…4c-channeld-chan#255714: Can't send commit: nothing to send, feechange not wanted ({ RCVD_ADD_ACK_REVOCATION:864 }) blockheight not wanted ({ RCVD_ADD_ACK_REVOCATION:0 })

Now we are expecting to receive a new commitment from the peer.


  1. We decide to forward a payment over the channel, so we add HTLC 35773 in the SENT_ADD_HTLC state:
2025-03-21T04:12:42.658Z DEBUG   02…a2-chan#255606: Decided to forward htlc_id=43107 over channel with scid=…50x…58x0 with peer 03…4c
2025-03-21T04:12:42.667Z DEBUG   03…4c-channeld-chan#255714: NEW:: HTLC LOCAL 35773 = SENT_ADD_HTLC/RCVD_ADD_HTLC 
2025-03-21T04:12:42.667Z DEBUG   03…4c-channeld-chan#255714: Adding HTLC 35773 amount=69663541msat cltv=888979 gave CHANNEL_ERR_ADD_OK
2025-03-21T04:12:42.667Z DEBUG   03…4c-channeld-chan#255714: peer_out WIRE_UPDATE_ADD_HTLC
2025-03-21T04:12:42.667Z DEBUG   03…4c-channeld-chan#255714: REPLY WIRE_CHANNELD_OFFER_HTLC_REPLY with 0 fds

Now we are expected to send a new commitment. We are also still expecting to receive a new commitment from the peer.

  1. We send a new commitment, moving our HTLC 35773 into the SENT_ADD_COMMIT state:
2025-03-21T04:12:42.682Z DEBUG   03…4c-hsmd: Got WIRE_HSMD_SIGN_REMOTE_COMMITMENT_TX
2025-03-21T04:12:42.682Z DEBUG   03…4c-hsmd: Got WIRE_HSMD_SIGN_REMOTE_HTLC_TX
2025-03-21T04:12:42.682Z DEBUG   03…4c-hsmd: Got WIRE_HSMD_SIGN_REMOTE_HTLC_TX
2025-03-21T04:12:42.682Z DEBUG   03…4c-hsmd: Got WIRE_HSMD_SIGN_REMOTE_HTLC_TX
2025-03-21T04:12:42.684Z DEBUG   03…4c-channeld-chan#255714: Trying commit
2025-03-21T04:12:42.684Z DEBUG   03…4c-channeld-chan#255714: htlc 35773: SENT_ADD_HTLC->SENT_ADD_COMMIT
2025-03-21T04:12:42.684Z DEBUG   03…4c-channeld-chan#255714: htlc added REMOTE: local 1252693760 remote 229569507
2025-03-21T04:12:42.684Z DEBUG   03…4c-channeld-chan#255714: -> local 1183030219 remote 229569507
2025-03-21T04:12:42.685Z DEBUG   03…4c-channeld-chan#255714: sending_commit: HTLC LOCAL 35773 = SENT_ADD_COMMIT/RCVD_ADD_COMMIT 
2025-03-21T04:12:42.685Z DEBUG   03…4c-channeld-chan#255714: Creating commit_sig signature 97038 30…01 for tx 02…20 wscript 52…ae key 02…55
2025-03-21T04:12:42.685Z DEBUG   03…4c-channeld-chan#255714: Creating HTLC signature 30…83 for tx 02…00 wscript 76…68 key 03…86
2025-03-21T04:12:42.685Z DEBUG   03…4c-channeld-chan#255714: Creating HTLC signature 30…83 for tx 02…00 wscript 76…68 key 03…86
2025-03-21T04:12:42.685Z DEBUG   03…4c-channeld-chan#255714: Creating HTLC signature 30…83 for tx 02…00 wscript 76…68 key 03…86
2025-03-21T04:12:42.685Z DEBUG   03…4c-channeld-chan#255714: Creating HTLC signature 30…83 for tx 02…00 wscript 76…68 key 03…86
2025-03-21T04:12:42.685Z DEBUG   03…4c-channeld-chan#255714: Creating HTLC signature 30…83 for tx 02…00 wscript 76…68 key 03…86
2025-03-21T04:12:42.685Z DEBUG   03…4c-channeld-chan#255714: Telling master we're about to commit...
2025-03-21T04:12:42.685Z DEBUG   03…4c-channeld-chan#255714: Sending master 1020
2025-03-21T04:12:42.687Z DEBUG   03…4c-chan#255714: HTLC out 35773 SENT_ADD_HTLC->SENT_ADD_COMMIT
2025-03-21T04:12:42.697Z DEBUG   03…4c-channeld-chan#255714: ... , awaiting 1120
2025-03-21T04:12:42.700Z DEBUG   03…4c-hsmd: Got WIRE_HSMD_SIGN_REMOTE_HTLC_TX
2025-03-21T04:12:42.700Z DEBUG   03…4c-hsmd: Got WIRE_HSMD_SIGN_REMOTE_HTLC_TX
2025-03-21T04:12:42.701Z DEBUG   03…4c-channeld-chan#255714: Got it!
2025-03-21T04:12:42.701Z DEBUG   03…4c-channeld-chan#255714: Sending commit_sig with 5 htlc sigs
2025-03-21T04:12:42.704Z DEBUG   03…4c-channeld-chan#255714: peer_out WIRE_COMMITMENT_SIGNED
2025-03-21T04:12:42.711Z DEBUG   03…4c-channeld-chan#255714: Can't send commit: waiting for revoke_and_ack

Now we are expecting to receive a revoke_and_ack from the peer. We are also still expecting to receive a new commitment from the peer.


  1. We receive a new commitment from the peer, moving our HTLC 35770 into the RCVD_REMOVE_COMMIT state:
2025-03-21T04:13:09.769Z DEBUG   03…4c-channeld-chan#255714: peer_in WIRE_COMMITMENT_SIGNED
2025-03-21T04:13:09.769Z DEBUG   03…4c-channeld-chan#255714: handle_peer_commit_sig(splice: 0, remote_splice: 0, index: 97034)
2025-03-21T04:13:09.769Z DEBUG   03…4c-channeld-chan#255714: Received commit
2025-03-21T04:13:09.769Z DEBUG   03…4c-channeld-chan#255714: htlc 35770: RCVD_REMOVE_HTLC->RCVD_REMOVE_COMMIT
2025-03-21T04:13:09.769Z DEBUG   03…4c-channeld-chan#255714: htlc added LOCAL: local 1252693760 remote 229569507
2025-03-21T04:13:09.769Z DEBUG   03…4c-channeld-chan#255714: -> local 1262953995 remote 229569507
2025-03-21T04:13:09.769Z DEBUG   03…4c-channeld-chan#255714: rcvd_commit: HTLC LOCAL 35770 = RCVD_REMOVE_COMMIT/SENT_REMOVE_COMMIT FAILED
2025-03-21T04:13:09.769Z DEBUG   03…4c-channeld-chan#255714: Feerates are 864/864
2025-03-21T04:13:09.769Z DEBUG   03…4c-channeld-chan#255714: Committed 3, removing 0, adding 0
2025-03-21T04:13:09.769Z DEBUG   03…4c-channeld-chan#255714: We need 22076sat at feerate 864 for 3 untrimmed htlcs: we have 229569507msat/229569507msat (will have 229569507msat)
2025-03-21T04:13:09.769Z DEBUG   03…4c-channeld-chan#255714: Derived key 03…55 from basepoint 03…93, point 02…ab
2025-03-21T04:13:09.770Z DEBUG   03…4c-channeld-chan#255714: Received commit_sig with 3 htlc sigs

Now we are expected to send a revoke_and_ack and then our own new commitment. We are also still expecting to receive a new commitment and a revoke_and_ack from the peer.

  1. We send the expected revoke_and_ack, moving our HTLC 35770 into the SENT_REMOVE_REVOCATION state:
2025-03-21T04:13:09.770Z DEBUG   03…4c-channeld-chan#255714: Sending revoke_and_ack
2025-03-21T04:13:09.770Z DEBUG   03…4c-channeld-chan#255714: htlc 35770: RCVD_REMOVE_COMMIT->SENT_REMOVE_REVOCATION
2025-03-21T04:13:09.770Z DEBUG   03…4c-channeld-chan#255714: sending_revoke_and_ack: HTLC LOCAL 35770 = SENT_REMOVE_REVOCATION/RCVD_REMOVE_REVOCATION FAILED
2025-03-21T04:13:09.770Z DEBUG   03…4c-channeld-chan#255714: revoke_and_ack made pending: commit timer
2025-03-21T04:13:09.770Z DEBUG   03…4c-channeld-chan#255714: Sending master 1021
2025-03-21T04:13:09.770Z DEBUG   03…4c-chan#255714: got commitsig 97034: feerate 864, blockheight: 0, 0 added, 0 fulfilled, 1 failed, 0 changed. 0 splice commitments.
2025-03-21T04:13:09.770Z DEBUG   03…4c-chan#255714: HTLC out 35770 SENT_ADD_ACK_REVOCATION->RCVD_REMOVE_COMMIT
2025-03-21T04:13:09.771Z DEBUG   03…4c-chan#255714: Our HTLC 35770 failed (4294967295)
2025-03-21T04:13:09.771Z DEBUG   03…4c-chan#255714: HTLC out 35770 RCVD_REMOVE_COMMIT->SENT_REMOVE_REVOCATION
2025-03-21T04:13:09.778Z DEBUG   03…4c-channeld-chan#255714: ... , awaiting 1121
2025-03-21T04:13:09.778Z DEBUG   03…4c-hsmd: Got WIRE_HSMD_VALIDATE_COMMITMENT_TX
2025-03-21T04:13:09.787Z DEBUG   03…4c-hsmd: Got WIRE_HSMD_REVOKE_COMMITMENT_TX
2025-03-21T04:13:09.787Z DEBUG   03…4c-channeld-chan#255714: Got it!
2025-03-21T04:13:09.787Z DEBUG   03…4c-channeld-chan#255714: peer_out WIRE_REVOKE_AND_ACK

Now we are expected to send a new commitment, but we can't because we are still waiting to receive a revoke_and_ack from the peer. We are also still expecting to receive a new commitment from the peer.


  1. The peer fails our HTLC 35771, moving it into the RCVD_REMOVE_HTLC state:
2025-03-21T04:13:09.787Z DEBUG   03…4c-channeld-chan#255714: peer_in WIRE_UPDATE_FAIL_HTLC
2025-03-21T04:13:09.787Z DEBUG   03…4c-channeld-chan#255714: FAIL:: HTLC LOCAL 35771 = RCVD_REMOVE_HTLC/SENT_REMOVE_HTLC 
2025-03-21T04:13:09.787Z DEBUG   03…4c-channeld-chan#255714: Can't send commit: waiting for revoke_and_ack

We are still expected to send a new commitment, but we still can't because we are still waiting to receive a revoke_and_ack from the peer. We are also still expecting to receive a new commitment from the peer.


  1. We receive a revoke_and_ack from the peer, moving our HTLC 35773 into the RCVD_ADD_REVOCATION state:
2025-03-21T04:13:12.938Z DEBUG   03…4c-channeld-chan#255714: peer_in WIRE_REVOKE_AND_ACK
2025-03-21T04:13:12.940Z DEBUG   03…4c-channeld-chan#255714: Received revoke_and_ack
2025-03-21T04:13:12.940Z DEBUG   03…4c-channeld-chan#255714: htlc 35773: SENT_ADD_COMMIT->RCVD_ADD_REVOCATION
2025-03-21T04:13:12.940Z DEBUG   03…4c-channeld-chan#255714: rcvd_revoke_and_ack: HTLC LOCAL 35773 = RCVD_ADD_REVOCATION/SENT_ADD_REVOCATION 
2025-03-21T04:13:12.940Z DEBUG   03…4c-channeld-chan#255714: Commits outstanding after recv revoke_and_ack
2025-03-21T04:13:12.940Z DEBUG   03…4c-channeld-chan#255714: HTLC 35773[LOCAL] => RCVD_ADD_REVOCATION
2025-03-21T04:13:12.940Z DEBUG   03…4c-hsmd: Got WIRE_HSMD_VALIDATE_REVOCATION
2025-03-21T04:13:12.940Z DEBUG   03…4c-hsmd: Got WIRE_HSMD_SIGN_PENALTY_TO_US
2025-03-21T04:13:12.940Z DEBUG   03…4c-channeld-chan#255714: Sending master 1022
2025-03-21T04:13:12.940Z DEBUG   03…4c-chan#255714: got revoke 97037: 1 changed
2025-03-21T04:13:12.940Z DEBUG   03…4c-chan#255714: HTLC out 35773 SENT_ADD_COMMIT->RCVD_ADD_REVOCATION
2025-03-21T04:13:12.951Z DEBUG   03…4c-channeld-chan#255714: ... , awaiting 1122
2025-03-21T04:13:12.952Z DEBUG   03…4c-channeld-chan#255714: Got it!
2025-03-21T04:13:12.952Z DEBUG   03…4c-channeld-chan#255714: revoke_and_ack REMOTE: remote_per_commit = 03…cd, old_remote_per_commit = 02…91

We are still expected to send a new commitment, and now we can. We are also still expecting to receive a new commitment from the peer.

  1. We send a new commitment, moving our HTLC 35770 into the SENT_REMOVE_ACK_COMMIT state:
2025-03-21T04:13:12.962Z DEBUG   03…4c-channeld-chan#255714: Trying commit
2025-03-21T04:13:12.962Z DEBUG   03…4c-channeld-chan#255714: htlc 35770: SENT_REMOVE_REVOCATION->SENT_REMOVE_ACK_COMMIT
2025-03-21T04:13:12.962Z DEBUG   03…4c-channeld-chan#255714: htlc added REMOTE: local 1183030219 remote 229569507
2025-03-21T04:13:12.962Z DEBUG   03…4c-channeld-chan#255714: -> local 1193290454 remote 229569507
2025-03-21T04:13:12.962Z DEBUG   03…4c-channeld-chan#255714: sending_commit: HTLC LOCAL 35770 = SENT_REMOVE_ACK_COMMIT/RCVD_REMOVE_ACK_COMMIT FAILED
2025-03-21T04:13:12.963Z DEBUG   03…4c-hsmd: Got WIRE_HSMD_SIGN_REMOTE_COMMITMENT_TX
2025-03-21T04:13:12.964Z DEBUG   03…4c-channeld-chan#255714: Creating commit_sig signature 97039 30…01 for tx 02…20 wscript 52…ae key 02…55
2025-03-21T04:13:12.964Z DEBUG   03…4c-hsmd: Got WIRE_HSMD_SIGN_REMOTE_HTLC_TX
2025-03-21T04:13:12.965Z DEBUG   03…4c-channeld-chan#255714: Creating HTLC signature 30…83 for tx 02…00 wscript 76…68 key 02…67
2025-03-21T04:13:12.965Z DEBUG   03…4c-hsmd: Got WIRE_HSMD_SIGN_REMOTE_HTLC_TX
2025-03-21T04:13:12.966Z DEBUG   03…4c-channeld-chan#255714: Creating HTLC signature 30…83 for tx 02…00 wscript 76…68 key 02…67
2025-03-21T04:13:12.967Z DEBUG   03…4c-hsmd: Got WIRE_HSMD_SIGN_REMOTE_HTLC_TX
2025-03-21T04:13:12.967Z DEBUG   03…4c-channeld-chan#255714: Creating HTLC signature 30…83 for tx 02…00 wscript 76…68 key 02…67
2025-03-21T04:13:12.967Z DEBUG   03…4c-hsmd: Got WIRE_HSMD_SIGN_REMOTE_HTLC_TX
2025-03-21T04:13:12.968Z DEBUG   03…4c-channeld-chan#255714: Creating HTLC signature 30…83 for tx 02…00 wscript 76…68 key 02…67
2025-03-21T04:13:12.968Z DEBUG   03…4c-channeld-chan#255714: Telling master we're about to commit...
2025-03-21T04:13:12.968Z DEBUG   03…4c-channeld-chan#255714: Sending master 1020
2025-03-21T04:13:12.968Z DEBUG   03…4c-chan#255714: HTLC out 35770 SENT_REMOVE_REVOCATION->SENT_REMOVE_ACK_COMMIT
2025-03-21T04:13:12.976Z DEBUG   03…4c-channeld-chan#255714: ... , awaiting 1120
2025-03-21T04:13:12.976Z DEBUG   03…4c-channeld-chan#255714: Got it!
2025-03-21T04:13:12.976Z DEBUG   03…4c-channeld-chan#255714: Sending commit_sig with 4 htlc sigs
2025-03-21T04:13:12.982Z DEBUG   03…4c-channeld-chan#255714: peer_out WIRE_COMMITMENT_SIGNED
2025-03-21T04:13:12.986Z DEBUG   03…4c-channeld-chan#255714: Can't send commit: waiting for revoke_and_ack

Now we are expecting to receive a revoke_and_ack from the peer. We are also still expecting to receive a new commitment from the peer.


  1. We receive a new commitment from the peer, moving our HTLC 35771 into the RCVD_REMOVE_COMMIT state and our HTLC 35773 into the RCVD_ADD_ACK_COMMIT state:
2025-03-21T04:13:33.870Z DEBUG   03…4c-channeld-chan#255714: peer_in WIRE_COMMITMENT_SIGNED
2025-03-21T04:13:33.870Z DEBUG   03…4c-channeld-chan#255714: handle_peer_commit_sig(splice: 0, remote_splice: 0, index: 97035)
2025-03-21T04:13:33.870Z DEBUG   03…4c-channeld-chan#255714: Received commit
2025-03-21T04:13:33.870Z DEBUG   03…4c-channeld-chan#255714: htlc 35773: RCVD_ADD_REVOCATION->RCVD_ADD_ACK_COMMIT
2025-03-21T04:13:33.870Z DEBUG   03…4c-channeld-chan#255714: htlc added LOCAL: local 1262953995 remote 229569507
2025-03-21T04:13:33.870Z DEBUG   03…4c-channeld-chan#255714: -> local 1193290454 remote 229569507
2025-03-21T04:13:33.871Z DEBUG   03…4c-channeld-chan#255714: rcvd_commit: HTLC LOCAL 35773 = RCVD_ADD_ACK_COMMIT/SENT_ADD_ACK_COMMIT 
2025-03-21T04:13:33.871Z DEBUG   03…4c-channeld-chan#255714: htlc 35771: RCVD_REMOVE_HTLC->RCVD_REMOVE_COMMIT
2025-03-21T04:13:33.871Z DEBUG   03…4c-channeld-chan#255714: htlc added LOCAL: local 1193290454 remote 229569507
2025-03-21T04:13:33.871Z DEBUG   03…4c-channeld-chan#255714: -> local 1200044891 remote 229569507
2025-03-21T04:13:33.871Z DEBUG   03…4c-channeld-chan#255714: rcvd_commit: HTLC LOCAL 35771 = RCVD_REMOVE_COMMIT/SENT_REMOVE_COMMIT FAILED
2025-03-21T04:13:33.871Z DEBUG   03…4c-channeld-chan#255714: Feerates are 864/864
2025-03-21T04:13:33.871Z DEBUG   03…4c-channeld-chan#255714: Committed 3, removing 0, adding 0
2025-03-21T04:13:33.871Z DEBUG   03…4c-channeld-chan#255714: We need 22076sat at feerate 864 for 3 untrimmed htlcs: we have 229569507msat/229569507msat (will have 229569507msat)
2025-03-21T04:13:33.871Z DEBUG   03…4c-channeld-chan#255714: Derived key 02…6d from basepoint 03…93, point 02…c7
2025-03-21T04:13:33.872Z DEBUG   03…4c-channeld-chan#255714: Received commit_sig with 3 htlc sigs
2025-03-21T04:13:33.872Z DEBUG   03…4c-hsmd: Got WIRE_HSMD_VALIDATE_COMMITMENT_TX

Now we are expected to send a revoke_and_ack and then our own new commitment. We are also still expecting to receive a revoke_and_ack from the peer.

  1. We send the expected revoke_and_ack, moving our HTLC 35771 into the SENT_REMOVE_REVOCATION state and our HTLC 35773 into the SENT_ADD_ACK_REVOCATION state:
2025-03-21T04:13:33.872Z DEBUG   03…4c-channeld-chan#255714: Sending revoke_and_ack
2025-03-21T04:13:33.872Z DEBUG   03…4c-channeld-chan#255714: htlc 35773: RCVD_ADD_ACK_COMMIT->SENT_ADD_ACK_REVOCATION
2025-03-21T04:13:33.872Z DEBUG   03…4c-channeld-chan#255714: sending_revoke_and_ack: HTLC LOCAL 35773 = SENT_ADD_ACK_REVOCATION/RCVD_ADD_ACK_REVOCATION 
2025-03-21T04:13:33.872Z DEBUG   03…4c-channeld-chan#255714: htlc 35771: RCVD_REMOVE_COMMIT->SENT_REMOVE_REVOCATION
2025-03-21T04:13:33.873Z DEBUG   03…4c-channeld-chan#255714: sending_revoke_and_ack: HTLC LOCAL 35771 = SENT_REMOVE_REVOCATION/RCVD_REMOVE_REVOCATION FAILED
2025-03-21T04:13:33.873Z DEBUG   03…4c-channeld-chan#255714: revoke_and_ack made pending: commit timer
2025-03-21T04:13:33.873Z DEBUG   03…4c-channeld-chan#255714: Sending master 1021
2025-03-21T04:13:33.873Z DEBUG   03…4c-chan#255714: got commitsig 97035: feerate 864, blockheight: 0, 0 added, 0 fulfilled, 1 failed, 1 changed. 0 splice commitments.
2025-03-21T04:13:33.873Z DEBUG   03…4c-chan#255714: HTLC out 35771 SENT_ADD_ACK_REVOCATION->RCVD_REMOVE_COMMIT
2025-03-21T04:13:33.873Z DEBUG   03…4c-chan#255714: Our HTLC 35771 failed (4294967295)
2025-03-21T04:13:33.874Z DEBUG   03…4c-chan#255714: HTLC out 35773 RCVD_ADD_REVOCATION->RCVD_ADD_ACK_COMMIT
2025-03-21T04:13:33.875Z DEBUG   03…4c-chan#255714: HTLC out 35771 RCVD_REMOVE_COMMIT->SENT_REMOVE_REVOCATION
2025-03-21T04:13:33.875Z DEBUG   03…4c-chan#255714: HTLC out 35773 RCVD_ADD_ACK_COMMIT->SENT_ADD_ACK_REVOCATION
2025-03-21T04:13:33.882Z DEBUG   03…4c-channeld-chan#255714: ... , awaiting 1121
2025-03-21T04:13:33.882Z DEBUG   03…4c-channeld-chan#255714: Got it!
2025-03-21T04:13:33.882Z DEBUG   03…4c-hsmd: Got WIRE_HSMD_REVOKE_COMMITMENT_TX
2025-03-21T04:13:33.883Z DEBUG   03…4c-channeld-chan#255714: peer_out WIRE_REVOKE_AND_ACK
2025-03-21T04:13:33.883Z DEBUG   03…4c-channeld-chan#255714: Can't send commit: waiting for revoke_and_ack

Now we are expected to send a new commitment, but we can't because we are still waiting to receive a revoke_and_ack from the peer.


  1. We receive a revoke_and_ack from the peer, moving our HTLC 35770 into the RCVD_REMOVE_ACK_REVOCATION state:
2025-03-21T04:13:40.034Z DEBUG   03…4c-channeld-chan#255714: peer_in WIRE_REVOKE_AND_ACK
2025-03-21T04:13:40.034Z DEBUG   03…4c-channeld-chan#255714: Received revoke_and_ack
2025-03-21T04:13:40.034Z DEBUG   03…4c-channeld-chan#255714: htlc 35770: SENT_REMOVE_ACK_COMMIT->RCVD_REMOVE_ACK_REVOCATION
2025-03-21T04:13:40.034Z DEBUG   03…4c-channeld-chan#255714: rcvd_revoke_and_ack: HTLC LOCAL 35770 = RCVD_REMOVE_ACK_REVOCATION/SENT_REMOVE_ACK_REVOCATION FAILED
2025-03-21T04:13:40.034Z DEBUG   03…4c-channeld-chan#255714: No commits outstanding after recv revoke_and_ack
2025-03-21T04:13:40.035Z DEBUG   03…4c-channeld-chan#255714: HTLC 35770[LOCAL] => RCVD_REMOVE_ACK_REVOCATION
2025-03-21T04:13:40.035Z DEBUG   03…4c-hsmd: Got WIRE_HSMD_VALIDATE_REVOCATION
2025-03-21T04:13:40.036Z DEBUG   03…4c-hsmd: Got WIRE_HSMD_SIGN_PENALTY_TO_US
2025-03-21T04:13:40.036Z DEBUG   03…4c-channeld-chan#255714: Sending master 1022
2025-03-21T04:13:40.036Z DEBUG   03…4c-chan#255714: got revoke 97038: 1 changed
2025-03-21T04:13:40.036Z DEBUG   03…4c-chan#255714: HTLC out 35770 SENT_REMOVE_ACK_COMMIT->RCVD_REMOVE_ACK_REVOCATION
2025-03-21T04:13:40.037Z DEBUG   03…4c-chan#255714: Removing out HTLC 35770 state RCVD_REMOVE_ACK_REVOCATION REMOTEFAIL
2025-03-21T04:13:40.045Z DEBUG   03…4c-channeld-chan#255714: ... , awaiting 1122
2025-03-21T04:13:40.045Z DEBUG   03…4c-channeld-chan#255714: Got it!
2025-03-21T04:13:40.045Z DEBUG   03…4c-channeld-chan#255714: revoke_and_ack REMOTE: remote_per_commit = 02…97, old_remote_per_commit = 03…cd

We are still expected to send a new commitment, and now we can.

  1. The peer fails our HTLC 35772, moving it into the RCVD_REMOVE_HTLC state:
2025-03-21T04:13:40.045Z DEBUG   03…4c-channeld-chan#255714: peer_in WIRE_UPDATE_FAIL_HTLC
2025-03-21T04:13:40.045Z DEBUG   03…4c-channeld-chan#255714: FAIL:: HTLC LOCAL 35772 = RCVD_REMOVE_HTLC/SENT_REMOVE_HTLC 

Now we are expecting to receive a new commitment from the peer. We are also still expected to send our own new commitment.

  1. We send a new commitment, moving our HTLC 35771 into the SENT_REMOVE_ACK_COMMIT state:
2025-03-21T04:13:40.055Z DEBUG   03…4c-channeld-chan#255714: Trying commit
2025-03-21T04:13:40.055Z DEBUG   03…4c-channeld-chan#255714: htlc 35771: SENT_REMOVE_REVOCATION->SENT_REMOVE_ACK_COMMIT
2025-03-21T04:13:40.055Z DEBUG   03…4c-channeld-chan#255714: htlc added REMOTE: local 1193290454 remote 229569507
2025-03-21T04:13:40.055Z DEBUG   03…4c-channeld-chan#255714: -> local 1200044891 remote 229569507
2025-03-21T04:13:40.055Z DEBUG   03…4c-channeld-chan#255714: sending_commit: HTLC LOCAL 35771 = SENT_REMOVE_ACK_COMMIT/RCVD_REMOVE_ACK_COMMIT FAILED
2025-03-21T04:13:40.066Z DEBUG   03…4c-hsmd: Got WIRE_HSMD_SIGN_REMOTE_COMMITMENT_TX
2025-03-21T04:13:40.067Z DEBUG   03…4c-hsmd: Got WIRE_HSMD_SIGN_REMOTE_HTLC_TX
2025-03-21T04:13:40.067Z DEBUG   03…4c-hsmd: Got WIRE_HSMD_SIGN_REMOTE_HTLC_TX
2025-03-21T04:13:40.067Z DEBUG   03…4c-hsmd: Got WIRE_HSMD_SIGN_REMOTE_HTLC_TX
2025-03-21T04:13:40.067Z DEBUG   03…4c-channeld-chan#255714: Creating commit_sig signature 97040 30…01 for tx 02…20 wscript 52…ae key 02…55
2025-03-21T04:13:40.067Z DEBUG   03…4c-channeld-chan#255714: Creating HTLC signature 30…83 for tx 02…00 wscript 76…68 key 02…38
2025-03-21T04:13:40.067Z DEBUG   03…4c-channeld-chan#255714: Creating HTLC signature 30…83 for tx 02…00 wscript 76…68 key 02…38
2025-03-21T04:13:40.068Z DEBUG   03…4c-channeld-chan#255714: Creating HTLC signature 30…83 for tx 02…00 wscript 76…68 key 02…38
2025-03-21T04:13:40.068Z DEBUG   03…4c-channeld-chan#255714: Telling master we're about to commit...
2025-03-21T04:13:40.068Z DEBUG   03…4c-channeld-chan#255714: Sending master 1020
2025-03-21T04:13:40.068Z DEBUG   03…4c-chan#255714: HTLC out 35771 SENT_REMOVE_REVOCATION->SENT_REMOVE_ACK_COMMIT
2025-03-21T04:13:40.077Z DEBUG   03…4c-channeld-chan#255714: ... , awaiting 1120
2025-03-21T04:13:40.083Z DEBUG   03…4c-channeld-chan#255714: Got it!
2025-03-21T04:13:40.083Z DEBUG   03…4c-channeld-chan#255714: Sending commit_sig with 3 htlc sigs
2025-03-21T04:13:40.087Z DEBUG   03…4c-channeld-chan#255714: peer_out WIRE_COMMITMENT_SIGNED
2025-03-21T04:13:40.094Z DEBUG   03…4c-channeld-chan#255714: Can't send commit: waiting for revoke_and_ack

Now we are expecting to receive a revoke_and_ack from the peer. We are also still expecting to receive a new commitment from the peer.


  1. We decide to forward a payment over the channel, so we add HTLC 35774 in the SENT_ADD_HTLC state:
2025-03-21T04:13:51.023Z DEBUG   02…fa-chan#255693: Decided to forward htlc_id=48486 over channel with scid=…50x…58x0 with peer 03…4c
2025-03-21T04:13:51.040Z DEBUG   03…4c-channeld-chan#255714: NEW:: HTLC LOCAL 35774 = SENT_ADD_HTLC/RCVD_ADD_HTLC 
2025-03-21T04:13:51.041Z DEBUG   03…4c-channeld-chan#255714: Adding HTLC 35774 amount=10260225msat cltv=889421 gave CHANNEL_ERR_ADD_OK
2025-03-21T04:13:51.041Z DEBUG   03…4c-channeld-chan#255714: peer_out WIRE_UPDATE_ADD_HTLC
2025-03-21T04:13:51.041Z DEBUG   03…4c-channeld-chan#255714: REPLY WIRE_CHANNELD_OFFER_HTLC_REPLY with 0 fds
2025-03-21T04:13:51.050Z DEBUG   03…4c-channeld-chan#255714: Can't send commit: waiting for revoke_and_ack

Now we are expected to send a new commitment, but we can't because we are still waiting to receive a revoke_and_ack from the peer. We are also still expecting to receive a new commitment from the peer.


  1. We time out while waiting to receive a revoke_and_ack from the peer, failing our HTLC 35774 and disconnecting:
2025-03-21T04:14:21.039Z UNUSUAL 03…4c-channeld-chan#255714: Adding HTLC 35774 too slow: killing connection
2025-03-21T04:14:21.039Z INFO    03…4c-chan#255714: Peer transient failure in CHANNELD_NORMAL: Adding HTLC timed out: killed connection
2025-03-21T04:14:21.039Z DEBUG   03…4c-channeld-chan#255714: Status closed, but not exited. Killing
2025-03-21T04:14:21.040Z DEBUG   03…4c-chan#255714: Failing HTLC 35774 due to peer death
2025-03-21T04:14:21.048Z DEBUG   03…4c-connectd: disconnect_peer
2025-03-21T04:14:21.048Z DEBUG   03…4c-lightningd: peer_disconnect_done
2025-03-21T04:14:21.049Z DEBUG   03…4c-connectd: Will try reconnect in 1 seconds
2025-03-21T04:14:21.049Z DEBUG   plugin-funder: Cleaning up inflights for peer id 03…4c

This has the effect of reverting our HTLC 35772 back to the SENT_ADD_ACK_REVOCATION state and dropping our HTLC 35774.

@whitslack
Copy link
Collaborator Author

The peer reconnects to us, and we reëstablish the channel with our HTLC 35771 in the SENT_REMOVE_ACK_COMMIT state and our HTLCs 35772 and 35773 in the SENT_ADD_ACK_REVOCATION state:

2025-03-21T04:14:22.997Z DEBUG   03…4c-connectd: Failed connected out: All addresses failed: 4ew…d.onion:9735: need a proxy. 
2025-03-21T04:14:22.997Z DEBUG   03…4c-connectd: Will try reconnect in 2 seconds
2025-03-21T04:14:25.444Z DEBUG   03…4c-connectd: Failed connected out: All addresses failed: 4ew…d.onion:9735: need a proxy. 
2025-03-21T04:14:25.444Z DEBUG   03…4c-connectd: Will try reconnect in 4 seconds
2025-03-21T04:14:30.090Z DEBUG   03…4c-connectd: Failed connected out: All addresses failed: 4ew…d.onion:9735: need a proxy. 
2025-03-21T04:14:30.090Z DEBUG   03…4c-connectd: Will try reconnect in 8 seconds
2025-03-21T04:14:32.315Z DEBUG   03…4c-connectd: Connect IN
2025-03-21T04:14:32.315Z DEBUG   03…4c-connectd: peer_out WIRE_INIT
2025-03-21T04:14:32.324Z DEBUG   03…4c-connectd: peer_in WIRE_INIT
2025-03-21T04:14:32.324Z DEBUG   03…4c-connectd: Handed peer, entering loop
2025-03-21T04:14:32.324Z DEBUG   03…4c-gossipd: seeker: disabling gossip
2025-03-21T04:14:32.324Z DEBUG   03…4c-chan#255714: Peer has reconnected, state CHANNELD_NORMAL: connecting subd
2025-03-21T04:14:32.351Z DEBUG   03…4c-channeld-chan#255714: pid 26348, msgfd 179
2025-03-21T04:14:32.351Z DEBUG   03…4c-chan#255714: Already have funding locked in
2025-03-21T04:14:32.353Z DEBUG   03…4c-connectd: peer_out WIRE_GOSSIP_TIMESTAMP_FILTER
2025-03-21T04:14:32.356Z DEBUG   03…4c-channeld-chan#255714: option_static_remotekey = 1, option_anchor_outputs = 0 option_anchors_zero_fee_htlc_tx = 1
2025-03-21T04:14:32.356Z DEBUG   03…4c-channeld-chan#255714: init REMOTE: remote_per_commit = 02…97, old_remote_per_commit = 03…cd next_idx_local = 97036 next_idx_remote = 97041 revocations_received = 97039 feerates { RCVD_ADD_ACK_REVOCATION:864 } range 253-12690 blockheights { RCVD_ADD_ACK_REVOCATION:0 }, our current 888712
2025-03-21T04:14:32.357Z DEBUG   03…4c-channeld-chan#255714: Restoring HTLC 0/4: id=35773 amount=69663541msat cltv=888979 payment_hash=71…10 
2025-03-21T04:14:32.357Z DEBUG   03…4c-channeld-chan#255714: NEW:: HTLC LOCAL 35773 = SENT_ADD_ACK_REVOCATION/RCVD_ADD_ACK_REVOCATION 
2025-03-21T04:14:32.357Z DEBUG   03…4c-channeld-chan#255714: Restoring HTLC 1/4: id=35771 amount=6754437msat cltv=889173 payment_hash=49…0d (failed)
2025-03-21T04:14:32.357Z DEBUG   03…4c-channeld-chan#255714: NEW:: HTLC LOCAL 35771 = SENT_REMOVE_ACK_COMMIT/RCVD_REMOVE_ACK_COMMIT 
2025-03-21T04:14:32.357Z DEBUG   03…4c-channeld-chan#255714: Restoring HTLC 2/4: id=35719 amount=214094428msat cltv=889003 payment_hash=bd…ca 
2025-03-21T04:14:32.357Z DEBUG   03…4c-channeld-chan#255714: NEW:: HTLC LOCAL 35719 = SENT_ADD_ACK_REVOCATION/RCVD_ADD_ACK_REVOCATION 
2025-03-21T04:14:32.357Z DEBUG   03…4c-channeld-chan#255714: Restoring HTLC 3/4: id=35772 amount=286627633msat cltv=889025 payment_hash=48…ad 
2025-03-21T04:14:32.357Z DEBUG   03…4c-channeld-chan#255714: NEW:: HTLC LOCAL 35772 = SENT_ADD_ACK_REVOCATION/RCVD_ADD_ACK_REVOCATION 
2025-03-21T04:14:32.358Z DEBUG   03…4c-channeld-chan#255714: peer_out WIRE_CHANNEL_REESTABLISH
2025-03-21T04:14:32.358Z DEBUG   03…4c-channeld-chan#255714: billboard: Sent reestablish, waiting for theirs
2025-03-21T04:14:32.358Z DEBUG   03…4c-hsmd: Got WIRE_HSMD_GET_PER_COMMITMENT_POINT
2025-03-21T04:14:32.358Z DEBUG   03…4c-hsmd: Got WIRE_HSMD_GET_PER_COMMITMENT_POINT

We are still expecting to receive a revoke_and_ack from the peer.


We receive a channel_reestablish from the peer and retransmit our last commitment:

2025-03-21T04:14:32.399Z DEBUG   03…4c-hsmd: Got WIRE_HSMD_REVOKE_COMMITMENT_TX
2025-03-21T04:14:32.399Z DEBUG   03…4c-hsmd: Got WIRE_HSMD_SIGN_REMOTE_COMMITMENT_TX
2025-03-21T04:14:32.400Z DEBUG   03…4c-hsmd: Got WIRE_HSMD_SIGN_REMOTE_HTLC_TX
2025-03-21T04:14:32.400Z DEBUG   03…4c-hsmd: Got WIRE_HSMD_SIGN_REMOTE_HTLC_TX
2025-03-21T04:14:32.400Z DEBUG   03…4c-hsmd: Got WIRE_HSMD_SIGN_REMOTE_HTLC_TX
2025-03-21T04:14:32.400Z DEBUG   03…4c-channeld-chan#255714: peer_in WIRE_CHANNEL_REESTABLISH
2025-03-21T04:14:32.400Z DEBUG   03…4c-channeld-chan#255714: Got reestablish commit=97040 revoke=97035 inflights: 0, active splices: 0
2025-03-21T04:14:32.401Z DEBUG   03…4c-channeld-chan#255714: Splice resume check with local_next_funding: omitted, remote_next_funding: empty, inflights: 0
2025-03-21T04:14:32.401Z DEBUG   03…4c-channeld-chan#255714: next_revocation_number = 97035
2025-03-21T04:14:32.401Z DEBUG   03…4c-channeld-chan#255714: option_static_remotekey: fields are correct
2025-03-21T04:14:32.401Z DEBUG   03…4c-channeld-chan#255714: Retransmitting commitment, feerate LOCAL=864 REMOTE=864, blockheight LOCAL=0 REMOTE=0
2025-03-21T04:14:32.401Z DEBUG   03…4c-channeld-chan#255714: Creating commit_sig signature 97040 30…01 for tx 02…20 wscript 52…ae key 02…55
2025-03-21T04:14:32.401Z DEBUG   03…4c-channeld-chan#255714: Creating HTLC signature 30…83 for tx 02…00 wscript 76…68 key 02…38
2025-03-21T04:14:32.401Z DEBUG   03…4c-channeld-chan#255714: Creating HTLC signature 30…83 for tx 02…00 wscript 76…68 key 02…38
2025-03-21T04:14:32.401Z DEBUG   03…4c-channeld-chan#255714: Creating HTLC signature 30…83 for tx 02…00 wscript 76…68 key 02…38
2025-03-21T04:14:32.401Z DEBUG   03…4c-channeld-chan#255714: peer_out WIRE_COMMITMENT_SIGNED
2025-03-21T04:14:32.401Z DEBUG   03…4c-channeld-chan#255714: No upgrade: we retransmitted
2025-03-21T04:14:32.401Z DEBUG   03…4c-chan#255714: channel_gossip: reestablished
2025-03-21T04:14:32.401Z DEBUG   03…4c-channeld-chan#255714: billboard perm: Reconnected, and reestablished.
2025-03-21T04:14:32.402Z DEBUG   03…4c-channeld-chan#255714: billboard: Channel ready for use.
2025-03-21T04:14:32.402Z DEBUG   03…4c-channeld-chan#255714: billboard: Channel ready for use.
2025-03-21T04:14:32.406Z DEBUG   03…4c-channeld-chan#255714: Can't send commit: waiting for revoke_and_ack

We are still expecting to receive a revoke_and_ack from the peer.


  1. We decide to forward a payment over the channel, so we add HTLC 35774 in the SENT_ADD_HTLC state:
2025-03-21T04:16:11.620Z DEBUG   03…48-chan#255636: Decided to forward htlc_id=33771 over channel with scid=…50x…58x0 with peer 03…4c
2025-03-21T04:16:11.633Z DEBUG   03…4c-channeld-chan#255714: NEW:: HTLC LOCAL 35774 = SENT_ADD_HTLC/RCVD_ADD_HTLC 
2025-03-21T04:16:11.633Z DEBUG   03…4c-channeld-chan#255714: Adding HTLC 35774 amount=100066505msat cltv=888997 gave CHANNEL_ERR_ADD_OK
2025-03-21T04:16:11.633Z DEBUG   03…4c-channeld-chan#255714: peer_out WIRE_UPDATE_ADD_HTLC
2025-03-21T04:16:11.633Z DEBUG   03…4c-channeld-chan#255714: REPLY WIRE_CHANNELD_OFFER_HTLC_REPLY with 0 fds
2025-03-21T04:16:11.646Z DEBUG   03…4c-channeld-chan#255714: Can't send commit: waiting for revoke_and_ack

Now we are expected to send a new commitment, but we can't because we are still waiting to receive a revoke_and_ack from the peer.


  1. We time out while waiting to receive a revoke_and_ack from the peer, failing our HTLC 35774 and disconnecting:
2025-03-21T04:16:41.634Z UNUSUAL 03…4c-channeld-chan#255714: Adding HTLC 35774 too slow: killing connection
2025-03-21T04:16:41.634Z INFO    03…4c-chan#255714: Peer transient failure in CHANNELD_NORMAL: Adding HTLC timed out: killed connection
2025-03-21T04:16:41.634Z DEBUG   03…4c-channeld-chan#255714: Status closed, but not exited. Killing
2025-03-21T04:16:41.634Z DEBUG   03…4c-chan#255714: Failing HTLC 35774 due to peer death
2025-03-21T04:16:41.647Z DEBUG   03…4c-connectd: disconnect_peer
2025-03-21T04:16:41.647Z DEBUG   03…4c-lightningd: peer_disconnect_done
2025-03-21T04:16:41.647Z DEBUG   03…4c-connectd: Will try reconnect in 16 seconds
2025-03-21T04:16:41.647Z DEBUG   plugin-funder: Cleaning up inflights for peer id 03…4c

This has the effect of dropping our HTLC 35774.


The peer reconnects to us, and we reëstablish the channel with our HTLC 35771 in the SENT_REMOVE_ACK_COMMIT state and our HTLCs 35772 and 35773 in the SENT_ADD_ACK_REVOCATION state:

2025-03-21T04:16:58.242Z DEBUG   03…4c-connectd: Failed connected out: All addresses failed: 4ew…d.onion:9735: need a proxy. 
2025-03-21T04:16:58.242Z DEBUG   03…4c-connectd: Will try reconnect in 32 seconds
2025-03-21T04:17:30.974Z DEBUG   03…4c-connectd: Failed connected out: All addresses failed: 4ew…d.onion:9735: need a proxy. 
2025-03-21T04:17:30.974Z DEBUG   03…4c-connectd: Will try reconnect in 64 seconds
2025-03-21T04:17:34.819Z DEBUG   03…4c-connectd: Connect IN
2025-03-21T04:17:34.819Z DEBUG   03…4c-connectd: peer_out WIRE_INIT
2025-03-21T04:17:34.860Z DEBUG   03…4c-connectd: peer_in WIRE_INIT
2025-03-21T04:17:34.860Z DEBUG   03…4c-connectd: Handed peer, entering loop
2025-03-21T04:17:34.860Z DEBUG   03…4c-gossipd: seeker: disabling gossip
2025-03-21T04:17:34.861Z DEBUG   03…4c-chan#255714: Peer has reconnected, state CHANNELD_NORMAL: connecting subd
2025-03-21T04:17:34.887Z DEBUG   03…4c-channeld-chan#255714: pid 26683, msgfd 179
2025-03-21T04:17:34.887Z DEBUG   03…4c-chan#255714: Already have funding locked in
2025-03-21T04:17:34.888Z DEBUG   03…4c-connectd: peer_out WIRE_GOSSIP_TIMESTAMP_FILTER
2025-03-21T04:17:34.893Z DEBUG   03…4c-channeld-chan#255714: option_static_remotekey = 1, option_anchor_outputs = 0 option_anchors_zero_fee_htlc_tx = 1
2025-03-21T04:17:34.893Z DEBUG   03…4c-channeld-chan#255714: init REMOTE: remote_per_commit = 02…97, old_remote_per_commit = 03…cd next_idx_local = 97036 next_idx_remote = 97041 revocations_received = 97039 feerates { RCVD_ADD_ACK_REVOCATION:864 } range 253-12690 blockheights { RCVD_ADD_ACK_REVOCATION:0 }, our current 888712
2025-03-21T04:17:34.893Z DEBUG   03…4c-hsmd: Got WIRE_HSMD_GET_PER_COMMITMENT_POINT
2025-03-21T04:17:34.894Z DEBUG   03…4c-channeld-chan#255714: Restoring HTLC 0/4: id=35773 amount=69663541msat cltv=888979 payment_hash=71…10 
2025-03-21T04:17:34.894Z DEBUG   03…4c-channeld-chan#255714: NEW:: HTLC LOCAL 35773 = SENT_ADD_ACK_REVOCATION/RCVD_ADD_ACK_REVOCATION 
2025-03-21T04:17:34.894Z DEBUG   03…4c-channeld-chan#255714: Restoring HTLC 1/4: id=35771 amount=6754437msat cltv=889173 payment_hash=49…0d (failed)
2025-03-21T04:17:34.894Z DEBUG   03…4c-channeld-chan#255714: NEW:: HTLC LOCAL 35771 = SENT_REMOVE_ACK_COMMIT/RCVD_REMOVE_ACK_COMMIT 
2025-03-21T04:17:34.894Z DEBUG   03…4c-channeld-chan#255714: Restoring HTLC 2/4: id=35719 amount=214094428msat cltv=889003 payment_hash=bd…ca 
2025-03-21T04:17:34.894Z DEBUG   03…4c-channeld-chan#255714: NEW:: HTLC LOCAL 35719 = SENT_ADD_ACK_REVOCATION/RCVD_ADD_ACK_REVOCATION 
2025-03-21T04:17:34.894Z DEBUG   03…4c-channeld-chan#255714: Restoring HTLC 3/4: id=35772 amount=286627633msat cltv=889025 payment_hash=48…ad 
2025-03-21T04:17:34.894Z DEBUG   03…4c-channeld-chan#255714: NEW:: HTLC LOCAL 35772 = SENT_ADD_ACK_REVOCATION/RCVD_ADD_ACK_REVOCATION 
2025-03-21T04:17:34.894Z DEBUG   03…4c-channeld-chan#255714: peer_out WIRE_CHANNEL_REESTABLISH
2025-03-21T04:17:34.894Z DEBUG   03…4c-channeld-chan#255714: billboard: Sent reestablish, waiting for theirs
2025-03-21T04:17:34.895Z DEBUG   03…4c-hsmd: Got WIRE_HSMD_GET_PER_COMMITMENT_POINT

We are still expecting to receive a revoke_and_ack from the peer.


We receive a channel_reestablish from the peer and retransmit our last commitment:

2025-03-21T04:17:44.756Z DEBUG   03…4c-channeld-chan#255714: peer_in WIRE_CHANNEL_REESTABLISH
2025-03-21T04:17:44.756Z DEBUG   03…4c-channeld-chan#255714: Got reestablish commit=97040 revoke=97035 inflights: 0, active splices: 0
2025-03-21T04:17:44.756Z DEBUG   03…4c-channeld-chan#255714: Splice resume check with local_next_funding: omitted, remote_next_funding: empty, inflights: 0
2025-03-21T04:17:44.756Z DEBUG   03…4c-channeld-chan#255714: next_revocation_number = 97035
2025-03-21T04:17:44.756Z DEBUG   03…4c-channeld-chan#255714: option_static_remotekey: fields are correct
2025-03-21T04:17:44.756Z DEBUG   03…4c-channeld-chan#255714: Retransmitting commitment, feerate LOCAL=864 REMOTE=864, blockheight LOCAL=0 REMOTE=0
2025-03-21T04:17:44.757Z DEBUG   03…4c-hsmd: Got WIRE_HSMD_REVOKE_COMMITMENT_TX
2025-03-21T04:17:44.758Z DEBUG   03…4c-hsmd: Got WIRE_HSMD_SIGN_REMOTE_COMMITMENT_TX
2025-03-21T04:17:44.758Z DEBUG   03…4c-channeld-chan#255714: Creating commit_sig signature 97040 30…01 for tx 02…20 wscript 52…ae key 02…55
2025-03-21T04:17:44.759Z DEBUG   03…4c-hsmd: Got WIRE_HSMD_SIGN_REMOTE_HTLC_TX
2025-03-21T04:17:44.759Z DEBUG   03…4c-channeld-chan#255714: Creating HTLC signature 30…83 for tx 02…00 wscript 76…68 key 02…38
2025-03-21T04:17:44.760Z DEBUG   03…4c-hsmd: Got WIRE_HSMD_SIGN_REMOTE_HTLC_TX
2025-03-21T04:17:44.760Z DEBUG   03…4c-channeld-chan#255714: Creating HTLC signature 30…83 for tx 02…00 wscript 76…68 key 02…38
2025-03-21T04:17:44.761Z DEBUG   03…4c-hsmd: Got WIRE_HSMD_SIGN_REMOTE_HTLC_TX
2025-03-21T04:17:44.762Z DEBUG   03…4c-channeld-chan#255714: Creating HTLC signature 30…83 for tx 02…00 wscript 76…68 key 02…38
2025-03-21T04:17:44.762Z DEBUG   03…4c-channeld-chan#255714: peer_out WIRE_COMMITMENT_SIGNED
2025-03-21T04:17:44.762Z DEBUG   03…4c-channeld-chan#255714: No upgrade: we retransmitted
2025-03-21T04:17:44.762Z DEBUG   03…4c-chan#255714: channel_gossip: reestablished
2025-03-21T04:17:44.762Z DEBUG   03…4c-channeld-chan#255714: billboard perm: Reconnected, and reestablished.
2025-03-21T04:17:44.762Z DEBUG   03…4c-channeld-chan#255714: billboard: Channel ready for use.
2025-03-21T04:17:44.763Z DEBUG   03…4c-channeld-chan#255714: billboard: Channel ready for use.
2025-03-21T04:17:44.771Z DEBUG   03…4c-channeld-chan#255714: Can't send commit: waiting for revoke_and_ack

We are still expecting to receive a revoke_and_ack from the peer.


  1. We decide to forward a payment over the channel, so we add HTLC 35774 in the SENT_ADD_HTLC state:
2025-03-21T04:17:56.483Z DEBUG   03…b8-chan#255503: Decided to forward htlc_id=102506 over channel with scid=…50x…58x0 with peer 03…4c
2025-03-21T04:17:56.502Z DEBUG   03…4c-channeld-chan#255714: NEW:: HTLC LOCAL 35774 = SENT_ADD_HTLC/RCVD_ADD_HTLC 
2025-03-21T04:17:56.502Z DEBUG   03…4c-channeld-chan#255714: Adding HTLC 35774 amount=248242235msat cltv=888975 gave CHANNEL_ERR_ADD_OK
2025-03-21T04:17:56.502Z DEBUG   03…4c-channeld-chan#255714: peer_out WIRE_UPDATE_ADD_HTLC
2025-03-21T04:17:56.502Z DEBUG   03…4c-channeld-chan#255714: REPLY WIRE_CHANNELD_OFFER_HTLC_REPLY with 0 fds
2025-03-21T04:17:56.511Z DEBUG   03…4c-channeld-chan#255714: Can't send commit: waiting for revoke_and_ack

Now we are expected to send a new commitment, but we can't because we are still waiting to receive a revoke_and_ack from the peer.


  1. The peer fails our HTLC 35772, moving it into the RCVD_REMOVE_HTLC state:
2025-03-21T04:18:19.818Z DEBUG   03…4c-channeld-chan#255714: peer_in WIRE_UPDATE_FAIL_HTLC
2025-03-21T04:18:19.818Z DEBUG   03…4c-channeld-chan#255714: FAIL:: HTLC LOCAL 35772 = RCVD_REMOVE_HTLC/SENT_REMOVE_HTLC 
2025-03-21T04:18:19.828Z DEBUG   03…4c-channeld-chan#255714: Can't send commit: waiting for revoke_and_ack

Now we are expecting to receive a new commitment from the peer. We are also still expected to send our own new commitment, but we can't because we are still waiting to receive a revoke_and_ack from the peer.


  1. We time out while waiting to receive a revoke_and_ack from the peer, failing our HTLC 35774 and disconnecting:
2025-03-21T04:18:26.504Z UNUSUAL 03…4c-channeld-chan#255714: Adding HTLC 35774 too slow: killing connection
2025-03-21T04:18:26.504Z INFO    03…4c-chan#255714: Peer transient failure in CHANNELD_NORMAL: Adding HTLC timed out: killed connection
2025-03-21T04:18:26.504Z DEBUG   03…4c-channeld-chan#255714: Status closed, but not exited. Killing
2025-03-21T04:18:26.505Z DEBUG   03…4c-chan#255714: Failing HTLC 35774 due to peer death
2025-03-21T04:18:26.511Z DEBUG   03…4c-connectd: disconnect_peer
2025-03-21T04:18:26.511Z DEBUG   03…4c-lightningd: peer_disconnect_done
2025-03-21T04:18:26.511Z DEBUG   03…4c-connectd: Will try reconnect in 128 seconds
2025-03-21T04:18:26.512Z DEBUG   plugin-funder: Cleaning up inflights for peer id 03…4c

This has the effect of reverting our HTLC 35772 back to the SENT_ADD_ACK_REVOCATION state and dropping our HTLC 35774.


The peer reconnects to us, and we reëstablish the channel with our HTLC 35771 in the SENT_REMOVE_ACK_COMMIT state and our HTLCs 35772 and 35773 in the SENT_ADD_ACK_REVOCATION state:

2025-03-21T04:18:35.467Z DEBUG   03…4c-connectd: Failed connected out: All addresses failed: 4ew…d.onion:9735: need a proxy. 
2025-03-21T04:18:35.468Z DEBUG   03…4c-connectd: Will try reconnect in 256 seconds
2025-03-21T04:19:00.354Z DEBUG   03…4c-connectd: Connect IN
2025-03-21T04:19:00.354Z DEBUG   03…4c-connectd: peer_out WIRE_INIT
2025-03-21T04:19:00.362Z DEBUG   03…4c-connectd: peer_in WIRE_INIT
2025-03-21T04:19:00.362Z DEBUG   03…4c-connectd: Handed peer, entering loop
2025-03-21T04:19:00.362Z DEBUG   03…4c-gossipd: seeker: disabling gossip
2025-03-21T04:19:00.362Z DEBUG   03…4c-chan#255714: Peer has reconnected, state CHANNELD_NORMAL: connecting subd
2025-03-21T04:19:00.389Z DEBUG   03…4c-channeld-chan#255714: pid 26810, msgfd 179
2025-03-21T04:19:00.389Z DEBUG   03…4c-chan#255714: Already have funding locked in
2025-03-21T04:19:00.390Z DEBUG   03…4c-connectd: peer_out WIRE_GOSSIP_TIMESTAMP_FILTER
2025-03-21T04:19:00.393Z DEBUG   03…4c-channeld-chan#255714: option_static_remotekey = 1, option_anchor_outputs = 0 option_anchors_zero_fee_htlc_tx = 1
2025-03-21T04:19:00.393Z DEBUG   03…4c-channeld-chan#255714: init REMOTE: remote_per_commit = 02…97, old_remote_per_commit = 03…cd next_idx_local = 97036 next_idx_remote = 97041 revocations_received = 97039 feerates { RCVD_ADD_ACK_REVOCATION:864 } range 253-12690 blockheights { RCVD_ADD_ACK_REVOCATION:0 }, our current 888712
2025-03-21T04:19:00.393Z DEBUG   03…4c-hsmd: Got WIRE_HSMD_GET_PER_COMMITMENT_POINT
2025-03-21T04:19:00.394Z DEBUG   03…4c-channeld-chan#255714: Restoring HTLC 0/4: id=35773 amount=69663541msat cltv=888979 payment_hash=71…10 
2025-03-21T04:19:00.394Z DEBUG   03…4c-channeld-chan#255714: NEW:: HTLC LOCAL 35773 = SENT_ADD_ACK_REVOCATION/RCVD_ADD_ACK_REVOCATION 
2025-03-21T04:19:00.394Z DEBUG   03…4c-channeld-chan#255714: Restoring HTLC 1/4: id=35771 amount=6754437msat cltv=889173 payment_hash=49…0d (failed)
2025-03-21T04:19:00.394Z DEBUG   03…4c-channeld-chan#255714: NEW:: HTLC LOCAL 35771 = SENT_REMOVE_ACK_COMMIT/RCVD_REMOVE_ACK_COMMIT 
2025-03-21T04:19:00.394Z DEBUG   03…4c-channeld-chan#255714: Restoring HTLC 2/4: id=35719 amount=214094428msat cltv=889003 payment_hash=bd…ca 
2025-03-21T04:19:00.394Z DEBUG   03…4c-channeld-chan#255714: NEW:: HTLC LOCAL 35719 = SENT_ADD_ACK_REVOCATION/RCVD_ADD_ACK_REVOCATION 
2025-03-21T04:19:00.394Z DEBUG   03…4c-channeld-chan#255714: Restoring HTLC 3/4: id=35772 amount=286627633msat cltv=889025 payment_hash=48…ad 
2025-03-21T04:19:00.394Z DEBUG   03…4c-channeld-chan#255714: NEW:: HTLC LOCAL 35772 = SENT_ADD_ACK_REVOCATION/RCVD_ADD_ACK_REVOCATION 
2025-03-21T04:19:00.395Z DEBUG   03…4c-channeld-chan#255714: peer_out WIRE_CHANNEL_REESTABLISH
2025-03-21T04:19:00.395Z DEBUG   03…4c-channeld-chan#255714: billboard: Sent reestablish, waiting for theirs
2025-03-21T04:19:00.395Z DEBUG   03…4c-hsmd: Got WIRE_HSMD_GET_PER_COMMITMENT_POINT

We are still expecting to receive a revoke_and_ack from the peer.


We receive a channel_reestablish from the peer and retransmit our last commitment:

2025-03-21T04:19:03.414Z DEBUG   03…4c-hsmd: Got WIRE_HSMD_REVOKE_COMMITMENT_TX
2025-03-21T04:19:03.414Z DEBUG   03…4c-channeld-chan#255714: peer_in WIRE_CHANNEL_REESTABLISH
2025-03-21T04:19:03.414Z DEBUG   03…4c-channeld-chan#255714: Got reestablish commit=97041 revoke=97035 inflights: 0, active splices: 0
2025-03-21T04:19:03.415Z DEBUG   03…4c-channeld-chan#255714: Splice resume check with local_next_funding: omitted, remote_next_funding: empty, inflights: 0
2025-03-21T04:19:03.415Z DEBUG   03…4c-channeld-chan#255714: next_revocation_number = 97035
2025-03-21T04:19:03.415Z DEBUG   03…4c-channeld-chan#255714: option_static_remotekey: fields are correct
2025-03-21T04:19:03.415Z DEBUG   03…4c-channeld-chan#255714: No upgrade: no next_to_send received
2025-03-21T04:19:03.415Z DEBUG   03…4c-chan#255714: channel_gossip: reestablished
2025-03-21T04:19:03.415Z DEBUG   03…4c-channeld-chan#255714: billboard perm: Reconnected, and reestablished.
2025-03-21T04:19:03.415Z DEBUG   03…4c-channeld-chan#255714: billboard: Channel ready for use.
2025-03-21T04:19:03.415Z DEBUG   03…4c-channeld-chan#255714: billboard: Channel ready for use.
2025-03-21T04:19:03.415Z DEBUG   03…4c-channeld-chan#255714: Can't send commit: waiting for revoke_and_ack

We are still expecting to receive a revoke_and_ack from the peer.


  1. The peer fails our HTLC 35772, moving it into the RCVD_REMOVE_HTLC state:
2025-03-21T04:19:07.402Z DEBUG   03…4c-channeld-chan#255714: peer_in WIRE_UPDATE_FAIL_HTLC
2025-03-21T04:19:07.402Z DEBUG   03…4c-channeld-chan#255714: FAIL:: HTLC LOCAL 35772 = RCVD_REMOVE_HTLC/SENT_REMOVE_HTLC 

Now we are expecting to receive a new commitment from the peer. We are also still expecting to receive a revoke_and_ack from the peer.


  1. We receive a new commitment from the peer, moving our HTLC 35772 into the RCVD_REMOVE_COMMIT state:
2025-03-21T04:19:07.402Z DEBUG   03…4c-channeld-chan#255714: peer_in WIRE_COMMITMENT_SIGNED
2025-03-21T04:19:07.402Z DEBUG   03…4c-channeld-chan#255714: handle_peer_commit_sig(splice: 0, remote_splice: 0, index: 97036)
2025-03-21T04:19:07.402Z DEBUG   03…4c-channeld-chan#255714: Received commit
2025-03-21T04:19:07.402Z DEBUG   03…4c-channeld-chan#255714: htlc 35772: RCVD_REMOVE_HTLC->RCVD_REMOVE_COMMIT
2025-03-21T04:19:07.402Z DEBUG   03…4c-channeld-chan#255714: htlc added LOCAL: local 1200044891 remote 229569507
2025-03-21T04:19:07.403Z DEBUG   03…4c-channeld-chan#255714: -> local 1486672524 remote 229569507
2025-03-21T04:19:07.403Z DEBUG   03…4c-channeld-chan#255714: rcvd_commit: HTLC LOCAL 35772 = RCVD_REMOVE_COMMIT/SENT_REMOVE_COMMIT FAILED
2025-03-21T04:19:07.403Z DEBUG   03…4c-channeld-chan#255714: Feerates are 864/864
2025-03-21T04:19:07.403Z DEBUG   03…4c-channeld-chan#255714: Committed 2, removing 0, adding 0
2025-03-21T04:19:07.403Z DEBUG   03…4c-channeld-chan#255714: We need 21928sat at feerate 864 for 2 untrimmed htlcs: we have 229569507msat/229569507msat (will have 229569507msat)
2025-03-21T04:19:07.403Z DEBUG   03…4c-channeld-chan#255714: Derived key 03…58 from basepoint 03…93, point 03…eb
2025-03-21T04:19:07.410Z DEBUG   03…4c-hsmd: Got WIRE_HSMD_VALIDATE_COMMITMENT_TX
2025-03-21T04:19:07.410Z DEBUG   03…4c-channeld-chan#255714: Received commit_sig with 2 htlc sigs

Now we are expected to send a revoke_and_ack and then our own new commitment. We are also still expecting to receive a revoke_and_ack from the peer.

  1. We send the expected revoke_and_ack, moving our HTLC 35772 into the SENT_REMOVE_REVOCATION state:
2025-03-21T04:19:07.410Z DEBUG   03…4c-channeld-chan#255714: Sending revoke_and_ack
2025-03-21T04:19:07.410Z DEBUG   03…4c-channeld-chan#255714: htlc 35772: RCVD_REMOVE_COMMIT->SENT_REMOVE_REVOCATION
2025-03-21T04:19:07.411Z DEBUG   03…4c-channeld-chan#255714: sending_revoke_and_ack: HTLC LOCAL 35772 = SENT_REMOVE_REVOCATION/RCVD_REMOVE_REVOCATION FAILED
2025-03-21T04:19:07.411Z DEBUG   03…4c-channeld-chan#255714: revoke_and_ack made pending: commit timer
2025-03-21T04:19:07.411Z DEBUG   03…4c-channeld-chan#255714: Sending master 1021
2025-03-21T04:19:07.411Z DEBUG   03…4c-chan#255714: got commitsig 97036: feerate 864, blockheight: 0, 0 added, 0 fulfilled, 1 failed, 0 changed. 0 splice commitments.
2025-03-21T04:19:07.412Z DEBUG   03…4c-chan#255714: HTLC out 35772 SENT_ADD_ACK_REVOCATION->RCVD_REMOVE_COMMIT
2025-03-21T04:19:07.412Z DEBUG   03…4c-chan#255714: Our HTLC 35772 failed (4294967295)
2025-03-21T04:19:07.413Z DEBUG   03…4c-chan#255714: HTLC out 35772 RCVD_REMOVE_COMMIT->SENT_REMOVE_REVOCATION
2025-03-21T04:19:07.425Z DEBUG   03…4c-channeld-chan#255714: ... , awaiting 1121
2025-03-21T04:19:07.426Z DEBUG   03…4c-channeld-chan#255714: Got it!
2025-03-21T04:19:07.426Z DEBUG   03…4c-hsmd: Got WIRE_HSMD_REVOKE_COMMITMENT_TX
2025-03-21T04:19:07.427Z DEBUG   03…4c-channeld-chan#255714: peer_out WIRE_REVOKE_AND_ACK
2025-03-21T04:19:07.427Z DEBUG   03…4c-channeld-chan#255714: Can't send commit: waiting for revoke_and_ack

Now we are expected to send a new commitment, but we can't because we are still waiting to receive a revoke_and_ack from the peer.


  1. We receive a revoke_and_ack from the peer, moving our HTLC 35771 into the RCVD_REMOVE_ACK_REVOCATION state:
2025-03-21T04:19:07.427Z DEBUG   03…4c-channeld-chan#255714: peer_in WIRE_REVOKE_AND_ACK
2025-03-21T04:19:07.427Z DEBUG   03…4c-channeld-chan#255714: Received revoke_and_ack
2025-03-21T04:19:07.427Z DEBUG   03…4c-channeld-chan#255714: htlc 35771: SENT_REMOVE_ACK_COMMIT->RCVD_REMOVE_ACK_REVOCATION
2025-03-21T04:19:07.427Z DEBUG   03…4c-channeld-chan#255714: rcvd_revoke_and_ack: HTLC LOCAL 35771 = RCVD_REMOVE_ACK_REVOCATION/SENT_REMOVE_ACK_REVOCATION FAILED
2025-03-21T04:19:07.427Z DEBUG   03…4c-channeld-chan#255714: No commits outstanding after recv revoke_and_ack
2025-03-21T04:19:07.427Z DEBUG   03…4c-channeld-chan#255714: HTLC 35771[LOCAL] => RCVD_REMOVE_ACK_REVOCATION
2025-03-21T04:19:07.427Z DEBUG   03…4c-hsmd: Got WIRE_HSMD_VALIDATE_REVOCATION
2025-03-21T04:19:07.428Z DEBUG   03…4c-hsmd: Got WIRE_HSMD_SIGN_PENALTY_TO_US
2025-03-21T04:19:07.429Z DEBUG   03…4c-channeld-chan#255714: Sending master 1022
2025-03-21T04:19:07.429Z DEBUG   03…4c-chan#255714: got revoke 97039: 1 changed
2025-03-21T04:19:07.429Z DEBUG   03…4c-chan#255714: HTLC out 35771 SENT_REMOVE_ACK_COMMIT->RCVD_REMOVE_ACK_REVOCATION
2025-03-21T04:19:07.430Z DEBUG   03…4c-chan#255714: Removing out HTLC 35771 state RCVD_REMOVE_ACK_REVOCATION REMOTEFAIL
2025-03-21T04:19:07.437Z DEBUG   03…4c-channeld-chan#255714: ... , awaiting 1122
2025-03-21T04:19:07.438Z DEBUG   03…4c-channeld-chan#255714: Got it!
2025-03-21T04:19:07.438Z DEBUG   03…4c-channeld-chan#255714: revoke_and_ack REMOTE: remote_per_commit = 03…b3, old_remote_per_commit = 02…97

We are still expected to send a new commitment, and now we can.

  1. We send a new commitment, moving our HTLC 35772 into the SENT_REMOVE_ACK_COMMIT state:
2025-03-21T04:19:07.448Z DEBUG   03…4c-channeld-chan#255714: Trying commit
2025-03-21T04:19:07.448Z DEBUG   03…4c-channeld-chan#255714: htlc 35772: SENT_REMOVE_REVOCATION->SENT_REMOVE_ACK_COMMIT
2025-03-21T04:19:07.448Z DEBUG   03…4c-channeld-chan#255714: htlc added REMOTE: local 1200044891 remote 229569507
2025-03-21T04:19:07.449Z DEBUG   03…4c-channeld-chan#255714: -> local 1486672524 remote 229569507
2025-03-21T04:19:07.449Z DEBUG   03…4c-channeld-chan#255714: sending_commit: HTLC LOCAL 35772 = SENT_REMOVE_ACK_COMMIT/RCVD_REMOVE_ACK_COMMIT FAILED
2025-03-21T04:19:07.450Z DEBUG   03…4c-hsmd: Got WIRE_HSMD_SIGN_REMOTE_COMMITMENT_TX
2025-03-21T04:19:07.450Z DEBUG   03…4c-channeld-chan#255714: Creating commit_sig signature 97041 30…01 for tx 02…20 wscript 52…ae key 02…55
2025-03-21T04:19:07.451Z DEBUG   03…4c-hsmd: Got WIRE_HSMD_SIGN_REMOTE_HTLC_TX
2025-03-21T04:19:07.452Z DEBUG   03…4c-channeld-chan#255714: Creating HTLC signature 30…83 for tx 02…00 wscript 76…68 key 03…7c
2025-03-21T04:19:07.461Z DEBUG   03…4c-hsmd: Got WIRE_HSMD_SIGN_REMOTE_HTLC_TX
2025-03-21T04:19:07.462Z DEBUG   03…4c-channeld-chan#255714: Creating HTLC signature 30…83 for tx 02…00 wscript 76…68 key 03…7c
2025-03-21T04:19:07.463Z DEBUG   03…4c-channeld-chan#255714: Telling master we're about to commit...
2025-03-21T04:19:07.463Z DEBUG   03…4c-channeld-chan#255714: Sending master 1020
2025-03-21T04:19:07.463Z DEBUG   03…4c-chan#255714: HTLC out 35772 SENT_REMOVE_REVOCATION->SENT_REMOVE_ACK_COMMIT
2025-03-21T04:19:07.474Z DEBUG   03…4c-channeld-chan#255714: ... , awaiting 1120
2025-03-21T04:19:07.480Z DEBUG   03…4c-channeld-chan#255714: Got it!
2025-03-21T04:19:07.480Z DEBUG   03…4c-channeld-chan#255714: Sending commit_sig with 2 htlc sigs
2025-03-21T04:19:07.483Z DEBUG   03…4c-channeld-chan#255714: peer_out WIRE_COMMITMENT_SIGNED
2025-03-21T04:19:07.490Z DEBUG   03…4c-channeld-chan#255714: Can't send commit: waiting for revoke_and_ack

Now we are expecting to receive a revoke_and_ack from the peer.


  1. We receive a revoke_and_ack from the peer, moving our HTLC 35772 into the RCVD_REMOVE_ACK_REVOCATION state:
2025-03-21T04:21:10.958Z DEBUG   03…4c-channeld-chan#255714: peer_in WIRE_REVOKE_AND_ACK
2025-03-21T04:21:10.958Z DEBUG   03…4c-channeld-chan#255714: Received revoke_and_ack
2025-03-21T04:21:10.958Z DEBUG   03…4c-channeld-chan#255714: htlc 35772: SENT_REMOVE_ACK_COMMIT->RCVD_REMOVE_ACK_REVOCATION
2025-03-21T04:21:10.958Z DEBUG   03…4c-channeld-chan#255714: rcvd_revoke_and_ack: HTLC LOCAL 35772 = RCVD_REMOVE_ACK_REVOCATION/SENT_REMOVE_ACK_REVOCATION FAILED
2025-03-21T04:21:10.958Z DEBUG   03…4c-channeld-chan#255714: No commits outstanding after recv revoke_and_ack
2025-03-21T04:21:10.958Z DEBUG   03…4c-channeld-chan#255714: HTLC 35772[LOCAL] => RCVD_REMOVE_ACK_REVOCATION
2025-03-21T04:21:10.959Z DEBUG   03…4c-channeld-chan#255714: Sending master 1022
2025-03-21T04:21:10.959Z DEBUG   03…4c-chan#255714: got revoke 97040: 1 changed
2025-03-21T04:21:10.959Z DEBUG   03…4c-chan#255714: HTLC out 35772 SENT_REMOVE_ACK_COMMIT->RCVD_REMOVE_ACK_REVOCATION
2025-03-21T04:21:10.960Z DEBUG   03…4c-chan#255714: Removing out HTLC 35772 state RCVD_REMOVE_ACK_REVOCATION REMOTEFAIL
2025-03-21T04:21:10.969Z DEBUG   03…4c-channeld-chan#255714: ... , awaiting 1122
2025-03-21T04:21:10.969Z DEBUG   03…4c-hsmd: Got WIRE_HSMD_VALIDATE_REVOCATION
2025-03-21T04:21:10.970Z DEBUG   03…4c-hsmd: Got WIRE_HSMD_SIGN_PENALTY_TO_US
2025-03-21T04:21:10.970Z DEBUG   03…4c-channeld-chan#255714: Got it!
2025-03-21T04:21:10.970Z DEBUG   03…4c-channeld-chan#255714: revoke_and_ack REMOTE: remote_per_commit = 03…cb, old_remote_per_commit = 03…b3
2025-03-21T04:21:10.980Z DEBUG   03…4c-channeld-chan#255714: Trying commit
2025-03-21T04:21:10.980Z DEBUG   03…4c-channeld-chan#255714: Can't send commit: nothing to send, feechange not wanted ({ RCVD_ADD_ACK_REVOCATION:864 }) blockheight not wanted ({ RCVD_ADD_ACK_REVOCATION:0 })

Now we're all sync'd up with our HTLCs 35771 and 35772 fully removed from the channel.


Much later…

The peer reconnects to us, and we reëstablish the channel with our HTLC 35773 (and four others) in the SENT_ADD_ACK_REVOCATION state:

2025-03-22T18:44:05.733Z DEBUG   03…4c-connectd: Failed connected out: All addresses failed: 4ew…d.onion:9735: need a proxy. 
2025-03-22T18:44:05.733Z DEBUG   03…4c-connectd: Will try reconnect in 2 seconds
2025-03-22T18:44:08.376Z DEBUG   03…4c-connectd: Failed connected out: All addresses failed: 4ew…d.onion:9735: need a proxy. 
2025-03-22T18:44:08.376Z DEBUG   03…4c-connectd: Will try reconnect in 4 seconds
2025-03-22T18:44:12.621Z DEBUG   03…4c-connectd: Failed connected out: All addresses failed: 4ew…d.onion:9735: need a proxy. 
2025-03-22T18:44:12.622Z DEBUG   03…4c-connectd: Will try reconnect in 8 seconds
2025-03-22T18:44:20.024Z DEBUG   03…4c-connectd: Connect IN
2025-03-22T18:44:20.024Z DEBUG   03…4c-connectd: peer_out WIRE_INIT
2025-03-22T18:44:20.032Z DEBUG   03…4c-connectd: peer_in WIRE_INIT
2025-03-22T18:44:20.032Z DEBUG   03…4c-connectd: Handed peer, entering loop
2025-03-22T18:44:20.032Z DEBUG   03…4c-gossipd: seeker: disabling gossip
2025-03-22T18:44:20.032Z DEBUG   03…4c-chan#255714: Peer has reconnected, state CHANNELD_NORMAL: connecting subd
2025-03-22T18:44:20.066Z DEBUG   03…4c-channeld-chan#255714: pid 26085, msgfd 179
2025-03-22T18:44:20.067Z DEBUG   03…4c-chan#255714: Already have funding locked in
2025-03-22T18:44:20.068Z DEBUG   03…4c-connectd: peer_out WIRE_GOSSIP_TIMESTAMP_FILTER
2025-03-22T18:44:20.071Z DEBUG   03…4c-channeld-chan#255714: option_static_remotekey = 1, option_anchor_outputs = 0 option_anchors_zero_fee_htlc_tx = 1
2025-03-22T18:44:20.072Z DEBUG   03…4c-channeld-chan#255714: init REMOTE: remote_per_commit = 02…66, old_remote_per_commit = 03…75 next_idx_local = 99234 next_idx_remote = 99235 revocations_received = 99234 feerates { RCVD_ADD_ACK_REVOCATION:7052 } range 941-70380 blockheights { RCVD_ADD_ACK_REVOCATION:0 }, our current 888946
2025-03-22T18:44:20.072Z DEBUG   03…4c-hsmd: Got WIRE_HSMD_GET_PER_COMMITMENT_POINT
2025-03-22T18:44:20.072Z DEBUG   03…4c-channeld-chan#255714: Restoring HTLC 0/5: id=35773 amount=69663541msat cltv=888979 payment_hash=71…10 
2025-03-22T18:44:20.072Z DEBUG   03…4c-channeld-chan#255714: NEW:: HTLC LOCAL 35773 = SENT_ADD_ACK_REVOCATION/RCVD_ADD_ACK_REVOCATION 
2025-03-22T18:44:20.072Z DEBUG   03…4c-channeld-chan#255714: Restoring HTLC 1/5: id=36784 amount=122203153msat cltv=889997 payment_hash=3a…e5 
2025-03-22T18:44:20.072Z DEBUG   03…4c-channeld-chan#255714: NEW:: HTLC LOCAL 36784 = SENT_ADD_ACK_REVOCATION/RCVD_ADD_ACK_REVOCATION 
2025-03-22T18:44:20.073Z DEBUG   03…4c-channeld-chan#255714: Restoring HTLC 2/5: id=35719 amount=214094428msat cltv=889003 payment_hash=bd…ca 
2025-03-22T18:44:20.073Z DEBUG   03…4c-channeld-chan#255714: NEW:: HTLC LOCAL 35719 = SENT_ADD_ACK_REVOCATION/RCVD_ADD_ACK_REVOCATION 
2025-03-22T18:44:20.073Z DEBUG   03…4c-channeld-chan#255714: Restoring HTLC 3/5: id=36782 amount=25986187msat cltv=889173 payment_hash=f5…ac 
2025-03-22T18:44:20.073Z DEBUG   03…4c-channeld-chan#255714: NEW:: HTLC LOCAL 36782 = SENT_ADD_ACK_REVOCATION/RCVD_ADD_ACK_REVOCATION 
2025-03-22T18:44:20.073Z DEBUG   03…4c-channeld-chan#255714: Restoring HTLC 4/5: id=36783 amount=317256926msat cltv=889228 payment_hash=8e…e2 
2025-03-22T18:44:20.073Z DEBUG   03…4c-channeld-chan#255714: NEW:: HTLC LOCAL 36783 = SENT_ADD_ACK_REVOCATION/RCVD_ADD_ACK_REVOCATION 
2025-03-22T18:44:20.073Z DEBUG   03…4c-channeld-chan#255714: peer_out WIRE_CHANNEL_REESTABLISH
2025-03-22T18:44:20.073Z DEBUG   03…4c-channeld-chan#255714: billboard: Sent reestablish, waiting for theirs
2025-03-22T18:44:20.074Z DEBUG   03…4c-hsmd: Got WIRE_HSMD_GET_PER_COMMITMENT_POINT

We receive a channel_reestablish from the peer and have nothing to retransmit:

2025-03-22T18:44:41.559Z DEBUG   03…4c-channeld-chan#255714: peer_in WIRE_CHANNEL_REESTABLISH
2025-03-22T18:44:41.559Z DEBUG   03…4c-channeld-chan#255714: Got reestablish commit=99235 revoke=99233 inflights: 0, active splices: 0
2025-03-22T18:44:41.560Z DEBUG   03…4c-channeld-chan#255714: Splice resume check with local_next_funding: omitted, remote_next_funding: empty, inflights: 0
2025-03-22T18:44:41.560Z DEBUG   03…4c-channeld-chan#255714: next_revocation_number = 99233
2025-03-22T18:44:41.560Z DEBUG   03…4c-channeld-chan#255714: option_static_remotekey: fields are correct
2025-03-22T18:44:41.560Z DEBUG   03…4c-channeld-chan#255714: No upgrade: no next_to_send received
2025-03-22T18:44:41.560Z DEBUG   03…4c-chan#255714: channel_gossip: reestablished
2025-03-22T18:44:41.560Z DEBUG   03…4c-channeld-chan#255714: billboard perm: Reconnected, and reestablished.
2025-03-22T18:44:41.560Z DEBUG   03…4c-channeld-chan#255714: billboard: Channel ready for use.
2025-03-22T18:44:41.560Z DEBUG   03…4c-channeld-chan#255714: billboard: Channel ready for use.
2025-03-22T18:44:41.560Z DEBUG   03…4c-hsmd: Got WIRE_HSMD_REVOKE_COMMITMENT_TX
2025-03-22T18:44:41.570Z DEBUG   03…4c-channeld-chan#255714: Trying commit
2025-03-22T18:44:41.570Z DEBUG   03…4c-channeld-chan#255714: Can't send commit: nothing to send, feechange not wanted ({ RCVD_ADD_ACK_REVOCATION:7052 }) blockheight not wanted ({ RCVD_ADD_ACK_REVOCATION:0 })

Later still… (with no intervening disconnections)

  1. Our HTLC 35773 hits its deadline, and we are forced to unilaterally close the channel:
2025-03-22T23:11:48.144Z UNUSUAL 03…4c-chan#255714: Peer permanent failure in CHANNELD_NORMAL: Offered HTLC 35773 SENT_ADD_ACK_REVOCATION cltv 888979 hit deadline (reason=protocol)
2025-03-22T23:11:48.144Z DEBUG   03…4c-channeld-chan#255714: Status closed, but not exited. Killing
2025-03-22T23:11:48.145Z INFO    03…4c-chan#255714: State changed from CHANNELD_NORMAL to AWAITING_UNILATERAL
2025-03-22T23:11:48.174Z DEBUG   03…4c-chan#255714: We have 3 anchor points to use
2025-03-22T23:11:48.227Z DEBUG   03…4c-chan#255714: Worth fee 0sat for local commit tx to get 263913860msat in 889510 blocks at feerate 4853perkw
2025-03-22T23:11:48.230Z DEBUG   03…4c-chan#255714: Worth fee 0sat for local commit tx to get 125090514msat in 889307 blocks at feerate 4853perkw
2025-03-22T23:11:48.232Z DEBUG   03…4c-chan#255714: Worth fee 0sat for local commit tx to get 214094428msat in 889003 blocks at feerate 4853perkw
2025-03-22T23:11:48.235Z DEBUG   03…4c-chan#255714: Worth fee 15458sat for local commit tx to get 69663541msat in 888979 blocks at feerate 10891perkw
2025-03-22T23:11:48.235Z DEBUG   03…4c-chan#255714: Anchorspend for local commit tx fee 15458sat (w=714), commit_tx fee 10051sat (w=1628): package feerate 10891 perkw
2025-03-22T23:11:48.238Z INFO    03…4c-chan#255714: Creating anchor spend for local commit tx 09…f3: we're paying fee 15458sat
2025-03-22T23:11:48.246Z DEBUG   03…4c-connectd: peer_downgrade
2025-03-22T23:11:52.775Z DEBUG   03…4c-connectd: Activating for message WIRE_COMMITMENT_SIGNED
2025-03-22T23:11:52.775Z DEBUG   03…4c-lightningd: Telling connectd to send error 00…65
2025-03-22T23:11:52.776Z DEBUG   03…4c-connectd: peer_out WIRE_ERROR
2025-03-22T23:11:52.776Z DEBUG   03…4c-connectd: disconnect_peer
2025-03-22T23:11:52.776Z DEBUG   03…4c-lightningd: peer_disconnect_done
2025-03-22T23:11:52.776Z DEBUG   plugin-funder: Cleaning up inflights for peer id 03…4c

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant