[RFC] Implement a way to do BOLT 12 Proof of Payment #3593

vincenzopalazzo · 2025-02-07T23:14:43Z

This is already possible if we manually_handle_bolt12_invoices, but
this approach requires a significant amount of work from the user.

This commit adds the bolt12 invoice to the PaymentSend event when
the payment is completed. This always gives the user the option to
provide proof of payment.

I am opening this up for an initial review, but I need to look inside the DNSSECProof to add this proof, but
probably be also a follow-up PR

~~Opening in draft for now to make the CI happy~~

Fixes: #3344

lightning/src/events/mod.rs

codecov · 2025-02-10T20:28:47Z

Codecov Report

Attention: Patch coverage is 89.42308% with 11 lines in your changes missing coverage. Please review.

Project coverage is 89.30%. Comparing base (1fc2726) to head (0cd70fa).
Report is 8 commits behind head on main.

Files with missing lines	Patch %	Lines
lightning/src/offers/static_invoice.rs	0.00%	8 Missing ⚠️
lightning/src/ln/offers_tests.rs	92.85%	1 Missing ⚠️
lightning/src/ln/outbound_payment.rs	96.55%	1 Missing ⚠️
lightning/src/util/ser_macros.rs	0.00%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3593      +/-   ##
==========================================
+ Coverage   89.25%   89.30%   +0.05%     
==========================================
  Files         155      156       +1     
  Lines      119708   121825    +2117     
  Branches   119708   121825    +2117     
==========================================
+ Hits       106845   108796    +1951     
- Misses      10250    10388     +138     
- Partials     2613     2641      +28

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

TheBlueMatt

Thank you for picking this up! Sorry about the delay in review, not sure why this slipped through the cracks.

lightning/src/ln/outbound_payment.rs

lightning/src/events/mod.rs

vincenzopalazzo · 2025-02-16T13:06:50Z

Thanks for the review @TheBlueMatt, I applied the suggestion in the review. In the meanwhile, I rebased on top of main to fix the conflict and ran the CI again

lightning/src/ln/outbound_payment.rs

lightning/src/offers/mod.rs

lightning/src/ln/outbound_payment.rs

vincenzopalazzo · 2025-03-05T19:40:43Z

Ok @TheBlueMatt this PR is really funny to work! I worked to clean up the PR a bit and divide the code to support both Bolt12 and static invoices. However, I am currently working on supporting the async payment functionality. As far as I understand, the code involving async payments is a bit tricky to support for the following reasons (code available 85c3f73).

The async workflow is initialized with initiate_async_payment, where we pass down the payment_id and the static invoice value. Then we call static_invoice_received, which generates the event PendingOutboundPayment::StaticInvoiceReceived. After that, we iterate over the entry in send_payment_for_static_invoice and follow the Bolt12 pay code path (at this point, we need to pass down the static invoice).

The only way I have to access the static invoice when passing it down inside PendingOutboundPayment::StaticInvoiceReceived breaks the storage code because the static invoice is not available without cfg(async_payment).

Is there any way to use the persistence macros (impl_writeable_tlv_based_enum_upgradable) with a cfg?
If we want to read and pass the static invoice in PendingOutboundPayment::StaticInvoiceReceived, we could store a serialized version of the static invoice (a sequence of bytes), but this seems rather hacky.
I still need to delve into the mechanics of async payment as implemented in LDK, but if I understand correctly, there is a refactoring that I can do to make access to the static invoice a bit easier?

I would prefer the 3rd point, but this requires a follow-up PR to rework a little bit how the async payment is handled and I do not know if there is any plan for that,t or there is other code that will be depending on this architecture in the pipeline.

Probably @valentinewallace will be interested in this conversation

TheBlueMatt · 2025-03-10T14:28:57Z

Is there any way to use the persistence macros (impl_writeable_tlv_based_enum_upgradable) with a cfg?

Generally we just copy and paste the impl with different cfg flags and go back and clean it up after we drop the cfg.

I'll let @valentinewallace comment on the rest.

valentinewallace · 2025-03-10T16:00:42Z

Ok @TheBlueMatt this PR is really funny to work! I worked to clean up the PR a bit and divide the code to support both Bolt12 and static invoices. However, I am currently working on supporting the async payment functionality. As far as I understand, the code involving async payments is a bit tricky to support for the following reasons (code available 85c3f73).

The async workflow is initialized with initiate_async_payment, where we pass down the payment_id and the static invoice value. Then we call static_invoice_received, which generates the event PendingOutboundPayment::StaticInvoiceReceived. After that, we iterate over the entry in send_payment_for_static_invoice and follow the Bolt12 pay code path (at this point, we need to pass down the static invoice).

The only way I have to access the static invoice when passing it down inside PendingOutboundPayment::StaticInvoiceReceived breaks the storage code because the static invoice is not available without cfg(async_payment).
* Is there any way to use the persistence macros (`impl_writeable_tlv_based_enum_upgradable`) with a cfg?

* If we want to read and pass the static invoice in `PendingOutboundPayment::StaticInvoiceReceived`, we could store a serialized version of the static invoice (a sequence of bytes), but this seems rather hacky.

* I still need to delve into the mechanics of async payment as implemented in LDK, but if I understand correctly, there is a refactoring that I can do to make access to the static invoice a bit easier?
I would prefer the 3rd point, but this requires a follow-up PR to rework a little bit how the async payment is handled and I do not know if there is any plan for that,t or there is other code that will be depending on this architecture in the pipeline.

Probably @valentinewallace will be interested in this conversation

Hey sorry I missed this tag. In #3618 right now we actually un-cfg-gate StaticInvoices. Would that solve this? I could try to get conceptual ACKs on that to unblock you if so.

vincenzopalazzo · 2025-03-11T20:52:22Z

Generally we just copy and paste the impl with different cfg flags and go back and clean it up after we drop the cfg.

Yes, but in this case, we need to duplicate all the impl_writeable_tlv_based_enum_upgradable code which sounds a really bad thing, I think I will go to steal the commit 890ce7d and see how the code will looks like :) but I think StaticInvoice out of the cfg is all what I needs

vincenzopalazzo · 2025-04-01T13:01:42Z

Ok at my desk trying to finish this PR ASAP.

I had to rebase the PR to fix the conflicts to be able to run the CI! now waiting for the latter to see if I am forgetting something else to fix.

Thanks for your time in review this PR @valentinewallace

Probably the only thing left to fix Is the lining with the new change at the docs https://github.com/lightningdevkit/rust-lightning/actions/runs/14196272391/job/39772046844?pr=3593

lightning/src/events/mod.rs

lightning/src/ln/functional_test_utils.rs

lightning/src/offers/invoice_macros.rs

TheBlueMatt

I think you can go ahead and squash and we can probably do final passes?

lightning/src/events/mod.rs

We need to include static invoices in the public API as part of the onion messages we're adding for static invoice server support. Utilities to create these static invoices and other parts of the async receive API will remain cfg-gated for now. Generally, we can't totally avoid exposing half baked async receive support in the public API because OnionMessenger is parameterized by an async payments message handler, which can't be cfg-gated easily. Signed-off-by: Vincenzo Palazzo <[email protected]>

vincenzopalazzo · 2025-04-05T10:38:38Z

Ok should be ready for another round of review! However, I am not sure why the linter is failing now, looks like unrelated to this PR?

TheBlueMatt

LGTM one nit

lightning/src/ln/outbound_payment.rs

valentinewallace

LGTM after Matt's last comment

valentinewallace · 2025-04-07T20:41:11Z

lightning/src/util/ser_macros.rs

@@ -1200,7 +1200,7 @@ macro_rules! impl_writeable_tlv_based_enum {
 					$($tuple_variant_id => {
 						let length: $crate::util::ser::BigSize = $crate::util::ser::Readable::read(reader)?;
 						let mut s = $crate::util::ser::FixedLengthReader::new(reader, length.0);
-						let res = $crate::util::ser::Readable::read(&mut s)?;
+						let res = $crate::util::ser::LengthReadable::read_from_fixed_length_buffer(&mut s)?;


It actually still says that this is a temporary workaround, but not blocking

@valentinewallace

When introducing the LengthLimitedRead trait in [1], for some serialization macros we need this change to avoid that we have the following compilation error error[E0277]: the trait bound `Bolt12Invoice: Readable` is not satisfied --> lightning/src/util/ser_macros.rs:1243:35 | 1243 | Ok($st::$tuple_variant_name($crate::util::ser::Readable::read(reader)?)) | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ the trait `Readable` is not implemented for `Bolt12Invoice` | ::: lightning/src/events/mod.rs:2463:1 | 2463 | / impl_writeable_tlv_based_enum_legacy!(PaidInvoice, 2464 | | ; 2465 | | (0, Bolt12Invoice), 2466 | | (2, StaticInvoice) 2467 | | ); | |_- in this macro invocation | = help: the following other types implement trait `Readable`: () (A, B) (A, B, C) (A, B, C, D) (A, B, C, D, E) (A, B, C, D, E, F) (A, B, C, D, E, F, G) AnnouncementSigsState and 200 others = note: this error originates in the macro `impl_writeable_tlv_based_enum_legacy` (in Nightly builds, run with -Z macro-backtrace for more info) This changes are needed since [1] is landed on mainline. [1] lightningdevkit#3640 Co-Developed-by: @valentinewallace Suggested-by: @valentinewallace Signed-off-by: Vincenzo Palazzo <[email protected]>

This commit make two things possible: 1. make persistent BOLT12 invoice through PendingOutboundPayment This commit prepares the code to pass down the BOLT12 invoice inside the `PaymentSent` event. To achieve this, the `bolt12` field has been added to the `PendingOutboundPayment::Retryable` enum, allowing it to be attached to the `PaymentSent` event when the payment is completed. 2. To enable proof of payment, we need to share the bolt12 invoice with the library user. This is already possible if we `manually_handle_bolt12_invoices`, but this approach requires a significant amount of work from the user. This commit adds the bolt12 invoice to the PaymentSend event when the payment is completed. This allows the user to always have the option to perform proof of payment. We also Inject the static invoice inside the PendingOutboundPayment::StaticInvoiceReceived to allow to use the static invoice inside the PendingOutboundPayment::PaymentReceived. Link: lightningdevkit#3344 Signed-off-by: Vincenzo Palazzo <[email protected]>

vincenzopalazzo · 2025-04-07T21:07:08Z

It actually still says that this is a temporary workaround, but not blocking

Sorry I forgot to read also the upper part of the commit body, luckily git rebase -i is free :)

TheBlueMatt

One issue, but I'm gonna go ahead and land this as fixing the issue can be separate.

TheBlueMatt · 2025-04-08T00:28:40Z

lightning/src/ln/outbound_payment.rs

@@ -2374,6 +2388,7 @@ impl OutboundPayments {
 					payment_metadata: None, // only used for retries, and we'll never retry on startup
 					keysend_preimage: None, // only used for retries, and we'll never retry on startup
 					invoice_request: None, // only used for retries, and we'll never retry on startup
+					bolt12_invoice: None, // only used for retries, and we'll never retry on startup!


Oof. Sorry I missed this until now. This is not, in fact, "only used for retries", we use it on claims only, in fact. If a user is relying on the event field for PoP, what this can mean is that we can initiate a send, restart with a stale ChannelManager, notice the payment is pending, then when it claims fail to provide the invoice (only the preimage) to the payer.

In practice, to fix this, we'll need to include the PaidBolt12Invoice in the HTLCSource::OutboundRoute, I believe.

vincenzopalazzo marked this pull request as draft February 7, 2025 23:18

vincenzopalazzo force-pushed the macros/proof-of-payment-bolt12 branch from c7e9e61 to cc714ae Compare February 7, 2025 23:33

vincenzopalazzo commented Feb 7, 2025

View reviewed changes

lightning/src/events/mod.rs Outdated Show resolved Hide resolved

vincenzopalazzo force-pushed the macros/proof-of-payment-bolt12 branch from a5c211c to 9cdbb44 Compare February 10, 2025 16:59

vincenzopalazzo marked this pull request as ready for review February 10, 2025 16:59

vincenzopalazzo force-pushed the macros/proof-of-payment-bolt12 branch from 9cdbb44 to 8f5dc5e Compare February 10, 2025 20:19

TheBlueMatt reviewed Feb 13, 2025

View reviewed changes

lightning/src/ln/outbound_payment.rs Outdated Show resolved Hide resolved

lightning/src/ln/outbound_payment.rs Show resolved Hide resolved

lightning/src/ln/outbound_payment.rs Outdated Show resolved Hide resolved

lightning/src/events/mod.rs Outdated Show resolved Hide resolved

vincenzopalazzo mentioned this pull request Feb 16, 2025

BOLT 12 Proof of Payment #3344

Closed

vincenzopalazzo force-pushed the macros/proof-of-payment-bolt12 branch from 7da68f6 to 0e7b503 Compare February 16, 2025 13:05

vincenzopalazzo force-pushed the macros/proof-of-payment-bolt12 branch from 0e7b503 to a4897fb Compare February 16, 2025 13:11

vincenzopalazzo requested a review from TheBlueMatt February 16, 2025 13:12

TheBlueMatt reviewed Feb 18, 2025

View reviewed changes

lightning/src/ln/outbound_payment.rs Outdated Show resolved Hide resolved

vincenzopalazzo force-pushed the macros/proof-of-payment-bolt12 branch from a4897fb to 2c65d6a Compare March 3, 2025 19:15

TheBlueMatt reviewed Mar 5, 2025

View reviewed changes

lightning/src/offers/mod.rs Outdated Show resolved Hide resolved

lightning/src/offers/mod.rs Outdated Show resolved Hide resolved

lightning/src/ln/outbound_payment.rs Outdated Show resolved Hide resolved

vincenzopalazzo force-pushed the macros/proof-of-payment-bolt12 branch from 9b72246 to 2c65d6a Compare March 5, 2025 16:31

vincenzopalazzo mentioned this pull request Mar 7, 2025

Multiple InvoiceReceived events for single pay_for_offer call #3653

Open

TheBlueMatt requested a review from valentinewallace March 10, 2025 14:27

vincenzopalazzo force-pushed the macros/proof-of-payment-bolt12 branch 7 times, most recently from 734c955 to 525c01e Compare March 12, 2025 15:01

vincenzopalazzo force-pushed the macros/proof-of-payment-bolt12 branch 4 times, most recently from d69787b to 3929d07 Compare April 1, 2025 12:59

vincenzopalazzo requested a review from valentinewallace April 2, 2025 17:41

TheBlueMatt reviewed Apr 4, 2025

View reviewed changes

vincenzopalazzo force-pushed the macros/proof-of-payment-bolt12 branch 2 times, most recently from 7450529 to 9ed2e06 Compare April 4, 2025 16:47

vincenzopalazzo requested a review from TheBlueMatt April 4, 2025 16:47

TheBlueMatt reviewed Apr 4, 2025

View reviewed changes

lightning/src/events/mod.rs Outdated Show resolved Hide resolved

vincenzopalazzo force-pushed the macros/proof-of-payment-bolt12 branch from 9ed2e06 to 22f8c50 Compare April 5, 2025 10:19

vincenzopalazzo requested a review from TheBlueMatt April 5, 2025 10:35

TheBlueMatt reviewed Apr 7, 2025

View reviewed changes

lightning/src/ln/outbound_payment.rs Outdated Show resolved Hide resolved

valentinewallace reviewed Apr 7, 2025

View reviewed changes

vincenzopalazzo force-pushed the macros/proof-of-payment-bolt12 branch 2 times, most recently from 2abe14a to 44bb850 Compare April 7, 2025 19:09

vincenzopalazzo requested review from valentinewallace and TheBlueMatt April 7, 2025 19:09

valentinewallace approved these changes Apr 7, 2025

View reviewed changes

vincenzopalazzo added 2 commits April 7, 2025 23:05

vincenzopalazzo force-pushed the macros/proof-of-payment-bolt12 branch from 44bb850 to 57772d3 Compare April 7, 2025 21:06

TheBlueMatt approved these changes Apr 8, 2025

View reviewed changes

TheBlueMatt mentioned this pull request Apr 8, 2025

Fix (or doc, for 0.2) missing Event::PaymentSent::bolt12_invoice on payments loaded from monitors #3714

Open

TheBlueMatt merged commit 077df33 into lightningdevkit:main Apr 8, 2025
24 of 27 checks passed

vincenzopalazzo deleted the macros/proof-of-payment-bolt12 branch April 8, 2025 15:30

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[RFC] Implement a way to do BOLT 12 Proof of Payment #3593

[RFC] Implement a way to do BOLT 12 Proof of Payment #3593

vincenzopalazzo commented Feb 7, 2025 •

edited

Loading

codecov bot commented Feb 10, 2025 •

edited

Loading

TheBlueMatt left a comment

vincenzopalazzo commented Feb 16, 2025 •

edited

Loading

vincenzopalazzo commented Mar 5, 2025

TheBlueMatt commented Mar 10, 2025

valentinewallace commented Mar 10, 2025

vincenzopalazzo commented Mar 11, 2025

vincenzopalazzo commented Apr 1, 2025 •

edited

Loading

TheBlueMatt left a comment

vincenzopalazzo commented Apr 5, 2025

TheBlueMatt left a comment

valentinewallace left a comment

valentinewallace Apr 7, 2025

vincenzopalazzo commented Apr 7, 2025

TheBlueMatt left a comment

TheBlueMatt Apr 8, 2025

[RFC] Implement a way to do BOLT 12 Proof of Payment #3593

[RFC] Implement a way to do BOLT 12 Proof of Payment #3593

Conversation

vincenzopalazzo commented Feb 7, 2025 • edited Loading

codecov bot commented Feb 10, 2025 • edited Loading

Codecov Report

TheBlueMatt left a comment

Choose a reason for hiding this comment

vincenzopalazzo commented Feb 16, 2025 • edited Loading

vincenzopalazzo commented Mar 5, 2025

TheBlueMatt commented Mar 10, 2025

valentinewallace commented Mar 10, 2025

vincenzopalazzo commented Mar 11, 2025

vincenzopalazzo commented Apr 1, 2025 • edited Loading

TheBlueMatt left a comment

Choose a reason for hiding this comment

vincenzopalazzo commented Apr 5, 2025

TheBlueMatt left a comment

Choose a reason for hiding this comment

valentinewallace left a comment

Choose a reason for hiding this comment

valentinewallace Apr 7, 2025

Choose a reason for hiding this comment

vincenzopalazzo commented Apr 7, 2025

TheBlueMatt left a comment

Choose a reason for hiding this comment

TheBlueMatt Apr 8, 2025

Choose a reason for hiding this comment

vincenzopalazzo commented Feb 7, 2025 •

edited

Loading

codecov bot commented Feb 10, 2025 •

edited

Loading

vincenzopalazzo commented Feb 16, 2025 •

edited

Loading

vincenzopalazzo commented Apr 1, 2025 •

edited

Loading