Introduce MPT support (XLS-33d): #5143

gregtatcam · 2024-09-19T07:41:44Z

High Level Overview of Change

New Transactions:

MPTokenIssuanceCreate
MPTokenIssuanceDestory
MPTokenIssuanceSet
MPTokenAuthorize

Modified Transactions:

Payment
Clawback

New Objects:

MPTokenIssuance
MPToken

API updates:

ledger_entry
account_objects
ledger_data

Refactor:

Add new types Asset, MPTIssue, MPTAmount to handle MPT amount.
Add SOElement constructor overload to describe the amount fields, which may have MPT amount (currently only sfAmount). The constructors are used in TxFormats to define fields supporting MPT.

Context of Change

XLS-33d specification

Type of Change

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to not work as expected)
Refactor (non-breaking change that only restructures code)
Performance (increase or change in throughput and/or latency)
Tests (you added tests for code that already exists, or your new feature included in this PR)
Documentation update
Chore (no impact to binary, e.g. .gitignore, formatting, dropping support for older tooling)
Release

API Impact

Public API: New feature (new methods and/or new fields)
Public API: Breaking change (in general, breaking changes should only impact the next api_version)
libxrpl change (any change that may affect libxrpl or dependents of libxrpl)
Peer protocol change (must be backward compatible or bump the peer protocol version)

Test Plan

Added test for new feature:

MPT

Future Tasks

Integrate MPT into XRPL DEX.

codecov · 2024-09-19T08:05:17Z

Codecov Report

Attention: Patch coverage is 86.32931% with 181 lines in your changes missing coverage. Please review.

Project coverage is 77.5%. Comparing base (63209c2) to head (9ce2061).
Report is 1 commits behind head on develop.

Files with missing lines	Patch %	Lines
src/libxrpl/protocol/STAmount.cpp	79.4%	35 Missing ⚠️
src/xrpld/app/tx/detail/InvariantCheck.cpp	66.3%	30 Missing ⚠️
src/xrpld/ledger/detail/View.cpp	88.8%	20 Missing ⚠️
src/libxrpl/protocol/MPTIssue.cpp	46.7%	16 Missing ⚠️
src/libxrpl/basics/MPTAmount.cpp	12.5%	14 Missing ⚠️
src/xrpld/rpc/handlers/LedgerEntry.cpp	68.6%	11 Missing ⚠️
src/xrpld/app/tx/detail/MPTokenAuthorize.cpp	92.2%	9 Missing ⚠️
src/libxrpl/protocol/Asset.cpp	71.4%	6 Missing ⚠️
src/libxrpl/protocol/STParsedJSON.cpp	62.5%	6 Missing ⚠️
src/xrpld/app/tx/detail/Payment.cpp	95.5%	6 Missing ⚠️
... and 12 more

Additional details and impacted files

@@            Coverage Diff            @@
##           develop   #5143     +/-   ##
=========================================
+ Coverage     76.1%   77.5%   +1.4%     
=========================================
  Files          762     777     +15     
  Lines        61469   65829   +4360     
  Branches      8121    8185     +64     
=========================================
+ Hits         46807   51024   +4217     
- Misses       14662   14805    +143

Files with missing lines	Coverage Δ
include/xrpl/basics/MPTAmount.h	`100.0% <100.0%> (ø)`
include/xrpl/basics/Number.h	`100.0% <100.0%> (ø)`
include/xrpl/basics/XRPAmount.h	`98.8% <ø> (+<0.1%)`	⬆️
include/xrpl/basics/base_uint.h	`96.8% <ø> (+0.1%)`	⬆️
include/xrpl/protocol/AmountConversions.h	`87.7% <100.0%> (+0.2%)`	⬆️
include/xrpl/protocol/Feature.h	`100.0% <ø> (ø)`
include/xrpl/protocol/Indexes.h	`100.0% <ø> (ø)`
include/xrpl/protocol/Issue.h	`100.0% <100.0%> (ø)`
include/xrpl/protocol/MPTIssue.h	`100.0% <100.0%> (ø)`
include/xrpl/protocol/Rate.h	`100.0% <ø> (ø)`
... and 56 more

... and 530 files with indirect coverage changes

New Transactions: - MPTokenIssuanceCreate - MPTokenIssuanceDestory - MPTokenIssuanceSet - MPTokenAuthorize Modified Transactions: - Payment - Clawback New Objects: - MPTokenIssuance - MPTokenAuthorize API updates: - ledger_entry - account_objects - ledger_data Read full spec: https://github.com/XRPLF/XRPL-Standards/tree/master/XLS-0033d-multi-purpose-tokens --------- Co-authored-by: Shawn Xie <[email protected]> Co-authored-by: Howard Hinnant <[email protected]>

HowardHinnant · 2024-09-30T19:08:56Z

include/xrpl/protocol/LedgerFormats.h

+    lsfMPTCanClawback = 0x00000040,
+
+    // ltMPTOKEN
+    lsfMPTAuthorized = 0x00000002,


lsfMPTLocked, lsfMPTCanLock and lsfMPTAuthorized do not have unique values within this enum. Is that on purpose?

The LedgerSpecificFlags enum is really just a way to define a bunch of constants. The flags are unique per ledger object type. Those are divided by the comment / label before each one. It might be worthwhile to split them into one per object type, but that's probably beyond the scope of this PR.

thejohnfreeman

Submitting the first phase of my review, a complete review of the changes to libxrpl.

include/xrpl/basics/MPTAmount.h

include/xrpl/protocol/Asset.h

include/xrpl/protocol/UintTypes.h

src/libxrpl/protocol/LedgerFormats.cpp

src/libxrpl/protocol/STTx.cpp

ximinez

Partial review:

include/xrpl/basics/MPTAmount.h

src/test/jtx/impl/mpt.cpp

include/xrpl/basics/MPTAmount.h

include/xrpl/protocol/Asset.h

include/xrpl/protocol/LedgerFormats.h

include/xrpl/protocol/SOTemplate.h

src/libxrpl/protocol/STAmount.cpp

Co-authored-by: Ed Hennis <[email protected]>

Update MPT Payment errors to be consistent with IOU Payment.

ximinez

Partial review

ximinez · 2024-10-01T19:07:42Z

include/xrpl/basics/MPTAmount.h

+
+namespace ripple {
+
+class MPTAmount : private boost::totally_ordered<MPTAmount>,


Seeing how similar MPTAmount is to XRPAmount it's a pity that they aren't implemented with common code. I.e. through a base class, or a template or something. Rather than diving down the potential rabbit hole of implementing it, I'm just going to leave this note here for someone else or for future reference.

My opinion:

They should be called MPTNumber and XRPNumber because they represent quantities, while STAmount represents a quantity plus an asset / issue / unit (however you want to think of it). These are more like Number, and convert directly to and from it.

All of the arithmetic is moving to Number after the switchover anyway. In due time, when we retire that amendment, effectively locking it in permanently, then I think we can remove MPTAmount and XRPAmount.

Should IOUAmount change too? I think that Amount better communicates what the value is. I don't think you'd say number when talking about tokens or currencies even if the values don't have associate unit. And XRPAmount doesn't need an issue, it's implicit. Also doesn't seem like this refactoring has to be done in MPT PR. But this is just my opinion. I'll change if everyone thinks Number is better. There are over 300 instances of XRPAmount, MPTAmount, IOUAmount.

I created a ticket to refactor MPTAmount and XRPAmount to use a common code + renaming.

Yeah, I'm not asking it to be done here, or at all even. Was just stating my position. My opinion is that IOUAmount should be renamed too (in an ideal world), and my only reason for these renames is that STAmount has quantity + issue, while these {XRP,IOU,MPT}Amounts only have quantity, like Number, which is their common representation. The alternative fix is to rename STAmount to a different suffix, but I think that would be even more disruptive, especially to external projects.

include/xrpl/protocol/STAmount.h

src/xrpld/app/misc/NetworkOPs.cpp

src/libxrpl/protocol/LedgerFormats.cpp

src/libxrpl/protocol/MPTIssue.cpp

src/libxrpl/protocol/STAmount.cpp

src/xrpld/ledger/detail/View.cpp

Co-authored-by: Ed Hennis <[email protected]>

include/xrpl/protocol/Asset.h

thejohnfreeman · 2024-10-02T18:32:42Z

include/xrpl/basics/MPTAmount.h

+
+namespace ripple {
+
+class MPTAmount : private boost::totally_ordered<MPTAmount>,


My opinion:

They should be called MPTNumber and XRPNumber because they represent quantities, while STAmount represents a quantity plus an asset / issue / unit (however you want to think of it). These are more like Number, and convert directly to and from it.

All of the arithmetic is moving to Number after the switchover anyway. In due time, when we retire that amendment, effectively locking it in permanently, then I think we can remove MPTAmount and XRPAmount.

include/xrpl/basics/MPTAmount.h

include/xrpl/protocol/Indexes.h

include/xrpl/protocol/MPTIssue.h

include/xrpl/protocol/jss.h

src/libxrpl/protocol/STTx.cpp

include/xrpl/protocol/UintTypes.h

ximinez

Partial review

src/libxrpl/protocol/STAmount.cpp

Co-authored-by: Ed Hennis <[email protected]>

use std::min, std::max Co-authored-by: Ed Hennis <[email protected]>

* Refactor getSNValue/getMPTValue * Use std::min, std::max * Fix min/max values for MPT

include/xrpl/protocol/TxFlags.h

Co-authored-by: Ed Hennis <[email protected]>

thejohnfreeman · 2024-10-22T02:47:18Z

src/xrpld/app/tx/detail/MPTokenIssuanceSet.cpp

+    if (flagsIn != flagsOut)
+        sle->setFieldU32(sfFlags, flagsOut);
+
+    view().update(sle);


You can move this statement into the previous branch.

I think we'd like to keep it here because other transactors have it similar (eg. AccountSet). Also potential in the future we may add new functionalities to this transactor so might as well leave it here.

include/xrpl/protocol/Asset.h

include/xrpl/protocol/detail/transactions.macro

src/xrpld/app/tx/detail/MPTokenAuthorize.cpp

thejohnfreeman · 2024-10-22T14:58:25Z

src/xrpld/app/tx/detail/MPTokenAuthorize.cpp

+                auto const sleMptIssuance = ctx.view.read(
+                    keylet::mptIssuance(ctx.tx[sfMPTokenIssuanceID]));
+                assert(sleMptIssuance);
+                if (!sleMptIssuance)
+                    return tefINTERNAL;


I feel like this check is technically unnecessary. We assert it is always true.

Ed proposed this change

It would make more sense to remove the assert

thejohnfreeman · 2024-10-22T17:59:01Z

src/xrpld/app/tx/detail/MPTokenAuthorize.cpp

+    if (flagsIn != flagsOut)
+        sleMpt->setFieldU32(sfFlags, flagsOut);
+
+    view.update(sleMpt);


This line can move into the previous then-block.

Doesn't hurt to leave it as it as is IMO

src/xrpld/ledger/View.h

src/xrpld/ledger/detail/View.cpp

thejohnfreeman · 2024-10-22T18:45:48Z

src/xrpld/app/tx/detail/MPTokenIssuanceCreate.cpp

+    if (!ctx.rules.enabled(featureMPTokensV1))
+        return temDISABLED;
+
+    if (auto const ret = preflight1(ctx); !isTesSuccess(ret))


These branches, here and elsewhere, can all be simplified.

Suggested change

if (auto const ret = preflight1(ctx); !isTesSuccess(ret))

if (auto const ret = preflight1(ctx))

I think this is fine. Every other transactions has this. And we are returning the tec code if it is not tesSuccess, so I dont see how this simplification would work.

"Other transactions have this" is not a good reason. This simplification works because a TER evaluates to true if it is not success, just like a Linux error code.

You know, I've gotten so used to seeing the symbolic isTesSuccess, that I've never really thought about the fact that tesSUCCESS == 0. But, yeah, this optimization / simplification should work.

The question is what does it gain us?
On one hand, if I see isTesSuccess (or any of the other is* test functions), I don't have to think about the underlying enum values. Additionally, isTesSuccess is defined as

inline bool isTesSuccess(TER x) { return ((x) == tesSUCCESS); }

Since that's inline, when compiled it should just boil down to x == 0. As an optimization, this probably doesn't gain much.

On the other hand, as a simplification, it does read a lot nicer, and fits the pattern we use everywhere else the result can be evaluated as a bool. I think I personally would adjust pretty quickly to seeing it this way, so I still wouldn't have to think about the underlying values.

My vote is to remove the call to isTesSuccess, and optionally add a comment anywhere it's removed to remind readers that this works.

Suggested change

if (auto const ret = preflight1(ctx); !isTesSuccess(ret))

// tesSUCCESS = 0, so any other return will be returned.

if (auto const ret = preflight1(ctx))

Another idea, similar to my first suggestion:

Suggested change

if (auto const ret = preflight1(ctx); !isTesSuccess(ret))

// tesSUCCESS == 0. Any other value indicates an error.

if (auto const err = preflight1(ctx))

(then return err;)
Using the name err instantly communicates "This function will return an error if it doesn't succeed."

Co-authored-by: John Freeman <[email protected]>

Bronek · 2024-10-23T17:39:03Z

#5042 is blocked by this PR

ximinez

I'm adding this comment just to reset Github's "Show changes since your last review" pointer.

ximinez

Done! (Other than pending changes / requests.)

src/test/app/MPToken_test.cpp

Co-authored-by: Ed Hennis <[email protected]>

* Update MPT unit-test

src/xrpld/app/tx/detail/MPTokenIssuanceCreate.cpp

src/xrpld/ledger/detail/View.cpp

include/xrpl/protocol/detail/sfields.macro

include/xrpl/protocol/Protocol.h

src/xrpld/app/tx/detail/MPTokenIssuanceCreate.cpp

src/test/app/MPToken_test.cpp

* rename * space

ximinez

A couple of my comments remain unaddressed:

#5143 (comment)

#5143 (review)

src/test/app/MPToken_test.cpp
src/xrpld/app/tx/detail/MPTokenIssuanceCreate.h

* Remove asserts in tests * Update copyright

ximinez

I think we're there.

Amendment: - MPTokensV1 New Transactions: - MPTokenIssuanceCreate - MPTokenIssuanceDestroy - MPTokenIssuanceSet - MPTokenAuthorize Modified Transactions: - Payment - Clawback New Objects: - MPTokenIssuance - MPToken API updates: - ledger_entry - account_objects - ledger_data Other: - Add += and -= operators to ValueProxy Read full spec: https://github.com/XRPLF/XRPL-Standards/tree/master/XLS-0033d-multi-purpose-tokens --------- Co-authored-by: Shawn Xie <[email protected]> Co-authored-by: Howard Hinnant <[email protected]> Co-authored-by: Ed Hennis <[email protected]> Co-authored-by: John Freeman <[email protected]>

gregtatcam mentioned this pull request Sep 19, 2024

Introduce MPT V1 support (XLS-33d) #5108

Closed

13 tasks

gregtatcam requested review from HowardHinnant and ximinez September 19, 2024 07:43

gregtatcam force-pushed the feature/mpt-v1-var-issues branch 2 times, most recently from 4ca85a2 to ba5f67c Compare September 19, 2024 14:22

gregtatcam force-pushed the feature/mpt-v1-var-issues branch from ba5f67c to 6d6fda2 Compare September 27, 2024 14:01

HowardHinnant reviewed Sep 30, 2024

View reviewed changes

thejohnfreeman requested changes Sep 30, 2024

View reviewed changes

Merge branch 'develop' into feature/mpt-v1-var-issues

aaf7fe5

ximinez requested changes Sep 30, 2024

View reviewed changes

gregtatcam and others added 3 commits October 1, 2024 13:25

Apply suggestions from code review

f84e9ea

Co-authored-by: Ed Hennis <[email protected]>

Address reviewer's feedback

18515dd

Allow MPT SendMax in Payment tx.

aef1426

Update MPT Payment errors to be consistent with IOU Payment.

ximinez requested changes Oct 1, 2024

View reviewed changes

gregtatcam and others added 2 commits October 2, 2024 07:47

Apply suggestions from code review

9136a89

Co-authored-by: Ed Hennis <[email protected]>

Address reviewer's feedback

16a029a

thejohnfreeman requested changes Oct 2, 2024

View reviewed changes

ximinez requested changes Oct 2, 2024

View reviewed changes

gregtatcam and others added 4 commits October 3, 2024 08:45

Manually align jss and disable clang-format

2beb2d9

Apply suggestions from code review

3cc7003

Co-authored-by: Ed Hennis <[email protected]>

Fix clang format

069273c

Apply suggestions from code review

86bcebd

use std::min, std::max Co-authored-by: Ed Hennis <[email protected]>

thejohnfreeman mentioned this pull request Oct 3, 2024

Consolidate definitions of fields, objects, transactions, and features #5122

Merged

Address reviewer's feedback

b9cb9ca

* Refactor getSNValue/getMPTValue * Use std::min, std::max * Fix min/max values for MPT

thejohnfreeman reviewed Oct 3, 2024

View reviewed changes

include/xrpl/protocol/TxFlags.h Outdated Show resolved Hide resolved

shawnxie999 and others added 3 commits October 3, 2024 11:07

remove unneeded fields (#36)

8af452e

rename codes (#37)

fab0c78

Address reviewer feedback

ea887bf

gregtatcam and others added 4 commits October 21, 2024 13:15

Address reviewer's feedback - refactor mpt.[h,cpp]

e8c8329

Apply suggestions from code review - update MPT unit-tests

c53df59

Co-authored-by: Ed Hennis <[email protected]>

Fix clang-format

b8f3b83

Address reviewer's feedback - add MPT unit-tests

5349827

thejohnfreeman requested changes Oct 22, 2024

View reviewed changes

gregtatcam and others added 3 commits October 22, 2024 15:09

Apply suggestions from code review

0905e37

Co-authored-by: John Freeman <[email protected]>

Address reviewer's feedback

9d84f3b

address comments (#46)

e8201f3

Bronek mentioned this pull request Oct 23, 2024

Add Antithesis intrumentation #5042

Closed

9 tasks

ximinez reviewed Oct 23, 2024

View reviewed changes

ximinez requested changes Oct 24, 2024

View reviewed changes

gregtatcam and others added 2 commits October 24, 2024 18:48

Apply suggestions from code review

5aa36e4

Co-authored-by: Ed Hennis <[email protected]>

Address reviewer's feedback

7a6272e

* Update MPT unit-test

sappenin suggested changes Oct 24, 2024

View reviewed changes

sappenin reviewed Oct 25, 2024

View reviewed changes

src/xrpld/app/tx/detail/MPTokenIssuanceCreate.cpp Show resolved Hide resolved

comments (#48)

3439bfa

ximinez reviewed Oct 25, 2024

View reviewed changes

src/test/app/MPToken_test.cpp Outdated Show resolved Hide resolved

src/test/app/MPToken_test.cpp Show resolved Hide resolved

Rename mpt holder (#49)

b636836

* rename * space

sappenin approved these changes Oct 28, 2024

View reviewed changes

ximinez requested changes Oct 29, 2024

View reviewed changes

gregtatcam and others added 2 commits October 29, 2024 12:37

Address reviewer's feedback

7c08b58

* Remove asserts in tests * Update copyright

Ed's comments (#50)

9ce2061

thejohnfreeman approved these changes Oct 29, 2024

View reviewed changes

ximinez approved these changes Oct 29, 2024

View reviewed changes

shawnxie999 added the Passed Passed code review & PR owner thinks it's ready to merge. Perf sign-off may still be required. label Oct 29, 2024

ximinez merged commit 23c37fa into XRPLF:develop Oct 29, 2024
19 of 20 checks passed

This was referenced Nov 6, 2024

Proposed 2.3.0-rc1 #5182

Closed

Proposed 2.3.0-rc1 #5183

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Introduce MPT support (XLS-33d): #5143

Introduce MPT support (XLS-33d): #5143

gregtatcam commented Sep 19, 2024

codecov bot commented Sep 19, 2024 •

edited

Loading

HowardHinnant Sep 30, 2024

ximinez Sep 30, 2024

thejohnfreeman left a comment

ximinez left a comment

ximinez left a comment

ximinez Oct 1, 2024

thejohnfreeman Oct 2, 2024

gregtatcam Oct 3, 2024

gregtatcam Oct 3, 2024

thejohnfreeman Oct 3, 2024

thejohnfreeman Oct 2, 2024

ximinez left a comment

thejohnfreeman Oct 22, 2024

shawnxie999 Oct 23, 2024

thejohnfreeman Oct 22, 2024

shawnxie999 Oct 23, 2024

ximinez Oct 25, 2024

thejohnfreeman Oct 22, 2024

shawnxie999 Oct 23, 2024

thejohnfreeman Oct 22, 2024

shawnxie999 Oct 23, 2024 •

edited

Loading

thejohnfreeman Oct 24, 2024

ximinez Oct 24, 2024

ximinez Oct 25, 2024

Bronek commented Oct 23, 2024

ximinez left a comment

ximinez left a comment

ximinez left a comment

ximinez left a comment


		namespace ripple {

		class MPTAmount : private boost::totally_ordered<MPTAmount>,

	if (auto const ret = preflight1(ctx); !isTesSuccess(ret))
	if (auto const ret = preflight1(ctx))

	if (auto const ret = preflight1(ctx); !isTesSuccess(ret))
	// tesSUCCESS = 0, so any other return will be returned.
	if (auto const ret = preflight1(ctx))

	if (auto const ret = preflight1(ctx); !isTesSuccess(ret))
	// tesSUCCESS == 0. Any other value indicates an error.
	if (auto const err = preflight1(ctx))

Introduce MPT support (XLS-33d): #5143

Introduce MPT support (XLS-33d): #5143

Conversation

gregtatcam commented Sep 19, 2024

High Level Overview of Change

Context of Change

Type of Change

API Impact

Test Plan

Future Tasks

codecov bot commented Sep 19, 2024 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

thejohnfreeman left a comment

Choose a reason for hiding this comment

ximinez left a comment

Choose a reason for hiding this comment

ximinez left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ximinez left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

shawnxie999 Oct 23, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Bronek commented Oct 23, 2024

ximinez left a comment

Choose a reason for hiding this comment

ximinez left a comment

Choose a reason for hiding this comment

ximinez left a comment

Choose a reason for hiding this comment

ximinez left a comment

Choose a reason for hiding this comment

codecov bot commented Sep 19, 2024 •

edited

Loading

shawnxie999 Oct 23, 2024 •

edited

Loading