Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Builtins] Only be 'Lazy' for the result of a builtin application #4607

Closed

Conversation

effectfully
Copy link
Contributor

Same as #4581 except the history is correct.

@effectfully
Copy link
Contributor Author

/benchmark plutus-benchmark:validation

Copy link
Member

@zliu41 zliu41 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few questions to help me understand this. Nice improvements by the way!

@@ -51,7 +54,12 @@ instance NFData (RuntimeScheme n) where
-- argument of the denotation and calling 'makeKnown' over its result.
type ToRuntimeDenotationType :: GHC.Type -> Peano -> GHC.Type
type family ToRuntimeDenotationType val n where
ToRuntimeDenotationType val 'Z = MakeKnownM val
-- We use 'Lazy' here, because we don't want to compute the denotation when it's fully saturated
-- before figuring out what it's going to cost. An alternative is to be really careful and never
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we don't want to compute the denotation when it's fully saturated before figuring out what it's going to cost

What does this mean? Do we sometimes not compute the result if the cost is too high?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we sometimes not compute the result if the cost is too high?

Exactly. If something costs more than what the budget allows, we don't compute the builtin application and return an out-of-ex failure immediately.

~(ToRuntimeDenotationType val n) -- Must be lazy, because we don't want to compute the
-- denotation when it's fully saturated before figuring
-- out what it's going to cost.
(ToRuntimeDenotationType val n)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you have an example of this value being unnecessarily forced before? I didn't spot any.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be very wrong to let people run what they don't have a budget for, so we have tests for that. If you remove that tilda on master and run cabal test plutus-core-test, you'll get

     ExpensiveSucc:                                          FAIL
        untyped-plutus-core/test/Evaluation/Builtins/Definition.hs:200:
        expected: Right (Right EvaluationFailure)
         but got: Right (Left BuiltinErrorCall)
        Use -p '/ExpensiveSucc/' to rerun this test only.
      ExpensivePlus:                                          FAIL
        untyped-plutus-core/test/Evaluation/Builtins/Definition.hs:230:
        expected: Right (Right EvaluationFailure)
         but got: Right (Left BuiltinErrorCall)
        Use -p '/ExpensivePlus/' to rerun this test only.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah but I was wondering why Lazy is helpful. The reason you added Lazy is because this value was unnecessarily forced to WHNF somewhere - right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, yeah, see the competing PR that removes the unnecessary forcing. Somehow it was worse than the Lazy approach.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well maybe I need some more coffee 😂 but the competing PR simply avoids constructing and destructing BuiltinRuntime. But since the ~(ToRuntimeDenotationType val n) field is lazy, constructing and destructing BuiltinRuntime doesn't force it, does it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the competing PR simply avoids constructing and destructing BuiltinRuntime. But since the ~(ToRuntimeDenotationType val n) field is lazy

Constructing and deconstructing is what forces us to have that tilda there. The competing PR avoids the constructing and deconstructing, which allows it to remove the tilda (the first file in the diff).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does that mean the saving in the competing PR didn't come from avoiding unnecessary forcing (since there wasn't unnecessary forcing before either), it came from avoiding the construction and destruction? Then why would this PR help since it doesn't avoid construction/destruction, but just replace the tilda with Lazy?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it came from avoiding the construction and destruction?

Avoiding the construction and deconstruction is why the competing PR should've been faster than the Lazy approach. However it's not the reason why the two PR exist. The point is that while we want to avoid premature forcing of the result, we don't want to avoid forcing of every intermediate partial application as keeping those thunks likely costs us some. I.e. we want to be lazy only for a fully saturated builtin and not for any partial application.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah Ok, thanks, now I get it 😂

@iohk-devops
Copy link

Comparing benchmark results of 'plutus-benchmark:validation' on '4af22f9fb' (base) and 'a2c72aa7d' (PR)

Results table
Script 4af22f9 a2c72aa Change
auction_1-1 201.6 μs 205.2 μs +1.8%
auction_1-2 817.7 μs 835.0 μs +2.1%
auction_1-3 807.5 μs 823.6 μs +2.0%
auction_1-4 257.8 μs 261.3 μs +1.4%
auction_2-1 201.8 μs 205.5 μs +1.8%
auction_2-2 818.6 μs 833.2 μs +1.8%
auction_2-3 1.045 ms 1.065 ms +1.9%
auction_2-4 811.7 μs 823.8 μs +1.5%
auction_2-5 258.1 μs 262.3 μs +1.6%
crowdfunding-success-1 236.8 μs 240.3 μs +1.5%
crowdfunding-success-2 237.0 μs 240.1 μs +1.3%
crowdfunding-success-3 236.1 μs 240.0 μs +1.7%
currency-1 300.9 μs 306.2 μs +1.8%
escrow-redeem_1-1 429.7 μs 435.2 μs +1.3%
escrow-redeem_1-2 429.2 μs 434.9 μs +1.3%
escrow-redeem_2-1 504.9 μs 512.5 μs +1.5%
escrow-redeem_2-2 504.9 μs 513.0 μs +1.6%
escrow-redeem_2-3 504.0 μs 512.9 μs +1.8%
escrow-refund-1 177.3 μs 182.6 μs +3.0%
future-increase-margin-1 301.9 μs 308.1 μs +2.1%
future-increase-margin-2 683.3 μs 690.6 μs +1.1%
future-increase-margin-3 683.0 μs 691.8 μs +1.3%
future-increase-margin-4 641.6 μs 649.6 μs +1.2%
future-increase-margin-5 1.015 ms 1.026 ms +1.1%
future-pay-out-1 302.1 μs 306.7 μs +1.5%
future-pay-out-2 681.3 μs 689.1 μs +1.1%
future-pay-out-3 681.8 μs 688.9 μs +1.0%
future-pay-out-4 1.013 ms 1.022 ms +0.9%
future-settle-early-1 301.2 μs 306.2 μs +1.7%
future-settle-early-2 678.7 μs 690.8 μs +1.8%
future-settle-early-3 678.7 μs 690.4 μs +1.7%
future-settle-early-4 768.4 μs 780.0 μs +1.5%
game-sm-success_1-1 485.6 μs 493.2 μs +1.6%
game-sm-success_1-2 221.8 μs 223.8 μs +0.9%
game-sm-success_1-3 810.2 μs 819.6 μs +1.2%
game-sm-success_1-4 232.1 μs 260.2 μs +12.1%
game-sm-success_2-1 486.2 μs 496.0 μs +2.0%
game-sm-success_2-2 221.1 μs 225.2 μs +1.9%
game-sm-success_2-3 813.1 μs 823.6 μs +1.3%
game-sm-success_2-4 233.5 μs 260.4 μs +11.5%
game-sm-success_2-5 812.8 μs 821.4 μs +1.1%
game-sm-success_2-6 233.7 μs 259.7 μs +11.1%
multisig-sm-1 503.4 μs 510.5 μs +1.4%
multisig-sm-2 491.1 μs 496.5 μs +1.1%
multisig-sm-3 493.5 μs 502.5 μs +1.8%
multisig-sm-4 499.9 μs 508.1 μs +1.6%
multisig-sm-5 717.7 μs 729.1 μs +1.6%
multisig-sm-6 501.3 μs 511.1 μs +2.0%
multisig-sm-7 488.0 μs 496.5 μs +1.7%
multisig-sm-8 493.7 μs 501.5 μs +1.6%
multisig-sm-9 500.4 μs 507.5 μs +1.4%
multisig-sm-10 718.2 μs 728.6 μs +1.4%
ping-pong-1 412.8 μs 418.8 μs +1.5%
ping-pong-2 410.7 μs 416.8 μs +1.5%
ping-pong_2-1 240.0 μs 242.5 μs +1.0%
prism-1 183.8 μs 185.1 μs +0.7%
prism-2 521.6 μs 532.3 μs +2.1%
prism-3 443.6 μs 449.7 μs +1.4%
pubkey-1 156.0 μs 158.3 μs +1.5%
stablecoin_1-1 1.120 ms 1.129 ms +0.8%
stablecoin_1-2 217.6 μs 218.3 μs +0.3%
stablecoin_1-3 1.282 ms 1.292 ms +0.8%
stablecoin_1-4 231.1 μs 230.6 μs -0.2%
stablecoin_1-5 1.605 ms 1.613 ms +0.5%
stablecoin_1-6 284.4 μs 289.1 μs +1.7%
stablecoin_2-1 1.120 ms 1.131 ms +1.0%
stablecoin_2-2 217.0 μs 219.2 μs +1.0%
stablecoin_2-3 1.277 ms 1.288 ms +0.9%
stablecoin_2-4 229.9 μs 231.4 μs +0.7%
token-account-1 224.4 μs 229.0 μs +2.0%
token-account-2 399.6 μs 407.6 μs +2.0%
uniswap-1 511.4 μs 525.3 μs +2.7%
uniswap-2 261.9 μs 267.4 μs +2.1%
uniswap-3 2.073 ms 2.101 ms +1.4%
uniswap-4 379.0 μs 386.6 μs +2.0%
uniswap-5 1.413 ms 1.436 ms +1.6%
uniswap-6 364.3 μs 370.8 μs +1.8%
vesting-1 430.3 μs 441.9 μs +2.7%

@effectfully
Copy link
Contributor Author

Hm, Ok, I guess this somehow interferes with the MakeKnownM optimization, although I have no idea how. Perhaps it's time to try again the other PR.

@effectfully
Copy link
Contributor Author

/benchmark plutus-benchmark:validation

@iohk-devops
Copy link

Comparing benchmark results of 'plutus-benchmark:validation' on '4af22f9fb' (base) and 'fb6a43af7' (PR)

Results table
Script 4af22f9 fb6a43a Change
auction_1-1 202.3 μs 204.7 μs +1.2%
auction_1-2 818.1 μs 823.8 μs +0.7%
auction_1-3 806.3 μs 809.9 μs +0.4%
auction_1-4 258.2 μs 260.3 μs +0.8%
auction_2-1 203.1 μs 203.8 μs +0.3%
auction_2-2 817.6 μs 820.1 μs +0.3%
auction_2-3 1.042 ms 1.042 ms 0.0%
auction_2-4 805.4 μs 806.3 μs +0.1%
auction_2-5 258.6 μs 259.9 μs +0.5%
crowdfunding-success-1 236.0 μs 238.4 μs +1.0%
crowdfunding-success-2 237.1 μs 238.3 μs +0.5%
crowdfunding-success-3 237.4 μs 238.9 μs +0.6%
currency-1 302.5 μs 300.7 μs -0.6%
escrow-redeem_1-1 428.4 μs 430.0 μs +0.4%
escrow-redeem_1-2 428.5 μs 429.0 μs +0.1%
escrow-redeem_2-1 504.9 μs 504.6 μs -0.1%
escrow-redeem_2-2 504.1 μs 504.0 μs -0.0%
escrow-redeem_2-3 504.3 μs 504.2 μs -0.0%
escrow-refund-1 179.3 μs 179.5 μs +0.1%
future-increase-margin-1 301.3 μs 299.9 μs -0.5%
future-increase-margin-2 676.9 μs 678.4 μs +0.2%
future-increase-margin-3 678.7 μs 677.2 μs -0.2%
future-increase-margin-4 637.7 μs 637.9 μs +0.0%
future-increase-margin-5 1.012 ms 1.008 ms -0.4%
future-pay-out-1 302.1 μs 299.8 μs -0.8%
future-pay-out-2 680.2 μs 676.0 μs -0.6%
future-pay-out-3 679.8 μs 675.8 μs -0.6%
future-pay-out-4 1.012 ms 1.010 ms -0.2%
future-settle-early-1 301.1 μs 300.6 μs -0.2%
future-settle-early-2 680.9 μs 677.7 μs -0.5%
future-settle-early-3 682.6 μs 677.2 μs -0.8%
future-settle-early-4 774.9 μs 767.2 μs -1.0%
game-sm-success_1-1 488.6 μs 484.9 μs -0.8%
game-sm-success_1-2 221.9 μs 220.0 μs -0.9%
game-sm-success_1-3 811.9 μs 808.2 μs -0.5%
game-sm-success_1-4 231.8 μs 259.4 μs +11.9%
game-sm-success_2-1 485.0 μs 486.8 μs +0.4%
game-sm-success_2-2 220.5 μs 221.3 μs +0.4%
game-sm-success_2-3 809.1 μs 806.0 μs -0.4%
game-sm-success_2-4 232.5 μs 258.0 μs +11.0%
game-sm-success_2-5 810.7 μs 805.4 μs -0.7%
game-sm-success_2-6 231.7 μs 257.7 μs +11.2%
multisig-sm-1 504.5 μs 501.6 μs -0.6%
multisig-sm-2 490.5 μs 487.3 μs -0.7%
multisig-sm-3 495.8 μs 496.7 μs +0.2%
multisig-sm-4 500.7 μs 503.3 μs +0.5%
multisig-sm-5 720.5 μs 719.3 μs -0.2%
multisig-sm-6 504.9 μs 505.7 μs +0.2%
multisig-sm-7 493.0 μs 489.3 μs -0.8%
multisig-sm-8 497.6 μs 496.5 μs -0.2%
multisig-sm-9 502.3 μs 502.1 μs -0.0%
multisig-sm-10 719.8 μs 720.0 μs +0.0%
ping-pong-1 411.9 μs 414.0 μs +0.5%
ping-pong-2 411.9 μs 413.3 μs +0.3%
ping-pong_2-1 240.8 μs 242.5 μs +0.7%
prism-1 183.5 μs 186.2 μs +1.5%
prism-2 522.6 μs 524.8 μs +0.4%
prism-3 440.8 μs 443.4 μs +0.6%
pubkey-1 156.4 μs 156.6 μs +0.1%
stablecoin_1-1 1.110 ms 1.110 ms 0.0%
stablecoin_1-2 215.6 μs 217.6 μs +0.9%
stablecoin_1-3 1.271 ms 1.263 ms -0.6%
stablecoin_1-4 229.0 μs 230.6 μs +0.7%
stablecoin_1-5 1.586 ms 1.577 ms -0.6%
stablecoin_1-6 282.5 μs 284.7 μs +0.8%
stablecoin_2-1 1.112 ms 1.106 ms -0.5%
stablecoin_2-2 215.1 μs 216.2 μs +0.5%
stablecoin_2-3 1.273 ms 1.262 ms -0.9%
stablecoin_2-4 230.1 μs 229.9 μs -0.1%
token-account-1 226.0 μs 224.8 μs -0.5%
token-account-2 400.7 μs 397.1 μs -0.9%
uniswap-1 515.0 μs 509.5 μs -1.1%
uniswap-2 262.8 μs 263.6 μs +0.3%
uniswap-3 2.058 ms 2.046 ms -0.6%
uniswap-4 375.9 μs 383.1 μs +1.9%
uniswap-5 1.400 ms 1.396 ms -0.3%
uniswap-6 362.6 μs 366.6 μs +1.1%
vesting-1 430.8 μs 429.2 μs -0.4%

@effectfully
Copy link
Contributor Author

Closing in favor of #4610. I like this one more, but the other one actually helps.

@effectfully effectfully deleted the effectfully/builtins/only-be-lazy-for-the-result-2 branch May 12, 2022 00:00
@effectfully effectfully restored the effectfully/builtins/only-be-lazy-for-the-result-2 branch May 13, 2022 21:04
@effectfully effectfully reopened this May 13, 2022
@effectfully effectfully deleted the effectfully/builtins/only-be-lazy-for-the-result-2 branch May 13, 2022 21:05
@effectfully effectfully restored the effectfully/builtins/only-be-lazy-for-the-result-2 branch May 15, 2022 13:32
@effectfully effectfully reopened this May 15, 2022
@effectfully
Copy link
Contributor Author

/benchmark plutus-benchmark:validation

@iohk-devops
Copy link

Comparing benchmark results of 'plutus-benchmark:validation' on '4af22f9fb' (base) and '9674d36d7' (PR)

Results table
Script 4af22f9 9674d36 Change
auction_1-1 225.8 μs 203.5 μs -9.9%
auction_1-2 850.0 μs 825.1 μs -2.9%
auction_1-3 845.3 μs 814.9 μs -3.6%
auction_1-4 292.9 μs 260.5 μs -11.1%
auction_2-1 226.8 μs 204.7 μs -9.7%
auction_2-2 849.0 μs 821.3 μs -3.3%
auction_2-3 1.076 ms 1.043 ms -3.1%
auction_2-4 837.2 μs 808.1 μs -3.5%
auction_2-5 291.1 μs 259.4 μs -10.9%
crowdfunding-success-1 267.2 μs 238.1 μs -10.9%
crowdfunding-success-2 266.3 μs 238.3 μs -10.5%
crowdfunding-success-3 266.6 μs 238.4 μs -10.6%
currency-1 319.0 μs 301.3 μs -5.5%
escrow-redeem_1-1 462.4 μs 430.0 μs -7.0%
escrow-redeem_1-2 461.2 μs 429.9 μs -6.8%
escrow-redeem_2-1 541.4 μs 504.2 μs -6.9%
escrow-redeem_2-2 540.3 μs 502.6 μs -7.0%
escrow-redeem_2-3 543.4 μs 502.9 μs -7.5%
escrow-refund-1 202.2 μs 179.5 μs -11.2%
future-increase-margin-1 320.2 μs 300.8 μs -6.1%
future-increase-margin-2 718.9 μs 679.1 μs -5.5%
future-increase-margin-3 719.3 μs 677.7 μs -5.8%
future-increase-margin-4 671.3 μs 640.7 μs -4.6%
future-increase-margin-5 1.045 ms 1.013 ms -3.1%
future-pay-out-1 318.0 μs 300.3 μs -5.6%
future-pay-out-2 717.9 μs 676.8 μs -5.7%
future-pay-out-3 718.5 μs 676.7 μs -5.8%
future-pay-out-4 1.038 ms 1.008 ms -2.9%
future-settle-early-1 319.2 μs 299.5 μs -6.2%
future-settle-early-2 718.6 μs 678.9 μs -5.5%
future-settle-early-3 717.8 μs 680.6 μs -5.2%
future-settle-early-4 798.1 μs 772.9 μs -3.2%
game-sm-success_1-1 519.1 μs 486.7 μs -6.2%
game-sm-success_1-2 250.4 μs 220.2 μs -12.1%
game-sm-success_1-3 851.0 μs 806.8 μs -5.2%
game-sm-success_1-4 292.3 μs 258.8 μs -11.5%
game-sm-success_2-1 522.9 μs 488.9 μs -6.5%
game-sm-success_2-2 249.9 μs 222.2 μs -11.1%
game-sm-success_2-3 846.4 μs 808.5 μs -4.5%
game-sm-success_2-4 291.7 μs 258.9 μs -11.2%
game-sm-success_2-5 845.3 μs 805.9 μs -4.7%
game-sm-success_2-6 290.4 μs 258.7 μs -10.9%
multisig-sm-1 531.8 μs 504.6 μs -5.1%
multisig-sm-2 521.1 μs 490.2 μs -5.9%
multisig-sm-3 528.1 μs 496.5 μs -6.0%
multisig-sm-4 533.1 μs 503.3 μs -5.6%
multisig-sm-5 746.0 μs 718.0 μs -3.8%
multisig-sm-6 533.9 μs 505.3 μs -5.4%
multisig-sm-7 519.3 μs 491.0 μs -5.4%
multisig-sm-8 526.4 μs 495.9 μs -5.8%
multisig-sm-9 532.3 μs 503.4 μs -5.4%
multisig-sm-10 746.7 μs 716.2 μs -4.1%
ping-pong-1 438.7 μs 413.7 μs -5.7%
ping-pong-2 437.5 μs 413.3 μs -5.5%
ping-pong_2-1 261.5 μs 242.2 μs -7.4%
prism-1 208.2 μs 186.5 μs -10.4%
prism-2 564.6 μs 523.0 μs -7.4%
prism-3 478.7 μs 441.9 μs -7.7%
pubkey-1 177.2 μs 156.2 μs -11.9%
stablecoin_1-1 1.175 ms 1.111 ms -5.4%
stablecoin_1-2 244.2 μs 217.4 μs -11.0%
stablecoin_1-3 1.349 ms 1.267 ms -6.1%
stablecoin_1-4 258.0 μs 229.9 μs -10.9%
stablecoin_1-5 1.698 ms 1.584 ms -6.7%
stablecoin_1-6 320.3 μs 285.2 μs -11.0%
stablecoin_2-1 1.173 ms 1.108 ms -5.5%
stablecoin_2-2 244.2 μs 217.3 μs -11.0%
stablecoin_2-3 1.353 ms 1.269 ms -6.2%
stablecoin_2-4 259.3 μs 230.6 μs -11.1%
token-account-1 242.6 μs 225.3 μs -7.1%
token-account-2 431.3 μs 397.7 μs -7.8%
uniswap-1 535.4 μs 509.6 μs -4.8%
uniswap-2 283.2 μs 262.9 μs -7.2%
uniswap-3 2.186 ms 2.049 ms -6.3%
uniswap-4 429.3 μs 384.2 μs -10.5%
uniswap-5 1.512 ms 1.404 ms -7.1%
uniswap-6 412.1 μs 367.3 μs -10.9%
vesting-1 457.7 μs 431.7 μs -5.7%

@effectfully
Copy link
Contributor Author

effectfully commented May 15, 2022

-7.07%. The only thing I did is moving Lazy from PlutusPrelude to Runtime, see the commit.

@bezirg
Copy link
Contributor

bezirg commented May 16, 2022

fyi: Looking around in hackage for something an established way of the Lazy datatype, I only found this https://hackage.haskell.org/package/vector-0.12.3.1/docs/Data-Vector-Fusion-Util.html#t:Box

@effectfully
Copy link
Contributor Author

-7% is worse than -8%, hence choosing #4610 over this one.

@effectfully effectfully deleted the effectfully/builtins/only-be-lazy-for-the-result-2 branch May 17, 2022 23:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants