Questions about testing smart contracts

[fallen-icarus]

In plutus-apps v1.2.0, there was the EmulatorTrace monad that allowed for creating arbitrary unit test scenarios. I was able to create general purpose endpoints like those in this module. I could then just create scenarios that submitted one transaction after another where I would test whether the last transaction in the scenario passed or failed. Here is an example module that used this approach (it submits one arbitrary transaction after another in a single unit test). The benefit of this approach was that it worked for all smart contracts and all transactions; I didn't need to create other endpoints just to see how other smart contract executions or transaction inputs would impact the contract I was testing. Even testing for double satisfaction was very straightforward.

In the process of trying to migrate away from plutus-apps v1.2.0, I have been trying to convert my smart contract tests to follow the examples here. However, after reading through a lot of the documentation and trying it out, it seems the tests make heavy use of modeling internal state and actions which does not seem appropriate for my use case. Ultimately, a blockchain action is just a transaction and the state is the current UTxO set, so I would rather just be able to specify what transaction to submit next like I did with EmulatorTrace. I tried using ContractModel like this (using a dummy state and a single Transact action) but the types don't seem to allow for this (see bottom of comment for example code). While quickcheck-contractmodel's ThreatModel does not seem to require modeling internal state, it seems very limited in what it can do (e.g., AFAICT it doesn't support testing reference script usage). Even the dynamic logic unit tests seem to require modeling internal state and actions.

The reason why I don't think modeling state and actions is the right approach for me (and arguably others) is that it makes it very difficult to test smart contract composability. All of my DApps (DEX, lending/borrowing, options trading, secondary market) are composable with each other and all other DApps within a single transaction. They are also composable with themselves within a single transaction (you can use 25 swaps from the DEX in a single transaction). Having to model the state and actions of all possible compositions just doesn't seem feasible and also doesn't seem necessary; I just want to know whether or not a transaction fails after a given series of previous transactions. Being able to just specify what transaction to submit next is IMO infinitely easier. I do think there is a place for property based testing like those in the examples, but unit tests like those I described seem to be more useful to me as a DApp developer.

So my questions are:

1. Did I miss something that would allow me to replicate the behavior of my previous tests?
2. If no, is there any intention to bring back some of the capabilities from EmulatorTrace?

What I've tried

-- | Mint or burn native tokens. Can use either a reference script or a local script.
data TokenMint = TokenMint 
  { mintTokens :: [(PlutusTx.TokenName,Integer)]
  , mintRedeemer :: PlutusTx.Redeemer
  , mintPolicy :: L.Versioned L.MintingPolicy
  , mintReference :: Maybe (L.Versioned L.TxOutRef)
  } deriving stock (Generic,Show,Eq)

-- | Create a transaction output at the specified address with the specified value, datum,
-- and reference script.
data Output = Output
  { outputAddress :: L.Address
  , outputValue :: V.Value
  , outputDatum :: PlutusTx.OutputDatum
  , outputReferenceScript :: L.ReferenceScript
  } deriving (Generic,Show,Eq)

-- | Used to create a transaction with the specified constraints.
data TransactionParams = TransactionParams
  { tokens :: [TokenMint]
  , outputs :: [Output]
  } deriving (Generic,Show,Eq)

-- | Create and submit a general transaction.
transact 
  :: (E.MonadEmulator m) 
  => CardanoAddress 
  -> [PaymentPrivateKey] 
  -> TransactionParams 
  -> m ()
transact mainWallet privKeys TransactionParams{..} = 
  let mintWitnessMap = C.BuildTxWith $ Map.fromList $ flip map tokens $ \TokenMint{..} -> 
        ( V.policyId mintPolicy
        , unsafeFromRight $ 
            C.toCardanoMintWitness mintRedeemer mintReference (Just mintPolicy) 
        )
      mintValue = flip foldMap tokens $ \TokenMint{..} -> 
        flip foldMap mintTokens $ \(tn,i) -> 
          V.singleton (V.policyId mintPolicy) (unsafeFromRight $ C.toCardanoAssetName tn) i
      outs = flip map outputs $ \Output{..} ->
        C.TxOut
          (unsafeFromRight $ C.toCardanoAddressInEra testnet outputAddress)
          (C.toCardanoTxOutValue outputValue)
          (unsafeFromRight $ C.toCardanoTxOutDatum outputDatum)
          outputReferenceScript
      tx =
        C.CardanoBuildTx $ E.emptyTxBodyContent
          { C.txMintValue = C.TxMintValue C.MaryEraOnwardsBabbage mintValue mintWitnessMap
          , C.txOuts = outs
          }
  in void $ E.submitUnbalancedTx mempty mainWallet privKeys tx
  
data TestState = TestState deriving (Generic,Show,Eq)

type Wallet = Integer

instance ContractModel TestState where
  data Action TestState = Transact Wallet [Wallet] TransactionParams
    deriving (Eq,Show,Generic)

  initialState = TestState

  nextState _ = QCCM.wait 1

  arbitraryAction _ = pure $ Transact 1 [1] $ TransactionParams [] []

But the above results in various Missing instance of HasVariables for non-Generic type ... errors.

[fallen-icarus]

I managed to get some basic tests working using my transact above (after changing the address type for Output) like:

test :: (E.MonadEmulator m) => m ()
test = do
  let w1 = W.knownMockWallet 1
      w2 = W.knownMockWallet 2
  transact (W.mockWalletAddress w1) [W.paymentPrivateKey w1] $ 
    TransactionParams 
      { tokens = []
      , outputs = 
          [ Output { outputAddress = W.mockWalletAddress w2
                   , outputValue = L.lovelaceToValue 100000000000000
                   , outputDatum = PlutusTx.NoOutputDatum
                   , outputReferenceScript = C.ReferenceScriptNone
                   }
          ]
      }

I could then just call it using checkPredicate with Test.Tasty.defaultMain. There are currently very few available EmulatorPredicates, though. This technically answered my question (I did miss something) so feel free to close this issue.

[fallen-icarus]

Actually, the checkPredicate functions and the EmulatorPredicates don't work for my needs. checkPredicateOptions assumes runEmulatorM will return a Right a for Either EmulatorError a but this assumption does not hold when a script execution fails. I personally have some test transactions that fail script evaluation during phase 2 and these always return a Left err. I would like to check that these scripts failed with the proper reason but I can't use the checkPredicate functions for this. I was able to get around this by making my own test functions that deliberately check the Left but this makes my test code fragile since errors like "UTxO output too small" still result in a Right for Either EmulatorError a (like when a reference script is stored with too little ADA). This means if I accidentally cause the test to fail for a different reason, it can result in a passing test just because it failed for a reason that still returns Right. I also can't seem to use the logs for checking phase 2 failure because the emulator will terminate before logging what happened. For example, with my phase 2 failing script execution, the last log I get is "Balancing an unbalanced transaction:".

Should I make this its own github issue?

[sjoerdvisscher]

Thanks for your interesting comments! These are sensible requests. At the moment I unfortunately can't promise these will be fixed soon.

[fallen-icarus]

At the moment I unfortunately can't promise these will be fixed soon.

All good! I know you guys are busy and I managed to find work-arounds for the above features. If you ever do find yourself with time, I would actually prefer if you prioritized my other issue since I have not been able to find a work-around for that one.

I appreciate the reply and I apologize for the rant-like nature of this issue 😅