Faster pattern matching for built-in types

[effectfully]

This issue is for discussing approaches to improving performance of pattern matching for built-in types.

The current approach is inefficient. Details can be found in this Note, the gist is that currently pattern matching over lists is implemented as (pseudocode)

matchList :: [a] -> b -> (a -> [a] -> b) -> b
matchList xs z f = chooseList xs (\_ -> z) (\_ -> f (head xs) (tail xs)) ()

where chooseList, head and tail are all built-in functions (chooseList returns either its second or its third argument depending on whether its first argument is null or not, respectively). Therefore in case of nil we end up performing one builtin call and in case of cons we end up performing three builtin calls. Note how we also have to pass a unit argument around in order not to always evaluate all branches: Plutus is strict, hence chooseList xs z (f (head xs) (tail xs)) would have the wrong semantics.

There have been two proposals on how to implement faster pattern matching for built-in types:

1. allow expressing pattern matching built-in functions directly, implemented in [Builtins] Add support for pattern matching builtins #5486
2. piggy-back on the pattern matching machinery that we use for sums-of-products, implemented in [Builtins] Add 'ListToConstr' and 'DataToConstr' #5704

Both PRs are documented, see the initial comment on each PR if you want to understand the details of design and implementation.

Both approaches work and give us comparable speedup on built-in-list-specific benchmarks.

This is what we get for (1):

This is what we get for (2):

It may appear that (2) is faster, however the benchmarking machine apparently is capable (see this slack discussion) of being wrong by 6% and probably even more, hence we can't really rely on these results, but it's still clear that both the approaches give us meaningful improvement of about the same scale.

If we analyze performance analytically, this is what we'll find:

1. (1) has to keep unit arguments around in order for pattern matching not to force all the branches at once and only force the picked one, while (2), being backed by SOPs, has no such restriction since case is specifically designed to be a proper pattern matching construct (unlike function application which is strict in Plutus). E.g. for (1) we have the following diff:

while for (2) it's

I.e. (2) wins for this one.

2. in (1) matchList is literally a single built-in function call, while in (2) it's a built-in function call + a case:

which affects not only performance, but also size, which is an even scarcer resource.

I.e. (1) wins for this one.

3. (1) requires us to amend the builtins machinery in a way that allows for returning an applied function. Not only is this a much larger change (propagating through many parts of the builtins machinery, including tests) compared to what (2) requires, it also introduces a slowdown for all existing builtins, because handling exactly one output of a built-in function is a bit faster than handling any non-zero number of them (if a builtin returns f x y we simply return all of those terms separately and let the evaluator handle reducing the application -- we have to do that, because each evaluator deals with reducing applications differently), because one doesn't need to case on the result to figure out if it contains a single output or multiple of them.

I.e. (2) wins for this one.

4. the whole issue we've been discussing here is improving performance of pattern matching for built-in types. Extending the builtins machinery seems very appropriate for that, particularly with something as straightforward as "allow for returning a function application", while teaching builtins about SOPs and hardcoding a specific SOP representation right into some built-in functions feels weird -- why do builtins and SOPs have to be intertwined this way?

Maybe it's fine, but I believe (1) wins for this one.

5. as per Michael's comment SOPs aren't supported with all versions of Plutus, so we need to figure out what to do for early ones that don't support SOPs

I.e. (1) wins for this one.

Overall, there's no clear winner. Performance comparison is unclear with both approaches delivering meaningful improvement, although my very subjective feeling is that (2) slightly wins when it comes to performance. But also (1) makes builtins more expressive, maybe for some another reason we'll end up needing to return function applications from builtins eventually anyway?

I'm personally torn between the two options. Which one should we choose?

[MicroProofs]

One thing I would like about the second one is if data_to_constr converted the constr tag to a sum of product as well.

Basically this
Constr i ds -> toConstr 0 [fromValue (toConstr i [fromValue ds])]

[effectfully]

One thing I would like about the second one is if data_to_constr converted the constr tag to a sum of product as well.

Oh yeah, that's on the menu, I just didn't implement it, because it's a bunch of boilerplate and I'm not sure if we're going to pick this option rather than the other one, hence might potentially result in time waste. I'm going to speed up pattern matching for tuples too.

Also we do remember about the request for the scrutinee to come last, I didn't reflect it here, because I didn't want to change anything mid-way. Once we've settled on an approach, we'll discuss the idea with the team and likely change the implementation accordingly.

[michaelpj]

(1) has to keep unit arguments around in order for pattern matching not to force all the branches at once and only force the picked one

I had to think about this a bit. If we abstract over the matcher with a function we lose this property (since function arguments get forced). But maybe we recover it with inlining, plus we could push more applications into case branches. (Although this is tricky in UPLC since we need to know how many lambdas to go under. Boo. An argument I didn't think of for making the bindings for case variables explicit in the term...)

Not only is this a much larger change (propagating through many parts of the builtins machinery, including tests) compared to what (2) requires, it also introduces a slowdown for all existing builtins

Is this visible in other benchmarks?

One thing I would like about the second one is if data_to_constr converted the constr tag to a sum of product as well.

Right, this is just "pattern matching on Data" and not "pattern-matching on datatypes encoded using Data", which would also be convenient but we still don't know how to type it.

---

I'm personally torn between the two options. Which one should we choose?

I don't know either. It's funny there is no performance difference, that was a potential decision point.

Did we get any input from @mjaskelioff about whether the change to allow builtins to return an application spine would affect the formalism at all?

[mjaskelioff]

Did we get any input from @mjaskelioff about whether the change to allow builtins to return an application spine would affect the formalism at all?

Returning an application spine would affect the formalism, but I don't see it as being too problematic. The main complexity in the builtin formalisation is in consuming the arguments, rather than in the return type.

From the point of view of specification (formal or not) it seems like a win for (2) as (2) is simpler to explain than (1), were we would need to introduce spines.

[effectfully]

If we abstract over the matcher with a function we lose this property (since function arguments get forced). But maybe we recover it with inlining

But we're not inlining matchList in case of (1), because it's literally a standalone builtin call and so there's nothing to inline. Or did I misunderstand you?

we could push more applications into case branches.

Yup.

Although this is tricky in UPLC since we need to know how many lambdas to go under. Boo.

Yeah, that sucks.

An argument I didn't think of for making the bindings for case variables explicit in the term...)

None of us did.

Is this visible in other benchmarks?

Yes, but the benchmarking machine is unreliable, so no.

I don't know either. It's funny there is no performance difference, that was a potential decision point.

Well I'm sure there's some, it's just that it's not big enough for us to be able to detect it using broken tools.

[kwxm]

I don't have much to say, I find this (about option1) a bit disturbing:

a much larger change (propagating through many parts of the builtins machinery, including tests)

And also this:

maybe for some another reason we'll end up needing to return function applications from builtins eventually anyway?

That sounds a bit like premature future-proofing that we may never need. I'm sure in the past I've been worried about the possibility of builtins returning more general terms for some reason, but I don't recall the details at the moment.

[kwxm]

About the possibility of buitlins returning applications: could we return a saturated application wrapped in a delay that you then have to force? I haven't thought it through properly, but such a thing would be a CekValue, so it would fit with the way the evaluator works at present.

[michaelpj]

Outcome of our discussion:

• Some mild feeling in favour of SOPs
• Need to work out what happens if you call a builtin that returns a SOP value in a PLC program that doesn't support SOPs
• Other potentially fast things we could do includes folds by returning a tree of applications
  • But: is this faster? Unclear: you factor out the recursion but you still have to process the tree of applications
  • Should try it: if it's not faster then this approach seems like a dead end

[michaelpj]

Hang on, doesn't the SOP approach also violate our old assumption that the only types which appear in the signatures of builtins are either a) builtin types or b) type variables? Now we would be able to have an SOP type there.

[mjaskelioff]

Yes. We would have to go for extending the notion of a) and saying that a SOP of builtin types is a builtin type.

[mjaskelioff]

In any case, what this discussion shows is that these are not just any new builtins: both ideas require a change to the language. So the discussion needs to go deeper and cannot rely solely on implementations: we need to understand the consequences on the language design. What would be the specification of the new features? How would they affect the type system of TPLC?