libexpr: use allocBytes() to allocate StringData

[Radvendii]

Motivation

EvalMemory::allocBytes() allows us to easily change how memory is allocated throughout the program. I have been using it to put all allocations in a bump allocator so I can more easily track memory leaks.

One problem is that this has to change the c bindings. I don't know how much of an issue this is as I haven't worked on the c bindings much until now.

Context

---

Add 👍 to pull requests you find important.

The Nix maintainer team uses a GitHub project board to schedule and track reviews.

[Ericson2314]

Looks good, just should update the C API first and let Robert know.

As far as I can tell, this is an unavoidable breaking change of the C API.

CC @roberth

[Radvendii]

To expand on what @Ericson2314 said, we discussed it in the meeting and we can anticipate that a similar change will be necessary for all the c api functions that create Values (eventually, when values themselves are stored in managed memory), so we should make those changes all now, and split it off into a separate PR. I'll try to get to that later this week.

[Radvendii]

As discussed in #14592, we don't actually need to change the C API for this (and shouldn't). When we're done with managed heap changes, maybe we should come out with an API v2, but in the meantime we can work around it for now.

This does come with some costs for the C API:

• nix_value is now 16 bytes on top of the 16 bytes for a nix::Value (it stores a Value * and EvalMemory *)
• We've added a layer of indirection to every nix_value access
• When we call functions with an array of arguments, we have to create a new array with the other type of pointer.

CC: @roberth @Ericson2314

[roberth]

some costs for the C API

This is usually insignificant. The number of values interacted with is generally not terribly big compared to the whole evaluation. For instance, something like nix build .#foo only needs to touch a handful of values that are returned to the CLI, whereas most values are intermediate results that are not affected by this.
It'd be a different story when, say, builtins.derivation were implemented through the C API, but I don't assume C API consumers operate at that scale.
If they do, I'd welcome their contributions, and a reasonable migration would be:

1. provide alternative functions that don't make use of the extra field
2. deprecate the old functions
3. wait for the latest Nix version without the new functions to go EOL
4. remove the old functions and the field

I see nix_init_string needs it. Any other functions?
The PR is ok, but perhaps we could do (1) so that we don't sit on the tech debt for too long.

[Radvendii]

Currently nix_init_string is the only one that needs it and didn't already have access, but when we get to putting Values in managed memory and referencing them by index, we'll need EvalMemory anywhere that we access a Value.

My concern with (1) is that it's not 100% clear what the interface should look like when we're done with more of the changes. I'd rather redesign it once when we're more sure what the constraints are going to be, rather than to redesign a bit now and then realise it needs even more redesign a later.

But that's a tradeoff. If you think it's better to do the intermediate redesign now I'll defer to you.

[roberth]

Got it. Can be done later, no problem.