Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion lib/default.nix
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,8 @@ let
mod compare splitByAndCompare functionArgs setFunctionArgs isFunction
toHexString toBaseDigits;
inherit (self.fixedPoints) fix fix' converge extends composeExtensions
composeManyExtensions makeExtensible makeExtensibleWithCustomName;
composeManyExtensions makeExtensible makeExtensibleWithCustomName
encapsulate;
inherit (self.attrsets) attrByPath hasAttrByPath setAttrByPath
getAttrFromPath attrVals attrValues getAttrs catAttrs filterAttrs
filterAttrsRecursive foldAttrs collect nameValuePair mapAttrs
Expand Down
90 changes: 90 additions & 0 deletions lib/fixed-points.nix
Original file line number Diff line number Diff line change
Expand Up @@ -110,4 +110,94 @@ rec {
fix' rattrs // {
${extenderName} = f: makeExtensibleWithCustomName extenderName (extends f rattrs);
};

/*
Creates an overridable attrset with encapsulation.

This is like `makeExtensible`, but only the `public` attribute of the fixed
point is returned.

Synopsis:

r = encapsulate (final@{extend, ...}: {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Putting extend in final is conceptually not valid. It should be a separate parameter.

(It's the kind of code you end up with when trying to stick too close to past patterns, as in #119942)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Putting extend in final is conceptually not valid. It should be a separate parameter.

I believe this comes from the discussion in #157056 (comment)? If so, I think the concerns there have been cleared

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed your comment seems to be about the same kind of problem.

I think I was setting the bar too high when I started this thread. What's the alternative? Add an extra parameter?
Reserving a name in the attrset of protected attributes is not pretty, but gets the job done quite well, without burdening the user with extra syntax.

The extra parameter will cause it to look like this:

lib.encapsulate (extend: this: {
  # ...
  public.withSomething = s: extend (extend: this: super: {
    something = s;
  });
});

Doesn't seem like a usability improvement, especially if you're not actually going to use extend.

this.extend seems better now, even if it means that users can't use that name.
That's probably for the best though, because we should having too many overriding mechanisms. If they do need to use that name, the problem is that they have one too many overriding mechanisms.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suppose a more realistic alternative is

lib.encapsulate ({ this, extend, ... }: {
  # ...
  public.withSomething = s: extend ({ this, extend, super, ... }: { something = s; }); });
}

We don't have renaming for such parameters though, which can make nested use awkward, as seen with not-just-data submodules, which have the same problem.

rant and analogy with module system

Also before you know it, someone will want to extend that attrset, and we'll have encapsulateWith { specialArgs = ...; }, even though super can fill the role of specialArgs. They're actually oddly similar.
Overlays have // for "lateral" composition (idk, is that term taken?), whereas the module system has option merging for that. extendModules then lines up with extends. extendModules relies on the lexical scope (ie config) to provide the "super" context, whereas in overlays it's an explicit parameter. specialArgs is effectively an extension of all modules' lexical scope, so it's a close analog of super.
This seems like an overreaction though.

The lack of nested "formals" ({ this@{ foo, ... }, ... }: <body>) could be a feature though, because then it's not possible to do the equivalent of what's currently lib.encapsulate ({ foo, ... }: <body>), which doesn't work, because such lambdas are strict (a phenomenon very relevant to lazyFunction, for some context).


# ... private attributes for `final` ...

public = {
# ... returned attributes for r, in terms of `final` ...
inherit extend; # optional, don't invoke too often; see below
};
})

s = r.extend (final: previous: {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
s = r.extend (final: previous: {
s = r.extend (this: old: {

I like this as a convention because it has a mnemonic: the reverse, "old this" can only be a description of one of the parameters and is therefore the wrong order.

We should probably not reuse names of overlays, so that overlays and encapsulate can be used together more easily. (For instance using an haskellPackages overlay inside a Nixpkgs overlay is quite common, and awkward.)

  • self: super:: overlays
  • final: prev:: Nixpkgs overlays
  • finalAttrs: prevAttrs:: overrideAttrs
  • this: old:: Things with overlay-like overriding and encapsulation.


# ... updates to private attributes ...

# optionally
public = previous.public // {
# ... updates to public attributes ...
};
})

= Performance

The `extend` function evaluates the whole fixed point all over, reusing
no "intermediate results" from the existing object.
This is necessary, because `final` has changed.
So the cost is quadratic; O(n^2) where n = number of chained invocations.
This has consequences for interface design.
Although enticing, `extend` is not suitable for directly implementing "fluent interfaces", where the caller makes many calls to `extend` via domain-specific "setters" or `with*` functions.
Fluent interfaces can not be implemented efficiently in Nix and have very little to offer over attribute sets in terms of usability.*

Example:

# cd nixpkgs; nix repl lib

nix-repl> multiplier = encapsulate (self: {
a = 1;
b = 1;
public = {
r = self.a * self.b;

# Publishing extend makes the attrset open for any kind of change.
inherit (self) extend;

# Instead, or additionally, you can add domain-specific functions.
# Offer a single method with multiple arguments, and not a
# "fluent interface" of a method per argument, because all extension
# functions are called for every `extend`. See the Performance section.
withParams = args@{ a ? null, b ? null }: # NB: defaults are not used
self.extend (self: super: args);

};
})

nix-repl> multiplier
{ extend = «lambda»; r = 1; withParams =«lambda»; }

nix-repl> multiplier.withParams { a = 42; b = 10; }
{ extend = «lambda»; r = 420; withParams =«lambda»; }

nix-repl> multiplier3 = multiplier.extend (self: super: {
c = 1;
public = super.public // {
r = super.public.r * self.c;
};
})

nix-repl> multiplier3.extend (self: super: { a = 2; b = 3; c = 10; })
{ extend = «lambda»; r = 60; withParams =«lambda»; }

(*) Final note on Fluent APIs: While the asymptotic complexity can be fixed
by avoiding overlay extension or perhaps using it only at the end of the
chain only, one problem remains. Every method invocation has to produce
a new, immutable state value, which means copying the whole state up to
that point.

*/
encapsulate = layerZero:
let
fixed = layerZero ({ extend = f: encapsulate (extends f layerZero); } // fixed);
Copy link
Member Author

@roberth roberth Mar 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Idea:

Suggested change
fixed = layerZero ({ extend = f: encapsulate (extends f layerZero); } // fixed);
fixed = layerZero (fixed // { extend = f: encapsulate ((fixed.extends or extends) f layerZero); });

This allows the base layer (or subsequent layers) to pick a "merge function" that implements extension differently - ie not overlay-style extension, but something more suitable for the use case.
Possible improvements that can be opted into this way: attribute deletion, nested merging, or a layered cousin of the module system.
Maybe the predictability of lib.extends is an important feature though.

in fixed.public;
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
in fixed.public;
in fixed.result;

Instead of public, we could use result, to avoid being too much like object-oriented programming, as that seems to be a concern.

Personally I actually prefer public, because I find the association with object-oriented programming useful, even if it doesn't implement messages.

Similarly, imperative languages that implement functional features don't reinvent all names. For instance, when a language implements a functional list processing API, they use the simple function names that were supposed to be used with pure functions, even though that can't be enforced. map is just a better name than traverse, let alone traverseIO.
This kind of thing is accepted.

If we do insist on something like result, we should also rename encapsulate, and make it all about "an attrset fixpoint where one of the attrs is the result", or "an overridable scope where one of the variables is returned". I have no idea how to turn that into a name.


}