libstore-c: add more derivation functions by RossComputerGuy · Pull Request #14031 · NixOS/nix

RossComputerGuy · 2025-09-19T22:52:17Z

Motivation

Upstreaming more C API exposure, specifically more derivation related things.

Context

Comes from DeterminateSystems#210

Add 👍 to pull requests you find important.

The Nix maintainer team uses a GitHub project board to schedule and track reviews.

src/libstore-c/nix_api_store.h

Ericson2314 · 2025-09-19T23:10:41Z

I have a preference that consumers should achieve these goals by deserializing the JSON themselves. Then we can make do with a smaller interface which will be easier to maintain and more stable.

src/libstore-c/nix_api_store.cc

Ericson2314 · 2025-09-19T23:13:35Z

src/libstore-c/nix_api_store.cc

+    NIXC_CATCH_ERRS
+}
+
+nix_err nix_derivation_get_outputs_and_optpaths(


This should just be the opt paths, which is useful for the FOD case. The outputs themselves can be gotten from the JSON.

This is me elaborating what I wrote in #14031 (comment)

RossComputerGuy · 2025-09-19T23:29:11Z

I have a preference that consumers should achieve these goals by deserializing the JSON themselves. Then we can make do with a smaller interface which will be easier to maintain and more stable.

How do I get the JSON myself? All I have is a store path.

Ericson2314 · 2025-09-19T23:34:33Z

Use your new function to read into memory derivation in store at given path
Use my function (recently added) to dump JSON from in memory drv

RossComputerGuy · 2025-09-19T23:37:47Z

Isn't there increased overhead if everything is being accessed through JSON? Wouldn't some operations make sense to do without deserializing?

Ericson2314 · 2025-09-19T23:50:46Z

I would say optimization should come later. Let's see evidence of there being a bottleneck in practice before we make the interface bigger.

Ericson2314 · 2025-09-23T03:52:31Z

Good to put from and to JSON next to each other
I still think we can get rid of nix_derivation entirely.
I want to make the output path setting easier in C++ too. I might try to land a patch for that.

Ericson2314 · 2025-09-23T20:11:22Z

Ah does anything use the BuildResult JSON today? Hopefully that is one I can fix as part of my big JSON rework next :)

RossComputerGuy · 2025-09-23T20:20:35Z

I'm looking at our usage and it looks like there's not much usage of it.

It doesn't work on unrealized paths.

Ericson2314 · 2025-10-01T01:03:24Z

src/libstore-c/nix_api_store.cc

+nix_err nix_derivation_make_outputs(
+    nix_c_context * context,
+    Store * store,
+    const char * json,


this should take a nix_derivation, not parse the JSON itself.

Then we'll have to remove the verify function to even be able to get a nix_derivation.

hmm? I am not sure what you mean. You mean to make an input-addressed derivation?

We'd have to remove this:

nix/src/libstore-c/nix_api_store.cc

Line 190 in 772a380

drv.checkInvariants(*store->ptr, drvPath);

The checkInvariants function failing with a mismatch outpath hash error can prevent a nix_derivation from returning. If nix_derivation_make_outputs needs a nix_derivation and you can't get one without getting the right outpath hashes, now we have a catch-22 and it'll be impossible to add derivations.

I am going to bed, but we can look into this soon, I think we should be able to untangle it perhaps simpler than you think

roberth · 2025-10-06T10:29:33Z

src/libstore-c/nix_api_store.h


+/**
+ * @brief Gets the derivation as a JSON string
+ *


doc: Which CLI command is equivalent?

I think it's similar to nix derivation show

roberth · 2025-10-06T10:30:25Z

src/libstore-c/nix_api_store.h

+    nix_c_context * context,
+    Store * store,
+    const char * json,
+    void (*callback)(void * userdata, const char * output_name, const char * path),


Probably a good idea to implement error handling here as well.

Suggested change

void (*callback)(void * userdata, const char * output_name, const char * path),

void (*callback)(nix_c_context * context, void * userdata, const char * output_name, const char * path),

I'm not sure how that helps

roberth · 2025-10-06T10:32:40Z

src/libstore-c/nix_api_store.h

+ * @brief Builds the paths, if they are a derivation then they get built.
+ *
+ * @note Path and result for the callback only exist for the lifetime of
+ * the call. Result is a string containing the build result in JSON.


doc: What kind of JSON? This should probably a link to the manual

I don't think the JSON is documented.

roberth · 2025-10-06T11:44:14Z

src/libstore-c/nix_api_store.h

+nix_err nix_store_build_paths(
+    nix_c_context * context,
+    Store * store,
+    const StorePath ** store_paths,
+    unsigned int num_store_paths,
+    void (*callback)(void * userdata, const char * path, const char * result),
+    void * userdata);


thought:
This mostly serves to support the use cases that are already well supported by nix-build/nix build.

This particular function would be subsumed by #14163. That may be ok, but if you want some of that functionality, you could pivot towards it and save the project the overhead of an extra "helper". Optional.

pkpbynum · 2025-12-16T14:44:50Z

@RossComputerGuy is this PR still active given #14555 is merged? I'm interested in nix_store_query_path_info. If not I can open a separate PR.

RossComputerGuy · 2026-02-26T17:50:06Z

To deal with the conflicts and size, I've decided to close this PR so it can be sliced up.

RossComputerGuy requested a review from edolstra as a code owner September 19, 2025 22:52

github-actions bot added the c api Nix as a C library with a stable interface label Sep 19, 2025

RossComputerGuy requested review from Ericson2314 and removed request for edolstra September 19, 2025 22:53

xokdvium reviewed Sep 19, 2025

View reviewed changes

src/libstore-c/nix_api_store.h Outdated Show resolved Hide resolved

Ericson2314 reviewed Sep 19, 2025

View reviewed changes

src/libstore-c/nix_api_store.cc Outdated Show resolved Hide resolved

Ericson2314 reviewed Sep 19, 2025

View reviewed changes

src/libstore-c/nix_api_store.cc Outdated Show resolved Hide resolved

Ericson2314 reviewed Sep 19, 2025

View reviewed changes

RossComputerGuy force-pushed the upstream-RossComputerGuy/feat/expose-drvfrompath branch from 521f2c9 to cd12d13 Compare September 19, 2025 23:17

RossComputerGuy force-pushed the upstream-RossComputerGuy/feat/expose-drvfrompath branch 4 times, most recently from df99081 to d0c6698 Compare September 22, 2025 18:25

RossComputerGuy mentioned this pull request Sep 22, 2025

feat: add derivation functions to nix_store nixops4/nixops4#122

Closed

RossComputerGuy force-pushed the upstream-RossComputerGuy/feat/expose-drvfrompath branch from d0c6698 to 4ffac98 Compare September 22, 2025 23:24

RossComputerGuy force-pushed the upstream-RossComputerGuy/feat/expose-drvfrompath branch 2 times, most recently from c1e3593 to bdf7104 Compare September 23, 2025 17:00

Ericson2314 mentioned this pull request Sep 23, 2025

Use std::variant to enforce BuildResult invariants #14060

Merged

RossComputerGuy and others added 4 commits September 30, 2025 14:51

libstore-c: add more derivation functions

4d3fd67

Log BuildResult

6651cb8

Don't use DerivedPath::toJSON()

b1b5b70

It doesn't work on unrealized paths.

libstore-c: make nix_store_query_path_info return json formatted

22a04d9

RossComputerGuy force-pushed the upstream-RossComputerGuy/feat/expose-drvfrompath branch from 517c686 to bec7506 Compare September 30, 2025 22:35

libstore-c: remove nix_derivation_output

96b3f09

RossComputerGuy force-pushed the upstream-RossComputerGuy/feat/expose-drvfrompath branch from bec7506 to 96b3f09 Compare September 30, 2025 22:42

Ericson2314 reviewed Oct 1, 2025

View reviewed changes

roberth reviewed Oct 6, 2025

View reviewed changes

Ericson2314 mentioned this pull request Nov 13, 2025

libstore-c: Add new derivation and store path functions #14555

Merged

pkpbynum mentioned this pull request Dec 10, 2025

C API: Add query_from_hash_part to Store API #14766

Merged

pkpbynum mentioned this pull request Jan 6, 2026

C API: Add query_path_info_json to Store API #14936

Open

RossComputerGuy closed this Feb 26, 2026

RossComputerGuy deleted the upstream-RossComputerGuy/feat/expose-drvfrompath branch February 26, 2026 17:50

RossComputerGuy mentioned this pull request Feb 26, 2026

Add nix_store_build_paths to libstore-c #15352

Open

	void (callback)(void userdata, const char * output_name, const char * path),
	void (callback)(nix_c_context context, void * userdata, const char * output_name, const char * path),

Uh oh!

Conversation

RossComputerGuy commented Sep 19, 2025

Motivation

Context

Uh oh!

Uh oh!

Ericson2314 commented Sep 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

RossComputerGuy commented Sep 19, 2025

Uh oh!

Ericson2314 commented Sep 19, 2025

Uh oh!

RossComputerGuy commented Sep 19, 2025

Uh oh!

Ericson2314 commented Sep 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Ericson2314 commented Sep 23, 2025

Uh oh!

Ericson2314 commented Sep 23, 2025

Uh oh!

RossComputerGuy commented Sep 23, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Ericson2314 Oct 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pkpbynum commented Dec 16, 2025

Uh oh!

RossComputerGuy commented Feb 26, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Ericson2314 commented Sep 19, 2025 •

edited

Loading

Ericson2314 commented Sep 19, 2025 •

edited

Loading

Ericson2314 Oct 1, 2025 •

edited

Loading