command-not-found: rewrite in Rust by Mic92 · Pull Request #88517 · NixOS/nixpkgs

Mic92 · 2020-05-21T13:05:52Z

drops perl + libraries dependencies

Rust version of #74789 as @edolstra prefers Rust over C++: #74789 (comment)

Motivation for this change

Things done

domenkozar · 2020-05-21T13:46:03Z

As we are slowly going to get rid of channels as a mean of distribution, it would be good to go towards something like https://github.com/bennofs/nix-index

Meanwhile command-not-found in rust will do :)

Mic92 · 2020-05-21T14:05:23Z

Agreed. I am just not sure how nix-index can be integrated into NixOS smoothly because it requires a user to run it at least once.

domenkozar · 2020-05-21T14:09:24Z

A couple of ways I can think of:

a) encourage users to display it: Command 'foo' not found. Run nix-index to propose how to install it.

b) hook into user profiile generation and run it then, like we do for some things that already depend on whole profile

emilazy · 2020-05-21T14:23:06Z

If moving towards a nix-index-style approach, it would be nice if the index for nixpkgs could be built on Hydra and cached so people don't have to do a 5 minute batch job to get command hints (consider e.g. the live CD use-case), though I guess that would probably require some thought towards splitting up the index to avoid large redownloads.

Mic92 · 2020-05-21T14:37:47Z

Just for statistics:

~/.cache/nix-index/files is currently 27mb.
A uncompressed nixpkgs channel with is 101MB, where the programs.sqlite itself is 4MB.

matthewbauer · 2020-05-21T17:35:23Z

nix-index has its own command-not-found btw: https://github.com/bennofs/nix-index/blob/master/command-not-found.sh

If moving towards a nix-index-style approach, it would be nice if the index for nixpkgs could be built on Hydra and cached so people don't have to do a 5 minute batch job to get command hints (consider e.g. the live CD use-case), though I guess that would probably require some thought towards splitting up the index to avoid large redownloads.

It's kind of awkward because nix-index relies on a completed Hydra jobset. We could probably set up a meta one just for nix-index though. Alternatively, we could go into the channel like programs.sqlite does (https://github.com/NixOS/nixos-channel-scripts/blob/master/generate-programs-index.cc). Unfortunately the db is pretty big (~30M) so I'm not sure if that's reasonable. See #39789

Mic92 · 2020-05-21T22:19:09Z

I wonder if it is actually worth having a database or if it would not be more efficient to just have a compressed text file that is search through linear.

doronbehar · 2020-05-22T07:06:49Z

I'm up for switching to nix-index and all that, but you should note that (I'm using it for a while now) it's database lookup seems slower then the current implementation, I suspect it's due to the size of the database as it has paths for every file there is. Maybe we could modify nix-index a bit so it'll be able to generate a database of^/bin/ directories only?

Mic92 · 2020-05-22T07:51:45Z

Maybe nix-index could accept multiple sources. Than we could ship one for bin by default and the other larger one has to be enabled explicitly.

Mic92 · 2020-05-22T08:06:02Z

I opened an issue for nix-index. Please focus on code review of this rewrite only. Replacing it with nix-index is beyond the scope.

puzzlewolf

Here's a review for the Rust code :)

I don't read perl, so I didn't compare the two implementations, but it looks like the Rust code is doing the right thing 👍

Most remarks are stylistic changes, with the exception of the inline use of exit, where I'm not sure if it's actually bad :)

nixos/modules/programs/command-not-found/rust/Cargo.toml

nixos/modules/programs/command-not-found/rust/src/main.rs

nixos/modules/programs/command-not-found/default.nix

nixos/modules/programs/command-not-found/rust/src/main.rs

nixos/modules/programs/command-not-found/rust/Cargo.toml

Mic92 · 2020-05-24T16:20:58Z

Anything else?

puzzlewolf

Just two things :)

nixos/modules/programs/command-not-found/rust/src/main.rs

edolstra · 2020-05-25T08:15:43Z

We should probably just get rid of command-not-found. It only works if you use the nixpkgs/nixos channels, and channels are on their way out.

Atemu · 2020-05-25T08:36:15Z

When Flakes reach stable, yes, it should be removed but until then it remains as useful as ever and a clean rewrite is appreciated.

Though even if they came in 20.09 this tool would be useful for at least another 4-5 months because I don't think we'll get Flakes in 20.03.

Mic92 · 2020-05-25T09:10:47Z

I would suggest when nix gets flakes to disable the command-not-found module and just use nix-index when enabled.

puzzlewolf

LGTM 👍

What do you think about backporting this in 20.03?

edolstra · 2020-05-25T11:02:21Z

@puzzlewolf That kind of change definitely shouldn't be backported to a stable branch.

Mic92 · 2020-05-25T11:23:39Z

Agreed. This should be not part of 20.03.

puzzlewolf · 2020-05-25T11:45:46Z

@edolstra Well, if you put it like that :D Sorry, don't know what i was thinking.

jtojnar

It also seems to remove NIX_AUTO_INSTALL and NIX_AUTO_RUN support, which I do not see mentioned.

nixos/modules/programs/command-not-found/default.nix

nixos/modules/programs/command-not-found/rust/src/main.rs

Mic92 · 2020-05-25T13:25:33Z

@edolstra asked me to remove NIX_AUTO_INSTALL and NIX_AUTO_RUN support in #74789 (comment) It could be re-added...

bennofs · 2020-06-04T21:25:47Z

I wonder if it is actually worth having a database or if it would not be more efficient to just have a compressed text file that is search through linear.

Well, that's basically what nix-index does. It just does some simple text processing before that to improve compression rates and search speed (the same thing that locate db does: sort and encode common prefixes using a simple "repeat N chars of last prefix" approach)

- drops perl + libraries dependencies Signed-off-by: Matthias Beyer <mail@beyermatthias.de>

Mic92 · 2020-06-05T05:58:20Z

I wonder if it is actually worth having a database or if it would not be more efficient to just have a compressed text file that is search through linear.

Well, that's basically what nix-index does. It just does some simple text processing before that to improve compression rates and search speed (the same thing that locate db does: sort and encode common prefixes using a simple "repeat N chars of last prefix" approach)

Basically it is just that the full database is a bit too large.

Mic92 · 2020-06-05T05:58:38Z

All issues resolved from point of view.

nixos/modules/programs/command-not-found/default.nix

Mic92 · 2020-06-23T00:19:25Z

I just switched to nix-index given that command-not-found will die eventually.

nixos-discourse · 2022-11-02T17:00:48Z

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/why-isnt-there-an-official-built-in-way-to-find-what-package-provides-a-specific-executable/22937/3

see: NixOS/nixpkgs#88517 (comment) use `nix-locate` instead

Mic92 requested review from 2chilled and edolstra and removed request for 2chilled May 21, 2020 13:07

Mic92 mentioned this pull request May 21, 2020

command-not-found: drop perl dependency #74789

Closed

10 tasks

puzzlewolf suggested changes May 23, 2020

View reviewed changes

Mic92 force-pushed the command-not-found-rust branch from 7ebd296 to 9f153b3 Compare May 24, 2020 16:15

puzzlewolf suggested changes May 24, 2020

View reviewed changes

nixos/modules/programs/command-not-found/rust/src/main.rs Outdated Show resolved Hide resolved

puzzlewolf reviewed May 24, 2020

View reviewed changes

nixos/modules/programs/command-not-found/rust/src/main.rs Outdated Show resolved Hide resolved

Mic92 force-pushed the command-not-found-rust branch from 9f153b3 to 56aeca2 Compare May 25, 2020 09:11

puzzlewolf approved these changes May 25, 2020

View reviewed changes

jtojnar reviewed May 25, 2020

View reviewed changes

nixos/modules/programs/command-not-found/default.nix Outdated Show resolved Hide resolved

nixos/modules/programs/command-not-found/rust/src/main.rs Outdated Show resolved Hide resolved

Mic92 force-pushed the command-not-found-rust branch from 56aeca2 to 54dddbd Compare May 26, 2020 15:34

Mic92 force-pushed the command-not-found-rust branch from 54dddbd to ef51e5c Compare June 5, 2020 05:55

command-not-found: rewrite in Rust

edaece2

- drops perl + libraries dependencies Signed-off-by: Matthias Beyer <mail@beyermatthias.de>

Mic92 force-pushed the command-not-found-rust branch from ef51e5c to edaece2 Compare June 5, 2020 05:57

edolstra reviewed Jun 5, 2020

View reviewed changes

nixos/modules/programs/command-not-found/default.nix Show resolved Hide resolved

Mic92 closed this Jun 23, 2020

Mic92 deleted the command-not-found-rust branch June 23, 2020 00:19

Mic92 mentioned this pull request Jun 23, 2020

nixos-generate-config: add Rust stub with tests #91353

Closed

r-k-b added a commit to r-k-b/nixos-config that referenced this pull request Jul 23, 2025

feat: provide a replacement for command-not-found

5c7d7a1

see: NixOS/nixpkgs#88517 (comment) use `nix-locate` instead

Uh oh!

Conversation

Mic92 commented May 21, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation for this change

Things done

Uh oh!

domenkozar commented May 21, 2020

Uh oh!

Mic92 commented May 21, 2020

Uh oh!

domenkozar commented May 21, 2020

Uh oh!

emilazy commented May 21, 2020

Uh oh!

Mic92 commented May 21, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

matthewbauer commented May 21, 2020

Uh oh!

Mic92 commented May 21, 2020

Uh oh!

doronbehar commented May 22, 2020

Uh oh!

Mic92 commented May 22, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Mic92 commented May 22, 2020

Uh oh!

puzzlewolf left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Mic92 commented May 24, 2020

Uh oh!

puzzlewolf left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

edolstra commented May 25, 2020

Uh oh!

Atemu commented May 25, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Mic92 commented May 25, 2020

Uh oh!

puzzlewolf left a comment

Choose a reason for hiding this comment

Uh oh!

edolstra commented May 25, 2020

Uh oh!

Mic92 commented May 25, 2020

Uh oh!

puzzlewolf commented May 25, 2020

Uh oh!

jtojnar left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Mic92 commented May 25, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bennofs commented Jun 4, 2020

Uh oh!

Mic92 commented Jun 5, 2020

Uh oh!

Mic92 commented Jun 5, 2020

Uh oh!

Uh oh!

Mic92 commented May 21, 2020 •

edited

Loading

Mic92 commented May 21, 2020 •

edited

Loading

Mic92 commented May 22, 2020 •

edited

Loading

puzzlewolf left a comment •

edited

Loading

Atemu commented May 25, 2020 •

edited

Loading

Mic92 commented May 25, 2020 •

edited

Loading