Skip to content

python312Packages.tree-sitter_0_21: remove from propagated dependencies#332747

Merged
natsukium merged 8 commits intoNixOS:masterfrom
natsukium:dont-propagate-versioned-tree-sitter
Aug 7, 2024
Merged

python312Packages.tree-sitter_0_21: remove from propagated dependencies#332747
natsukium merged 8 commits intoNixOS:masterfrom
natsukium:dont-propagate-versioned-tree-sitter

Conversation

@natsukium
Copy link
Member

Description of changes

Python packages must not propagate such versioned packages due to PYTHONPATH conflicts.

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

Add a 👍 reaction to pull requests you find important.

@github-actions github-actions bot added the 6.topic: python Python is a high-level, general-purpose programming language. label Aug 6, 2024
@ofborg ofborg bot added 8.has: clean-up This PR removes packages or removes other cruft 8.has: package (new) This PR adds a new package labels Aug 6, 2024
@ofborg ofborg bot requested review from doronbehar, fabaff and joelkoen August 6, 2024 17:46
@ofborg ofborg bot added 10.rebuild-darwin: 11-100 This PR causes between 11 and 100 packages to rebuild on Darwin. 10.rebuild-linux: 11-100 This PR causes between 11 and 100 packages to rebuild on Linux. labels Aug 6, 2024
@natsukium natsukium force-pushed the dont-propagate-versioned-tree-sitter branch from 3efc201 to 6351301 Compare August 7, 2024 05:37
@natsukium
Copy link
Member Author

natsukium commented Aug 7, 2024

Result of nixpkgs-review pr 332747 at a930fa80 run on x86_64-linux 1

4 packages marked as broken and skipped:
  • python311Packages.grep-ast
  • python311Packages.tree-sitter-languages
  • python312Packages.grep-ast
  • python312Packages.tree-sitter-languages
2 packages failed to build:
  • browsr
  • textual-paint
27 packages built successfully:
  • autotools-language-server
  • dooit
  • frogmouth
  • memray
  • oterm
  • python311Packages.graphrag
  • python311Packages.lsp-tree-sitter
  • python311Packages.manifestoo
  • python311Packages.pytest-textual-snapshot
  • python311Packages.textual
  • python311Packages.textual-dev
  • python311Packages.textual-universal-directorytree
  • python311Packages.tree-sitter_0_21
  • python312Packages.graphrag
  • python312Packages.lsp-tree-sitter
  • python312Packages.manifestoo
  • python312Packages.pytest-textual-snapshot
  • python312Packages.textual
  • python312Packages.textual-dev
  • python312Packages.textual-universal-directorytree
  • python312Packages.tree-sitter_0_21
  • rich-cli
  • smassh
  • tftui
  • toolong
  • upiano
  • wsrepl

Result of nixpkgs-review pr 332747 at a930fa80 run on aarch64-darwin 1

4 packages marked as broken and skipped:
  • python311Packages.grep-ast
  • python311Packages.tree-sitter-languages
  • python312Packages.grep-ast
  • python312Packages.tree-sitter-languages
15 packages failed to build:
  • browsr
  • frogmouth
  • oterm
  • python312Packages.graphrag
  • python312Packages.manifestoo
  • python312Packages.pytest-textual-snapshot
  • python312Packages.textual
  • python312Packages.textual-dev
  • python312Packages.textual-universal-directorytree
  • python312Packages.tree-sitter_0_21
  • rich-cli
  • textual-paint
  • tftui
  • upiano
  • wsrepl
13 packages built successfully:
  • autotools-language-server
  • dooit
  • python311Packages.graphrag
  • python311Packages.lsp-tree-sitter
  • python311Packages.manifestoo
  • python311Packages.pytest-textual-snapshot
  • python311Packages.textual
  • python311Packages.textual-dev
  • python311Packages.textual-universal-directorytree
  • python311Packages.tree-sitter_0_21
  • python312Packages.lsp-tree-sitter
  • smassh
  • toolong

Copy link
Contributor

@doronbehar doronbehar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes look good overall, I wonder how come so many packages got dependent on tree-sitter_0_21... I'm only a bit confused about the changes to python31{1,2}Packages.textual:

The optional-dependencies.syntax attribute is confusing to me - why not change there the tree-sitter dependency to tree-sitter_0_21? If the last commit will be included eventually, the optional-dependencies.syntax attribute will almost never be evaluated, but it is still confusing, especially since this PR is meant to avoid mixing different versions of the same python package in the dependency tree of packages.

Another thing that bothers me, is the fact that without the last commit, python312Packages.textual builds fine on Hydra. However, python311Packages.textual doesn't evaluate because
tree-sitter-languages is marked as broken on Darwin. Although the lib.optionals condition should not add it to optional-dependencies.syntax. Do you know how to explain that please @natsukium ?

Also, if the goal of the last commit is to fix the python311Packages.textual, the commit message should indicate that.

@doronbehar
Copy link
Contributor

Changes look good overall, I wonder how come so many packages got dependent on tree-sitter_0_21...

I take these words back... I iterated once more the commits and I do understand why these packages dependent on tree-sitter_0_21 - the comments explained this.

I would approve a PR that would change only python31{1,2}Packages.textual, because that is the only package that has mixed tree-sitter versions in it's dependencies.

@ofborg ofborg bot requested a review from doronbehar August 7, 2024 06:10
@natsukium
Copy link
Member Author

Did you notice that grep-ast depends on tree-sitter_0_21 through tree-sitter-languages in #331173 ?
Also, note that tree-sitter_0_21 will be propagated when a certain package adds textual.optional-dependencies.syntax to its dependencies.

We can't control the downstream packages or the user environment.
So the current tree-sitter-languages and lsp-tree-sitter are unacceptable.

@natsukium natsukium force-pushed the dont-propagate-versioned-tree-sitter branch from a930fa8 to 4ea27c0 Compare August 7, 2024 06:48
@natsukium
Copy link
Member Author

I found that textual does not require tree-sitter-languages for testing and removed it from nativeCheckInputs.

@doronbehar
Copy link
Contributor

Also, note that tree-sitter_0_21 will be propagated when a certain package adds textual.optional-dependencies.syntax to its dependencies.

Yea indeed it is a weird thing to do.

Did you notice that grep-ast depends on tree-sitter_0_21 through tree-sitter-languages in #331173 ?

No.. (perhaps I should have added my self to the maintainers.)

So the current tree-sitter-languages and lsp-tree-sitter are unacceptable.

OK, that is convincing.

I found that textual does not require tree-sitter-languages for testing and removed it from nativeCheckInputs.

That's good. I took the liberty to rebase the PR onto latest master, so it is a bit more clear what is the state of things before and after and changed a bit the commit order, and the textual commit's message.

@doronbehar doronbehar force-pushed the dont-propagate-versioned-tree-sitter branch from 4ea27c0 to 1e61017 Compare August 7, 2024 08:45
@natsukium
Copy link
Member Author

Thanks for understanding and adding context.

@natsukium natsukium merged commit b9f2737 into NixOS:master Aug 7, 2024
@natsukium natsukium deleted the dont-propagate-versioned-tree-sitter branch August 7, 2024 11:04
tree-sitter = callPackage ../development/python-modules/tree-sitter { };

tree-sitter0_21 = callPackage ../development/python-modules/tree-sitter0_21 { };
tree-sitter_0_21 = callPackage ../development/python-modules/tree-sitter/0_21.nix { };
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would have been ideal to add an entry in python-aliases for the rename.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Certainly, but it's an internal package and I didn't want to pollute the namespace too much.

Copy link
Contributor

@doronbehar doronbehar Aug 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would have been ideal to add an entry in python-aliases for the rename.

Certainly, but it's an internal package and I didn't want to pollute the namespace too much.

Also, that attribute was there for a ~month or less, and it is an outdated version, so I barely think anyone had started using it externally.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

6.topic: python Python is a high-level, general-purpose programming language. 8.has: clean-up This PR removes packages or removes other cruft 8.has: package (new) This PR adds a new package 10.rebuild-darwin: 11-100 This PR causes between 11 and 100 packages to rebuild on Darwin. 10.rebuild-linux: 11-100 This PR causes between 11 and 100 packages to rebuild on Linux.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants