pyenv does not work with brew pyenv-sync

[PheelaV]

brew doctor output

Your system is ready to brew.

Verification

• My "brew doctor output" above says Your system is ready to brew. and am still able to reproduce my issue.
• I ran brew update twice and am still able to reproduce my issue.
• This issue's title and/or description do not reference a single formula e.g. brew install wget. If they do, open an issue at https://github.com/Homebrew/homebrew-core/issues/new/choose instead.

brew config output

HOMEBREW_VERSION: 4.3.7
ORIGIN: https://github.com/Homebrew/brew
HEAD: 43eaeca50fe3b6c755f3fc9bc6f43669e0db1039
Last commit: 2 days ago
Core tap JSON: 25 Jun 23:34 UTC
Core cask tap JSON: 25 Jun 23:34 UTC
HOMEBREW_PREFIX: /opt/homebrew
HOMEBREW_CASK_OPTS: []
HOMEBREW_EDITOR: nvim
HOMEBREW_MAKE_JOBS: 12
Homebrew Ruby: 3.3.3 => /opt/homebrew/Library/Homebrew/vendor/portable-ruby/3.3.3/bin/ruby
CPU: dodeca-core 64-bit arm_lobos
Clang: 15.0.0 build 1500
Git: 2.39.3 => /Library/Developer/CommandLineTools/usr/bin/git
Curl: 8.6.0 => /usr/bin/curl
macOS: 14.5-arm64
CLT: 15.3.0.0.1.1708646388
Xcode: N/A
Rosetta 2: false

What were you trying to do (and why)?

brew pyenv-sync

I am trying to use brew-installed python distributions using pyenv.

What happened (include all command output)?

but when I execute python or pyenv which python I get

➜  .pyenv python
zsh: command not found: python
➜  .pyenv pyenv which python
pyenv: python: command not found

I see the symlinks:

ls ~/.pyenv/versions
3.10.0  3.10.11 3.10.14 3.10.4  3.10.7  3.11.0  3.11.3  3.11.6  3.11.9  3.12.2
3.10.1  3.10.12 3.10.2  3.10.5  3.10.8  3.11.1  3.11.4  3.11.7  3.12.0  3.12.3

Then I see

pyenv versions
  system
  3.10.0 --> /opt/homebrew/Cellar/[email protected]/3.10.14
  3.10.1 --> /opt/homebrew/Cellar/[email protected]/3.10.14
  3.10.2 --> /opt/homebrew/Cellar/[email protected]/3.10.14
  3.10.3 --> /opt/homebrew/Cellar/[email protected]/3.10.14
  3.10.4 --> /opt/homebrew/Cellar/[email protected]/3.10.14
  3.10.5 --> /opt/homebrew/Cellar/[email protected]/3.10.14
  3.10.6 --> /opt/homebrew/Cellar/[email protected]/3.10.14
  3.10.7 --> /opt/homebrew/Cellar/[email protected]/3.10.14
  3.10.8 --> /opt/homebrew/Cellar/[email protected]/3.10.14
  3.10.9 --> /opt/homebrew/Cellar/[email protected]/3.10.14
  3.10.10 --> /opt/homebrew/Cellar/[email protected]/3.10.14
  3.10.11 --> /opt/homebrew/Cellar/[email protected]/3.10.14
  3.10.12 --> /opt/homebrew/Cellar/[email protected]/3.10.14
  3.10.13 --> /opt/homebrew/Cellar/[email protected]/3.10.14
* 3.10.14 --> /opt/homebrew/Cellar/[email protected]/3.10.14 (set by /Users/filip/.pyenv/version)
  3.11.0 --> /opt/homebrew/Cellar/[email protected]/3.11.9
  3.11.1 --> /opt/homebrew/Cellar/[email protected]/3.11.9
  3.11.2 --> /opt/homebrew/Cellar/[email protected]/3.11.9
  3.11.3 --> /opt/homebrew/Cellar/[email protected]/3.11.9
  3.11.4 --> /opt/homebrew/Cellar/[email protected]/3.11.9
  3.11.5 --> /opt/homebrew/Cellar/[email protected]/3.11.9
  3.11.6 --> /opt/homebrew/Cellar/[email protected]/3.11.9
  3.11.7 --> /opt/homebrew/Cellar/[email protected]/3.11.9
  3.11.8 --> /opt/homebrew/Cellar/[email protected]/3.11.9
  3.11.9 --> /opt/homebrew/Cellar/[email protected]/3.11.9
  3.12.0 --> /opt/homebrew/Cellar/[email protected]/3.12.4
  3.12.1 --> /opt/homebrew/Cellar/[email protected]/3.12.4
  3.12.2 --> /opt/homebrew/Cellar/[email protected]/3.12.4
  3.12.3 --> /opt/homebrew/Cellar/[email protected]/3.12.4
  3.12.4 --> /opt/homebrew/Cellar/[email protected]/3.12.4

This is happening both when using a .python-version file via pyenv local 3.10 as well as for the global option pyenv global 3.10

As you can see, the correct python version is selected as denoted by the asterisk.

What did you expect to happen?

I expected that pyenv would select and appropriate pyhton version (brew-installed) and carry on.

This is relevant? /orgs/Homebrew/discussions/4664

Step-by-step reproduction instructions (by running brew commands)

brew install [email protected]

brew install pyenv

pyenv --init makes me put this in path, I have updated it with the shims and put it into my ~/.zshrc

>>> pyenv init >>>

export PYENV_ROOT="$HOME/.pyenv"
[[ -d $PYENV_ROOT/bin ]] && export PATH="$PYENV_ROOT/bin:$PATH"
[[ -d $PYENV_ROOT/shims ]] && export PATH="$PYENV_ROOT/shims:$PATH"
eval "$(pyenv init -)"

<<< pyenv init <<<<

source ~/.zshrc

pyenv which python; python --version

[rrotter]

I'm not sure how this ever worked unless the layout of the python formulae or the behavior of pyenv has changed since the pyenv-sync command was added. No python formula links python in the bin dir, only the version with the version number appended (python3, python3.12, etc), and only the newest version links python3, the rest only link python3.xx:

~ % ls $(brew --prefix)/Cellar/python*/*/bin/python* | grep -v config
/opt/homebrew/Cellar/[email protected]/3.10.14/bin/python3.10
/opt/homebrew/Cellar/[email protected]/3.11.9/bin/python3.11
/opt/homebrew/Cellar/[email protected]/3.12.4/bin/python3
/opt/homebrew/Cellar/[email protected]/3.12.4/bin/python3.12
/opt/homebrew/Cellar/[email protected]/3.9.19/bin/python3.9

The other links are present, but only under libexec/bin, which I assume pyenv doesn't read. If this is to be fixed I think it's either asking pyenv to use libexec/bin (probably not realistic) or building an additional tree of symlinks with a complete bin dir (presumably in the python formulae, but possibly in the pyenv-sync command) for pyenv-sync to link to. (I suppose a third option is installing a hacked version of pyenv, but that seems like the most problematic solution)

[DConcord]

I'm seeing the same behavior (no "python" command available, only "python3"). But I'm also seeing the that "pyenv local", "pyenv shell", and even pipenv doesn't work with any version of python linked/synced from brew; only versions installed by pyenv

[rrotter]

Yep. pyenv only loads what's in .pyenv/versions/$PYENV_VERSION/bin/. In the case of the two latest homebrew pythons that is:

~ % ls .pyenv/versions/3.12.4/bin
2to3			pydoc3.12
2to3-3.12		python3
idle3			python3-config
idle3.12		python3.12
pip3			python3.12-config
pip3.12			wheel3
pydoc3			wheel3.12
~ % ls .pyenv/versions/3.11.9/bin/
2to3-3.11		python3.11
idle3.11		python3.11-config
pip3.11			wheel3.11
pydoc3.11

Note that there is no overlap in these lists (i.e. no python3 under 3.11.9). That's a feature of the python formulae so they don't conflict with one another, but it's also why they don't work with pyenv.

[reitermarkus]

That's a feature of the python formulae so they don't conflict with one another, but it's also why they don't work with pyenv.

Exactly that. Note that Homebrew is not a version manager. The only reason versioned formulae exist is because other formulae don't work with the newest version.

[PheelaV]

Indeed, something may have changed. I have noticed ~/.pyenv/bin does not get created (unless $pyenv install). As I was not exactly sure how pyenv is handling things behind the scenes, I decided to open this issue.

For the time being I have just resolved to use pyenv with redundancy.

Inspecting again, I can see

~ % ls ~/.pyenv/versions
3.10.14 3.11.9  3.12.4

~ % ls ~/.pyenv/versions/3.12.4
bin     include lib     share

~ % ls ~/.pyenv/versions/3.12.4/bin
2to3              idle3.12          pydoc             python-config     python3.12-config
2to3-3.12         pip               pydoc3            python3           python3.12-gdb.py
idle              pip3              pydoc3.12         python3-config
idle3             pip3.12           python            python3.12

@rrotter Now I understand your comment

I'm not sure how this ever worked unless...

~ % /opt/homebrew/Cellar/[email protected]/3.12.4
Frameworks            LICENSE               bin                   sbom.spdx.json
IDLE 3.app            Python Launcher 3.app lib                   share
INSTALL_RECEIPT.json  README.rst            libexec

~ % ls /opt/homebrew/Cellar/[email protected]/3.12.4/bin
2to3              idle3.12          pydoc3            python3-config    wheel3
2to3-3.12         pip3              pydoc3.12         python3.12        wheel3.12
idle3             pip3.12           python3           python3.12-config

I would like to start playing with authoring/modifying homebrew formulaes, but do not have the time now (until September). In [email protected] formulae Install unversioned symlinks in libexec/bin. Perhaps the best solution as you say is to re-create the pyenv installation structure using the contents /libexec/bin.

• This will need to be maintained (do not know how often this changes, perhaps more so on pyenv's end than python's)
• What about pyenv uninstall? On my search for similar issues I have stumbled upon "pyenv removes/modifies python installation from homebrew cellar" - can't find it now, possibly pyenv or brew repository. In another words, how does brew protect its installations from being mangled with?

Possibly off topic, but providing context:
My own usecase is to use brew-installed python versions to seed my poetry-based projects for development, which does require for the base version installation to exist on a system to create its venv.

I have seen people both

1. use brew-installed python versions with pyenv (via manual/script linking)
2. use pyenv-installed python versions with homebrew

I was leaning towards 1. as I want ideally one place to think about installing/uninstalling software, but I was seeking automation and brew pyenv-sync was promising.

@reitermarkus I see your point, if you are fine with latest minor releases use homebrew, if you need something more specific go elsewhere (e.g. pyenv/virtualenv/conda/docker).

Relevant #2138
Original PR 15507

[MikeMcQuaid]

Perhaps the best solution as you say is to re-create the pyenv installation structure using the contents /libexec/bin.

This seems like a good idea and the best implementation here.