Homebrew 2.5.4 can't create virtualenv with python3.7

[mndeveci]

Hi,

We are trying to build our bottle by creating a virtual environment in python3.7 as it can be seen here; https://github.com/aws/homebrew-tap/blob/7c0e71838c690ed009117d3bf6eff46bba3fd164/Formula/aws-sam-cli.rb#L27

Starting from today, that line is failing with creation of the virtual environment, the exception is;

==> python3.7 -s /tmp/aws-sam-cli--homebrew-virtualenv-20201007-10553-26ty3f/target/bin/virtualenv -p python3.7 /home/linuxbrew/.linuxbrew/Cellar/aws-sam-cli/1.5.0/libexec
Last 15 lines from /home/appveyor/.cache/Homebrew/Logs/aws-sam-cli/06.python3.7:
  File "/tmp/aws-sam-cli--homebrew-virtualenv-20201007-10553-26ty3f/target/bin/virtualenv", line 6, in <module>
    from pkg_resources import load_entry_point
  File "/home/linuxbrew/.linuxbrew/opt/[email protected]/lib/python3.7/site-packages/pkg_resources/__init__.py", line 3254, in <module>
    @_call_aside
  File "/home/linuxbrew/.linuxbrew/opt/[email protected]/lib/python3.7/site-packages/pkg_resources/__init__.py", line 3238, in _call_aside
    f(*args, **kwargs)
  File "/home/linuxbrew/.linuxbrew/opt/[email protected]/lib/python3.7/site-packages/pkg_resources/__init__.py", line 3267, in _initialize_master_working_set
    working_set = WorkingSet._build_master()
  File "/home/linuxbrew/.linuxbrew/opt/[email protected]/lib/python3.7/site-packages/pkg_resources/__init__.py", line 584, in _build_master
    ws.require(__requires__)
  File "/home/linuxbrew/.linuxbrew/opt/[email protected]/lib/python3.7/site-packages/pkg_resources/__init__.py", line 901, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/home/linuxbrew/.linuxbrew/opt/[email protected]/lib/python3.7/site-packages/pkg_resources/__init__.py", line 787, in resolve
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'importlib-metadata<3,>=0.12' distribution was not found and is required by virtualenv
Do not report this issue to Homebrew/brew or Homebrew/core!
Traceback (most recent call last):
	32: from /home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/build.rb:229:in `<main>'
	31: from /home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/build.rb:140:in `install'
	30: from /home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/utils.rb:496:in `with_env'
	29: from /home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/build.rb:145:in `block in install'
	28: from /home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/formula.rb:1182:in `brew'
	27: from /home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/formula.rb:2156:in `stage'
	26: from /home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/vendor/portable-ruby/2.6.3_2/lib/ruby/2.6.0/forwardable.rb:230:in `stage'
	25: from /home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/resource.rb:86:in `stage'
	24: from /home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/resource.rb:111:in `unpack'
	23: from /home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/resource.rb:201:in `mktemp'
	22: from /home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/mktemp.rb:57:in `run'
	21: from /home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/mktemp.rb:57:in `chdir'
	20: from /home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/mktemp.rb:57:in `block in run'
	19: from /home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/resource.rb:116:in `block in unpack'
	18: from /home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/formula.rb:2176:in `block in stage'
	17: from /home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/utils.rb:496:in `with_env'
	16: from /home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/formula.rb:2177:in `block (2 levels) in stage'
	15: from /home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/formula.rb:1189:in `block in brew'
	14: from /home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/build.rb:177:in `block (2 levels) in install'
	13: from /home/appveyor/projects/homebrew-tap/Formula/aws-sam-cli.rb:27:in `install'
	12: from /home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/language/python.rb:157:in `virtualenv_create'
	11: from /home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/language/python.rb:235:in `create'
	10: from /home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/resource.rb:86:in `stage'
	 9: from /home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/resource.rb:111:in `unpack'
	 8: from /home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/resource.rb:201:in `mktemp'
	 7: from /home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/mktemp.rb:57:in `run'
	 6: from /home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/mktemp.rb:57:in `chdir'
	 5: from /home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/mktemp.rb:57:in `block in run'
	 4: from /home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/resource.rb:116:in `block in unpack'
	 3: from /home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/language/python.rb:249:in `block in create'
	 2: from /home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/formula.rb:1970:in `system'
	 1: from /home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/formula.rb:1970:in `open'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/formula.rb:2032:in `block in system': Failed executing: python3.7 -s /tmp/aws-sam-cli--homebrew-virtualenv-20201007-10553-26ty3f/target/bin/virtualenv -p python3.7 /home/linuxbrew/.linuxbrew/Cellar/aws-sam-cli/1.5.0/libexec (BuildError)
	5: from /home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/brew.rb:150:in `<main>'
	4: from /home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/brew.rb:162:in `rescue in <main>'
	3: from /home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/exceptions.rb:447:in `dump'
	2: from /home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/exceptions.rb:393:in `issues'
	1: from /home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/exceptions.rb:397:in `fetch_issues'
/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/utils/github.rb:322:in `issues_for_formula': undefined method `full_name' for nil:NilClass (NoMethodError)
❌ Failed to build Formula/aws-sam-cli.rb ❌
aws/tap
homebrew/core
Command exited with code 2
Build failed

We were able to build it on Oct 1st with brew version 2.5.2.

Could this be related to latest changes, especially the ones related to python3.9 and virtual environment upgrades?

Thanks

[Rylan12]

Try changing the virtualenv_create line to

venv = virtualenv_create(libexec, Formula["[email protected]"].opt_bin/"python3")

That worked for me locally.

I'll look into it more tomorrow to figure out why this only started happening with the latest version.

[iMichka]

All formulae using virtual envs are broken on Linux now. I just opened a patch for that, it should be fixed soon: Homebrew/brew#8880

[mndeveci]

We tried to build image with python3.8, it passes creating virtualenv on linux and mac. But on linux the created bottle has import issues. For instance our cli fails saying it can't find the module there;

==> Installing aws-sam-cli from aws/tap
🍺  /home/linuxbrew/.linuxbrew/Cellar/aws-sam-cli/1.4.0: 4,122 files, 90.7MB
++ sam --version
++ grep -Eo '[0-9]+\.[0-9]+\.[0-9a-zA-Z]+'
Traceback (most recent call last):
  File "/home/linuxbrew/.linuxbrew/bin/sam", line 5, in <module>
    from samcli.cli.main import cli
ModuleNotFoundError: No module named 'samcli'

[mndeveci]

@iMichka do we know when this fix will be released? We are stuck because of this issue now, and our linux bottle's are not generated correctly.

[iMichka]

You could use export HOMEBREW_DEVELOPER=1 and brew update to get the latest bleeding edge with the patch. We need to cut a new release, I will ask the people that know how to do that if we can make a new release soon.

Do not forget to reset HOMEBREW_DEVELOPER once we are done, as you might not need that if you want stable builds.

[Rylan12]

@mndeveci 2.5.5 has now been released so a simple brew update without setting the HOMEBREW_DEVELOPER should work now.