Generate legacy metadata in temporary directory

[sbidoul]

Progresses #7555 by getting rid of pip-egg-info as mentioned in #7882 (comment).

Pip generates legacy metadata by calling setup.py egg_info. For editables, it does that in the source directory, else it does it in a pip-egg-info subdirectory of source dir.

To avoid polluting source dir with this pip-egg-info if and when we will build in place, use a temporary directory instead.

Question to reviewers: can one think if there was a good reason to generate that pip-egg-info inside source dir?

If this PR sounds reasonable, one can further think about always generating legacy metadata in a temporary directory, even in the editable case. Indeed it looks like setup.py develop always runs egg_info even if the directory is already present. This would vastly simplify this area of the code. [update] this simplification in now part of this PR

[pradyunsg]

Editable installs need the egg-info to be in the location that it's in for working properly IIRC.

Other than that, this all sounds and looks good to me in principle.

[sbidoul]

Editable installs need the egg-info to be in the location that it's in for working properly IIRC.

Sure, but does pip need to have generate_metadata() put it there since setup.py develop that is invoked afterwards will generate it anyway.

[pradyunsg]

I'm pretty sure that setuptools doesn't see that, so it should be fine to put that in a temporary directory.

[sbidoul]

Ok, I added a commit with the full simplification of metadata_legacy.py, using a temporary directory in all cases.

[xavfernandez]

Suggested change

	def _find_egg_info(source_directory):
	def _find_egg_info(directory):

[uranusjr]

The function body and docstring also need to match this change :)

[xavfernandez]

Nice simplification 👍

[sbidoul]

Thanks for the review @xavfernandez

[pradyunsg]

These simplifications make me feel happy about factoring this code out of the depths of InstallRequirement + friends. :)

[sbidoul]

@RonnyPfannschmidt this is the PR that removed pip-egg-info. I see there is a hack.py in setuptools_scm that relies on that directory so... yeah. That particular hack is going to be ineffective now. I'm not sure in which case it was needed though.

[RonnyPfannschmidt]

When neither the scm nor a pkg-info file are part of the external source tree, setuptools_scm has no way to access the version information

[sbidoul]

@RonnyPfannschmidt yep, I know the case with setup.py in subdirectory, but in my experience it never worked with pip install/wheel . and setuptools_scm. Would you have a reproducer? Which pip version are you using to reproduce (must be master, because <20.1 does not have this PR and 20.1 builds in tree so should not be affected)?

[RonnyPfannschmidt]

I don't have a reproducer, i observed this breakage before your pr, and AFAIK out of tree builds are coming back due to issue

[sbidoul]

Ok, then if it's unrelated to this PR, the problem you mentioned on discuss is not new, and is part of the known issues with out of tree builds, as summarized in #7555.