Dependencies in setup.py tend to mess with conda config

[jgliss]

When installing pyaerocom in developing mode into an existing conda environment via

pip install --no-deps -e .

the command line interface crashes, that is:

pya --ppiaccess

crashes as somewhere during the import of pyaerocom the "scitools-pyke>=1.1.1" is not given:

pkg_resources.DistributionNotFound: The 'scitools-pyke>=1.1.1' distribution was not found and is required by pyaerocom

I think this is due to the fact that "scitools-pyke>=1.1.1" is listed in setup.py as requirement @avaldebe, which is actually not a direct requirement for pyaerocom but I believe for iris (and pip should install that via iris). We could add this in the conda env but I resist a little and think we should re-discuss the recommended installation pathways (I, personally do not like the setup.py to be too smart, trying to solve the dependencies - via pip - as well, particularly because this has such high potential to mess with conda environments if not invoked in the correct way, with pip and --no-deps).

@jgriesfeller also had some issues in his pycharm configuration because of that, I think.

For now I will make a PR that removes pyke from setup.py so that the CLI works again.

We should discuss this on Monday I think.

[jgliss]

But I guess removing pyke from setup.py will end up in crashing CI, @avaldebe, right?

[jgriesfeller]

Regarding pycharm: During the analysis of the project it comes by the error and then it offers to install the dependency using conda (if one is using a conda env for the development).
Because the institute provides the production software for the meteorologists using conda now, we need to make sure that pyaerocom can be easily installed using that.
Personally I don't like pip and use it only if there is no other choice (e.g. missing packages in conda). I don't want that to be the default way of installing pyaerocom. Only the Python world creates pip packages, while conda is quite open these days (e.g. R, nco, cdo, etc).

To be discussed...

[avaldebe]

I think this is due to the fact that "scitools-pyke>=1.1.1" is listed in setup.py as requirement @avaldebe, which is actually not a direct requirement for pyaerocom but I believe for iris (and pip should install that via iris). We could add this in the conda env but I resist a little and think we should re-discuss the recommended installation pathways (I, personally do not like the setup.py to be too smart, trying to solve the dependencies - via pip - as well, particularly because this has such high potential to mess with conda environments if not invoked in the correct way, with pip and --no-deps).

Was the environment created as in the docs?

# new environment from scratch
conda env create -n pya -f pyaerocom_env.yml
conda activate pya

# update existing environment
conda env update -f=pyaerocom_env.yml

[jgliss]

Seems like this pyke issue may be resolved with upcoming release of iris (they want to deprecate the dependency), see:

SciTools/iris#3597

and

SciTools/iris#4072

[avaldebe]

But I guess removing pyke from setup.py will end up in crashing CI, @avaldebe, right?

Correct, it will crash the pure pip steps

[avaldebe]

Because the institute provides the production software for the meteorologists using conda now, we need to make sure that pyaerocom can be easily installed using that.

pyearocom is installed into conda in CI without problems. This is about installing in development/editable mode, which not something an external (non dev) user would do.

[jgliss]

Was the environment created as in the docs?

Yes

[avaldebe]

Seems like this pyke issue may be resolved with upcoming release of iris (they want to deprecate the dependency), see:

SciTools/iris#3597

and

SciTools/iris#4072

They dropped pyke on PR SciTools/iris#4198, which is part of their 3.1 release

A fresh pya environment should not be affected as it requires iris >=2.1.0, <3.0.0

pyaerocom/pyaerocom_env.yml

Lines 5 to 6 in 7f463cd

	dependencies:
	- iris >=2.1.0, <3.0.0

Maybe a conda update messed up the environment?

[jgliss]

Moving this to v0.13.0