recommended setup vscode+python for project development

[mikofski]

What is the recommended setup for project development?

I'm a maintainer of the open source pvlib-python project, and I've been using vscode+python for about 2 years. We at pvlib-python have been recommending contributors to install pvlib as an editable library. Unfortunately since some time in fall of 2020, vscode+python no longer runs tests for projects installed this way. AFAICT this is because tests are now set to run "in isolation" which I believe means that the current folder has been removed from the Python path, and therefore the test modules can't import the very functions they are intended to test.

For example:

1. clone pvlib-python into a folder called "pvlib-python"
2. create either a conda environment or Python virtual environment with the requirements, and activate it
3. use pip install -e . to install pvlib-python into the environment.
4. start vscode and select the python interpreter from the environment

But this setup has the following outcomes:

1. ✔️ tests are discovered
2. ✔️ tests work in debug
3. ✔️ tests work from terminal
4. ❌ tests fail when pressing the green play button with an import error, module pvlib not found.

This is why I'm asking what the recommended dev proj setup is, because our existing contributor recommendation doesn't work with vscode+python anymore.

Thanks!

related

#15398
#14570

[brettcannon]

Did you try turning off isolation via "python.useIsolation": false? Otherwise why are you specifically suggesting using an editable install when your source code is already at the top of the repo under a pvlib directory? And how do you have your tests set up in the extension?

[brettcannon]

I would change "python.testing.pytestArgs": ["."] and see if that does it. Also, do you have a __init__.py file in your tests/ directory?

[mikofski]

Thanks for these suggestions.

change "python.testing.pytestArgs": ["."] and see if that does it.

No change: tests discovered ✔️, runs in debug ✔️, but run tests fails ❌

python c:\Users\mikm\.vscode\extensions\ms-python.python-2021.2.582707922\pythonFiles\testing_tools\run_adapter.py discover pytest -- --rootdir c:\Users\mikm\Projects\pvlib-python -s --cache-clear .
ImportError while loading conftest 'c:\Users\mikm\Projects\pvlib-python\pvlib\tests\conftest.py'.
pvlib\tests\conftest.py:10: in <module>
    import pvlib
E   ModuleNotFoundError: No module named 'pvlib'
Error: TypeError: Cannot read property 'testsuites' of null

do you have a __init__.py file in your tests/ directory

No, we do not. Adding __init__.py to the tests/ folder causes test discovery to fail ❌ with a very long error message:

python c:\Users\mikm\.vscode\extensions\ms-python.python-2021.2.582707922\pythonFiles\testing_tools\run_adapter.py discover pytest -- --rootdir c:\Users\mikm\Projects\pvlib-python -s --cache-clear .
Test Discovery failed: 
Error: ============================= test session starts =============================
platform win32 -- Python 3.6.8, pytest-6.2.2, py-1.10.0, pluggy-0.13.1
rootdir: c:\Users\mikm\Projects\pvlib-python, configfile: setup.cfg
plugins: cov-2.11.1, mock-3.5.1, remotedata-0.3.2, rerunfailures-9.1.1, timeout-1.4.2
collected 142 items / 30 errors / 112 selected

<Package tests>
  <Module test_infinite_sheds.py>
    <Function test_solar_projection_tangent>
    <Function test_solar_projection>
    <Function test_unshaded_ground_fraction>
    <Function test_gcr_prime>
    <Function test_ground_sky_angles>
    <Function test_ground_sky_angles_prev>
    <Function test_ground_sky_angles_next>
    <Function test_diffuse_fraction>
    <Function test_vf_ground_sky>
    <Function test_poa_ground_sky>
    <Function test_shade_line>
    <Function test_sky_angles>
    <Function test_sky_angle_tangent>
    <Function test_sky_angle_0_tangent>
  <Module test_scaling.py>
    <Function test_latlon_to_xy_zero>
    <Function test_latlon_to_xy_single>
    <Function test_latlon_to_xy_array>
    <Function test_latlon_to_xy_list>
    <Function test_compute_wavelet_series>
    <Function test_compute_wavelet_series_numindex>
    <Function test_compute_wavelet_array>
    <Function test_compute_wavelet_array_invalid>
    <Function test_wvm_series>
    <Function test_wvm_array>
    <Function test_wvm_series_xyaslist>
    <Function test_wvm_invalid>
  <Module test_shading.py>
    <Function test_masking_angle_series>
    <Function test_masking_angle_scalar>
    <Function test_masking_angle_passias_series>
    <Function test_masking_angle_passias_scalar>
    <Function test_sky_diffuse_passias_series>
    <Function test_sky_diffuse_passias_scalar>
  <Module test_singlediode.py>
    <Function test_method_spr_e20_327[brentq]>
    <Function test_method_spr_e20_327[newton]>
    <Function test_newton_fs_495[brentq]>
    <Function test_newton_fs_495[newton]>
    <Function test_pvsyst_recombination_loss[newton-1000-25-expected0-tol0]>
    <Function test_pvsyst_recombination_loss[newton-888-55-expected1-tol1]>
    <Function test_pvsyst_recombination_loss[brentq-1000-25-expected0-tol0]>
    <Function test_pvsyst_recombination_loss[brentq-888-55-expected1-tol1]>
    <Function test_pvsyst_breakdown[newton-brk_params0-recomb_params0-1000-25-expected0-tol0]>
    <Function test_pvsyst_breakdown[newton-brk_params1-recomb_params1-888-55-expected1-tol1]>
    <Function test_pvsyst_breakdown[brentq-brk_params0-recomb_params0-1000-25-expected0-tol0]>
    <Function test_pvsyst_breakdown[brentq-brk_params1-recomb_params1-888-55-expected1-tol1]>
  <Module test_spa.py>
    <UnitTestCase NumpySpaTest>
      <TestCaseFunction test_aberration_correction>
      <TestCaseFunction test_apparent_sidereal_time>
      <TestCaseFunction test_apparent_sun_longitude>
      <TestCaseFunction test_atmospheric_refraction_correction>
      <TestCaseFunction test_calculate_deltat>
      <TestCaseFunction test_earthsun_distance>
      <TestCaseFunction test_equation_of_time>
      <TestCaseFunction test_equatorial_horizontal_parallax>
      <TestCaseFunction test_geocentric_latitude>
      <TestCaseFunction test_geocentric_longitude>
      <TestCaseFunction test_geocentric_sun_declination>
      <TestCaseFunction test_geocentric_sun_right_ascension>
      <TestCaseFunction test_heliocentric_latitude>
      <TestCaseFunction test_heliocentric_longitude>
      <TestCaseFunction test_heliocentric_radius_vector>
      <TestCaseFunction test_julian_century>
      <TestCaseFunction test_julian_day>
      <TestCaseFunction test_julian_day_dt>
      <TestCaseFunction test_julian_ephemeris_century>
      <TestCaseFunction test_julian_ephemeris_day>
      <TestCaseFunction test_julian_ephemeris_millenium>
      <TestCaseFunction test_local_hour_angle>
      <TestCaseFunction test_longitude_nutation>
      <TestCaseFunction test_mean_anomaly_moon>
      <TestCaseFunction test_mean_anomaly_sun>
      <TestCaseFunction test_mean_ecliptic_obliquity>
      <TestCaseFunction test_mean_elongation>
      <TestCaseFunction test_mean_sidereal_time>
      <TestCaseFunction test_moon_argument_latitude>
      <TestCaseFunction test_moon_ascending_longitude>
      <TestCaseFunction test_obliquity_nutation>
      <TestCaseFunction test_parallax_sun_right_ascension>
      <TestCaseFunction test_solar_position>
      <TestCaseFunction test_topocentric_astronomers_azimuth>
      <TestCaseFunction test_topocentric_azimuth_angle>
      <TestCaseFunction test_topocentric_elevation_angle>
      <TestCaseFunction test_topocentric_elevation_angle_without_atmosphere>
      <TestCaseFunction test_topocentric_local_hour_angle>
      <TestCaseFunction test_topocentric_sun_declination>
      <TestCaseFunction test_topocentric_sun_right_ascension>
      <TestCaseFunction test_topocentric_zenith_angle>
      <TestCaseFunction test_transit_sunrise_sunset>
      <TestCaseFunction test_true_ecliptic_obliquity>
    <UnitTestCase NumbaSpaTest>
      <TestCaseFunction test_aberration_correction>
      <TestCaseFunction test_apparent_sidereal_time>
      <TestCaseFunction test_apparent_sun_longitude>
      <TestCaseFunction test_atmospheric_refraction_correction>
      <TestCaseFunction test_calculate_deltat>
      <TestCaseFunction test_earthsun_distance>
      <TestCaseFunction test_equation_of_time>
      <TestCaseFunction test_equatorial_horizontal_parallax>
      <TestCaseFunction test_geocentric_latitude>
      <TestCaseFunction test_geocentric_longitude>
      <TestCaseFunction test_geocentric_sun_declination>
      <TestCaseFunction test_geocentric_sun_right_ascension>
      <TestCaseFunction test_heliocentric_latitude>
      <TestCaseFunction test_heliocentric_longitude>
      <TestCaseFunction test_heliocentric_radius_vector>
      <TestCaseFunction test_julian_century>
      <TestCaseFunction test_julian_day>
      <TestCaseFunction test_julian_day_dt>
      <TestCaseFunction test_julian_ephemeris_century>
      <TestCaseFunction test_julian_ephemeris_day>
      <TestCaseFunction test_julian_ephemeris_millenium>
      <TestCaseFunction test_local_hour_angle>
      <TestCaseFunction test_longitude_nutation>
      <TestCaseFunction test_mean_anomaly_moon>
      <TestCaseFunction test_mean_anomaly_sun>
      <TestCaseFunction test_mean_ecliptic_obliquity>
      <TestCaseFunction test_mean_elongation>
      <TestCaseFunction test_mean_sidereal_time>
      <TestCaseFunction test_moon_argument_latitude>
      <TestCaseFunction test_moon_ascending_longitude>
      <TestCaseFunction test_obliquity_nutation>
      <TestCaseFunction test_parallax_sun_right_ascension>
      <TestCaseFunction test_solar_position>
      <TestCaseFunction test_solar_position_multithreaded>
      <TestCaseFunction test_solar_position_singlethreaded>
      <TestCaseFunction test_topocentric_astronomers_azimuth>
      <TestCaseFunction test_topocentric_azimuth_angle>
      <TestCaseFunction test_topocentric_elevation_angle>
      <TestCaseFunction test_topocentric_elevation_angle_without_atmosphere>
      <TestCaseFunction test_topocentric_local_hour_angle>
      <TestCaseFunction test_topocentric_sun_declination>
      <TestCaseFunction test_topocentric_sun_right_ascension>
      <TestCaseFunction test_topocentric_zenith_angle>
      <TestCaseFunction test_transit_sunrise_sunset>
      <TestCaseFunction test_true_ecliptic_obliquity>
  <Module test_tools.py>
    <Function test_build_kwargs[keys0-input_dict0-expected0]>
    <Function test_build_kwargs[keys1-input_dict1-expected1]>
    <Function test_build_kwargs[keys2-input_dict2-expected2]>
    <Function test_build_kwargs[keys3-input_dict3-expected3]>
<Module pvlib/tests/ivtools/test_sde.py>
  <Function test_fit_sandia_simple>
  <Function test_fit_sandia_simple_bad_iv>
  <Function test__fit_sandia_cocontent[i0-v0-2.0-expected0]>
  <Function test__fit_sandia_cocontent[i1-v1-10.0-expected1]>
  <Function test__fit_sandia_cocontent[i2-v2-15.0-expected2]>
  <Function test__fit_sandia_cocontent_fail>

=================================== ERRORS ====================================
_______________ ERROR collecting pvlib/tests/test_atmosphere.py _______________
ImportError while importing test module 'c:\Users\mikm\Projects\pvlib-python\pvlib\tests\test_atmosphere.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\importlib\__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
pvlib\tests\test_atmosphere.py:7: in <module>
    from conftest import assert_series_equal
E   ModuleNotFoundError: No module named 'conftest'
________________ ERROR collecting pvlib/tests/test_bifacial.py ________________
ImportError while importing test module 'c:\Users\mikm\Projects\pvlib-python\pvlib\tests\test_bifacial.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\importlib\__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
pvlib\tests\test_bifacial.py:4: in <module>
    from conftest import requires_pvfactors, assert_series_equal
E   ModuleNotFoundError: No module named 'conftest'
________________ ERROR collecting pvlib/tests/test_clearsky.py ________________
ImportError while importing test module 'c:\Users\mikm\Projects\pvlib-python\pvlib\tests\test_clearsky.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\importlib\__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
pvlib\tests\test_clearsky.py:11: in <module>
    from conftest import assert_frame_equal, assert_series_equal
E   ModuleNotFoundError: No module named 'conftest'
________________ ERROR collecting pvlib/tests/test_conftest.py ________________
ImportError while importing test module 'c:\Users\mikm\Projects\pvlib-python\pvlib\tests\test_conftest.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\importlib\__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
pvlib\tests\test_conftest.py:4: in <module>
    import conftest
E   ModuleNotFoundError: No module named 'conftest'
________________ ERROR collecting pvlib/tests/test_forecast.py ________________
ImportError while importing test module 'c:\Users\mikm\Projects\pvlib-python\pvlib\tests\test_forecast.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\importlib\__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
pvlib\tests\test_forecast.py:9: in <module>
    from conftest import (
E   ModuleNotFoundError: No module named 'conftest'
__________________ ERROR collecting pvlib/tests/test_iam.py ___________________
ImportError while importing test module 'c:\Users\mikm\Projects\pvlib-python\pvlib\tests\test_iam.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\importlib\__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
pvlib\tests\test_iam.py:11: in <module>
    from conftest import assert_series_equal
E   ModuleNotFoundError: No module named 'conftest'
________________ ERROR collecting pvlib/tests/test_inverter.py ________________
ImportError while importing test module 'c:\Users\mikm\Projects\pvlib-python\pvlib\tests\test_inverter.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\importlib\__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
pvlib\tests\test_inverter.py:4: in <module>
    from conftest import assert_series_equal
E   ModuleNotFoundError: No module named 'conftest'
_______________ ERROR collecting pvlib/tests/test_irradiance.py _______________
ImportError while importing test module 'c:\Users\mikm\Projects\pvlib-python\pvlib\tests\test_irradiance.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\importlib\__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
pvlib\tests\test_irradiance.py:14: in <module>
    from conftest import (
E   ModuleNotFoundError: No module named 'conftest'
________________ ERROR collecting pvlib/tests/test_location.py ________________
ImportError while importing test module 'c:\Users\mikm\Projects\pvlib-python\pvlib\tests\test_location.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\importlib\__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
pvlib\tests\test_location.py:7: in <module>
    from conftest import assert_frame_equal, assert_index_equal
E   ModuleNotFoundError: No module named 'conftest'
_______________ ERROR collecting pvlib/tests/test_modelchain.py _______________
ImportError while importing test module 'c:\Users\mikm\Projects\pvlib-python\pvlib\tests\test_modelchain.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\importlib\__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
pvlib\tests\test_modelchain.py:13: in <module>
    from conftest import assert_series_equal, assert_frame_equal
E   ModuleNotFoundError: No module named 'conftest'
__________ ERROR collecting pvlib/tests/test_numerical_precision.py ___________
ImportError while importing test module 'c:\Users\mikm\Projects\pvlib-python\pvlib\tests\test_numerical_precision.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\importlib\__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
pvlib\tests\test_numerical_precision.py:24: in <module>
    from conftest import DATA_DIR
E   ModuleNotFoundError: No module named 'conftest'
________________ ERROR collecting pvlib/tests/test_pvsystem.py ________________
ImportError while importing test module 'c:\Users\mikm\Projects\pvlib-python\pvlib\tests\test_pvsystem.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\importlib\__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
pvlib\tests\test_pvsystem.py:8: in <module>
    from conftest import (
E   ModuleNotFoundError: No module named 'conftest'
__________________ ERROR collecting pvlib/tests/test_snow.py __________________
ImportError while importing test module 'c:\Users\mikm\Projects\pvlib-python\pvlib\tests\test_snow.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\importlib\__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
pvlib\tests\test_snow.py:4: in <module>
    from conftest import assert_series_equal
E   ModuleNotFoundError: No module named 'conftest'
________________ ERROR collecting pvlib/tests/test_soiling.py _________________
ImportError while importing test module 'c:\Users\mikm\Projects\pvlib-python\pvlib\tests\test_soiling.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\importlib\__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
pvlib\tests\test_soiling.py:6: in <module>
    from conftest import assert_series_equal
E   ModuleNotFoundError: No module named 'conftest'
_____________ ERROR collecting pvlib/tests/test_solarposition.py ______________
ImportError while importing test module 'c:\Users\mikm\Projects\pvlib-python\pvlib\tests\test_solarposition.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\importlib\__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
pvlib\tests\test_solarposition.py:8: in <module>
    from conftest import assert_frame_equal, assert_series_equal
E   ModuleNotFoundError: No module named 'conftest'
________________ ERROR collecting pvlib/tests/test_spectrum.py ________________
ImportError while importing test module 'c:\Users\mikm\Projects\pvlib-python\pvlib\tests\test_spectrum.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\importlib\__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
pvlib\tests\test_spectrum.py:6: in <module>
    from conftest import DATA_DIR
E   ModuleNotFoundError: No module named 'conftest'
______________ ERROR collecting pvlib/tests/test_temperature.py _______________
ImportError while importing test module 'c:\Users\mikm\Projects\pvlib-python\pvlib\tests\test_temperature.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\importlib\__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
pvlib\tests\test_temperature.py:5: in <module>
    from conftest import DATA_DIR, assert_series_equal
E   ModuleNotFoundError: No module named 'conftest'
________________ ERROR collecting pvlib/tests/test_tracking.py ________________
ImportError while importing test module 'c:\Users\mikm\Projects\pvlib-python\pvlib\tests\test_tracking.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\importlib\__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
pvlib\tests\test_tracking.py:10: in <module>
    from conftest import DATA_DIR, assert_frame_equal
E   ModuleNotFoundError: No module named 'conftest'
______________ ERROR collecting pvlib/tests/iotools/test_crn.py _______________
ImportError while importing test module 'c:\Users\mikm\Projects\pvlib-python\pvlib\tests\iotools\test_crn.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\importlib\__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
pvlib\tests\iotools\test_crn.py:2: in <module>
    from conftest import assert_frame_equal
E   ModuleNotFoundError: No module named 'conftest'
___________ ERROR collecting pvlib/tests/iotools/test_ecmwf_macc.py ___________
ImportError while importing test module 'c:\Users\mikm\Projects\pvlib-python\pvlib\tests\iotools\test_ecmwf_macc.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\importlib\__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
pvlib\tests\iotools\test_ecmwf_macc.py:9: in <module>
    from conftest import requires_netCDF4, DATA_DIR
E   ModuleNotFoundError: No module named 'conftest'
______________ ERROR collecting pvlib/tests/iotools/test_epw.py _______________
ImportError while importing test module 'c:\Users\mikm\Projects\pvlib-python\pvlib\tests\iotools\test_epw.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\importlib\__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
pvlib\tests\iotools\test_epw.py:4: in <module>
    from conftest import DATA_DIR, RERUNS, RERUNS_DELAY
E   ModuleNotFoundError: No module named 'conftest'
______________ ERROR collecting pvlib/tests/iotools/test_midc.py ______________
ImportError while importing test module 'c:\Users\mikm\Projects\pvlib-python\pvlib\tests\iotools\test_midc.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\importlib\__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
pvlib\tests\iotools\test_midc.py:6: in <module>
    from conftest import DATA_DIR, RERUNS, RERUNS_DELAY
E   ModuleNotFoundError: No module named 'conftest'
______________ ERROR collecting pvlib/tests/iotools/test_psm3.py ______________
ImportError while importing test module 'c:\Users\mikm\Projects\pvlib-python\pvlib\tests\iotools\test_psm3.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\importlib\__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
pvlib\tests\iotools\test_psm3.py:7: in <module>
    from conftest import DATA_DIR, RERUNS, RERUNS_DELAY
E   ModuleNotFoundError: No module named 'conftest'
_____________ ERROR collecting pvlib/tests/iotools/test_pvgis.py ______________
ImportError while importing test module 'c:\Users\mikm\Projects\pvlib-python\pvlib\tests\iotools\test_pvgis.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\importlib\__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
pvlib\tests\iotools\test_pvgis.py:10: in <module>
    from conftest import DATA_DIR, RERUNS, RERUNS_DELAY
E   ModuleNotFoundError: No module named 'conftest'
_____________ ERROR collecting pvlib/tests/iotools/test_solrad.py _____________
ImportError while importing test module 'c:\Users\mikm\Projects\pvlib-python\pvlib\tests\iotools\test_solrad.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\importlib\__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
pvlib\tests\iotools\test_solrad.py:2: in <module>
    from conftest import assert_frame_equal
E   ModuleNotFoundError: No module named 'conftest'
______________ ERROR collecting pvlib/tests/iotools/test_srml.py ______________
ImportError while importing test module 'c:\Users\mikm\Projects\pvlib-python\pvlib\tests\iotools\test_srml.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\importlib\__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
pvlib\tests\iotools\test_srml.py:6: in <module>
    from conftest import DATA_DIR, RERUNS, RERUNS_DELAY
E   ModuleNotFoundError: No module named 'conftest'
____________ ERROR collecting pvlib/tests/iotools/test_surfrad.py _____________
ImportError while importing test module 'c:\Users\mikm\Projects\pvlib-python\pvlib\tests\iotools\test_surfrad.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\importlib\__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
pvlib\tests\iotools\test_surfrad.py:5: in <module>
    from conftest import DATA_DIR, RERUNS, RERUNS_DELAY
E   ModuleNotFoundError: No module named 'conftest'
______________ ERROR collecting pvlib/tests/iotools/test_tmy.py _______________
ImportError while importing test module 'c:\Users\mikm\Projects\pvlib-python\pvlib\tests\iotools\test_tmy.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\importlib\__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
pvlib\tests\iotools\test_tmy.py:4: in <module>
    from conftest import DATA_DIR
E   ModuleNotFoundError: No module named 'conftest'
______________ ERROR collecting pvlib/tests/ivtools/test_sdm.py _______________
ImportError while importing test module 'c:\Users\mikm\Projects\pvlib-python\pvlib\tests\ivtools\test_sdm.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\importlib\__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
pvlib\tests\ivtools\test_sdm.py:11: in <module>
    from conftest import DATA_DIR
E   ModuleNotFoundError: No module named 'conftest'
_____________ ERROR collecting pvlib/tests/ivtools/test_utils.py ______________
ImportError while importing test module 'c:\Users\mikm\Projects\pvlib-python\pvlib\tests\ivtools\test_utils.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\importlib\__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
pvlib\tests\ivtools\test_utils.py:7: in <module>
    from conftest import DATA_DIR
E   ModuleNotFoundError: No module named 'conftest'
=========================== short test summary info ===========================
ERROR pvlib/tests/test_atmosphere.py
ERROR pvlib/tests/test_bifacial.py
ERROR pvlib/tests/test_clearsky.py
ERROR pvlib/tests/test_conftest.py
ERROR pvlib/tests/test_forecast.py
ERROR pvlib/tests/test_iam.py
ERROR pvlib/tests/test_inverter.py
ERROR pvlib/tests/test_irradiance.py
ERROR pvlib/tests/test_location.py
ERROR pvlib/tests/test_modelchain.py
ERROR pvlib/tests/test_numerical_precision.py
ERROR pvlib/tests/test_pvsystem.py
ERROR pvlib/tests/test_snow.py
ERROR pvlib/tests/test_soiling.py
ERROR pvlib/tests/test_solarposition.py
ERROR pvlib/tests/test_spectrum.py
ERROR pvlib/tests/test_temperature.py
ERROR pvlib/tests/test_tracking.py
ERROR pvlib/tests/iotools/test_crn.py
ERROR pvlib/tests/iotools/test_ecmwf_macc.py
ERROR pvlib/tests/iotools/test_epw.py
ERROR pvlib/tests/iotools/test_midc.py
ERROR pvlib/tests/iotools/test_psm3.py
ERROR pvlib/tests/iotools/test_pvgis.py
ERROR pvlib/tests/iotools/test_solrad.py
ERROR pvlib/tests/iotools/test_srml.py
ERROR pvlib/tests/iotools/test_surfrad.py
ERROR pvlib/tests/iotools/test_tmy.py
ERROR pvlib/tests/ivtools/test_sdm.py
ERROR pvlib/tests/ivtools/test_utils.py
!!!!!!!!!!!!!!!!!! Interrupted: 30 errors during collection !!!!!!!!!!!!!!!!!!!
=================== 142 tests collected, 30 errors in 2.96s ===================

Traceback (most recent call last):
  File "c:\Users\mikm\.vscode\extensions\ms-python.python-2021.2.582707922\pythonFiles\testing_tools\run_adapter.py", line 22, in <module>
    main(tool, cmd, subargs, toolargs)
  File "c:\Users\mikm\.vscode\extensions\ms-python.python-2021.2.582707922\pythonFiles\testing_tools\adapter\__main__.py", line 100, in main
    parents, result = run(toolargs, **subargs)
  File "c:\Users\mikm\.vscode\extensions\ms-python.python-2021.2.582707922\pythonFiles\testing_tools\adapter\pytest\_discovery.py", line 44, in discover
    raise Exception("pytest discovery failed (exit code {})".format(ec))
Exception: pytest discovery failed (exit code 2)

Tests also fail from terminal if tests/ folder includes __init__.py:

(venv) C:\Users\mikm\Projects\pvlib-python>pytest pvlib\tests\test_atmosphere.py
==================================================================================================== test session starts =====================================================================================================
platform win32 -- Python 3.6.8, pytest-6.2.2, py-1.10.0, pluggy-0.13.1
rootdir: C:\Users\mikm\Projects\pvlib-python, configfile: setup.cfg
plugins: cov-2.11.1, mock-3.5.1, remotedata-0.3.2, rerunfailures-9.1.1, timeout-1.4.2
collected 0 items / 1 error                                                                                                                                                                                                   

=========================================================================================================== ERRORS ===========================================================================================================
______________________________________________________________________________________ ERROR collecting pvlib/tests/test_atmosphere.py _______________________________________________________________________________________
ImportError while importing test module 'C:\Users\mikm\Projects\pvlib-python\pvlib\tests\test_atmosphere.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\importlib\__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
pvlib\tests\test_atmosphere.py:7: in <module>
    from conftest import assert_series_equal
E   ModuleNotFoundError: No module named 'conftest'
================================================================================================== short test summary info ===================================================================================================
ERROR pvlib/tests/test_atmosphere.py
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
====================================================================================================== 1 error in 0.19s ======================================================================================================

Could this have anything to do with the conftests.py file in the tests folder? Or the fact that tests/ is inside of the top-level package pvlib instead of at the root of the project repo? EG: project root is "pvlib-python" with package pvlib and tests are in pvlib/tests. See https://github.com/pvlib/pvlib-python

[brettcannon]

It's quite possible that is what is messing things up. If you set "python.testing.pytestArgs": ["tests"]" does that change anything?

[mikofski]

Or the fact that tests/ is inside of the top-level package pvlib instead of at the root of the project repo?

I moved the tests/ folder from pvlib/ to the top repo project folder so it's side x side with the pvlib package, EG: pvlib-python/tests vs. pvlib-python/pvlib/tests, and the problem still persists: tests discovered, but do not run except in debug.

set "python.testing.pytestArgs": ["tests"]

• if I set this while the tests/ folder is inside pvlib/ then test discovery fails ❌
• if I set "python.testing.pytestArgs": ["pvlib\\tests"] with tests/ inside pvlib/ then test discovery works ✔️ but run tests fails ❌
• if I move the tests/ folder to the same top level project folder as pvlib/ and set "python.testing.pytestArgs": ["tests"] , then test discovery works ✔️ but run tests fails ❌
• other variations of "python.testing.pytestArgs" work for test discovery as well, but nothing gets the test to run other than while in debug. The error is always the same:

  ImportError while loading conftest 'c:\Users\mikm\Projects\pvlib-python\pvlib\tests\conftest.py'.
  pvlib\tests\conftest.py:10: in <module>
      import pvlib
  E   ModuleNotFoundError: No module named 'pvlib'
  Error: TypeError: Cannot read property 'testsuites' of null
  

I think this has to be a duplicate of #14570 because the error is always the same, pvlib can't be imported because the Python path is altered when tests are run to remove the current working directory. The only way around this is to either install pvlib in non-editable mode, or sneak in sys.path.append("path/to/pvlib") in tests/conftest.py to fix the Python path. I don't know why the "python.useIsolation": false setting didn't work, perhaps I did something wrong. I could try it again, or just wait until this feature makes it into production. Thanks so much for your help!!!

[brettcannon]

It's possible the "python.useIsolation" setting didn't get plumbed through enough and we missed a case that somehow involves testing.

[mikofski]

HI @brettcannon it turns out the conftest.py was the culprit. Resolved in pvlib/pvlib-python#1204 and marking this as answered, thanks again!