Example of using a "src" dir with gazelle

[dougthor42]

N.B.: This is half "example request", half "how do I..." question.

🚀 feature/example request

Relevant Rules

• py_*
• gazelle

Description

The Python Packaging User Guide recommends using a src dir with tests outside of the package (so they aren't shipped with the distribution/wheel), like so:

packaging_tutorial/
├── LICENSE
├── pyproject.toml
├── README.md
├── src/
│   └── mypackage/
│       ├── __init__.py
│       └── foo.py
└── tests/
    ├── __init__.py
    └── test_foo.py

pytest also recommends this.

However, none of the examples describe such a use case¹.

Critically, one major aspect of the above dir structure is that the project must be pip-installed²³ before tests can be run because test_foo.py looks like:

import unittest

from mypackage import foo  # here's the problem. Note that it's not `from src.mypackage import foo`

class TestFoo(unittest.testcase):
    def test_add(self) -> None:
        self.assertEqual(foo.add(1, 1), 2)

In addition, the documentation for gazelle is lacking and I haven't been able to figure out a way to get gazelle to work with a src dir.

Notes:

• I think that There is no way to specify a root of python's package structure bazel#6903 is similar.
• I'm not asking for bazel to be able to do editable installs
  • as said in [pip_install] ERROR: File "setup.py" not found. Directory cannot be installed in editable mode #434 (comment), doing so prevents hermeticity

Describe the solution you'd like

What I'd like to see is a new example added that showcases how to configure bazel and gazelle to work with a src dir.

In fact, I've already got a repo for it that we can use as a starting point. General bazel build|test|run works, but I am still struggling with gazelle. I'd be more than happy to build the example, but I'll need help doing so. [Edit 2024-04-11: With recent updates to gazelle, things are now working 😁]

The example would have the following structure (names are just suggestions, of course):

examples/src_dir_with_separate_tests/
├── BUILD
├── MODULE.bazel
├── README.md
├── pyproject.toml
├── requirements.in
├── src
│   └── mypackage
│       ├── BUILD
│       ├── __init__.py
│       ├── foo.py
│       └── subpackage
│           ├── BUILD
│           ├── __init__.py
│           └── subfoo.py
└── tests
    ├── BUILD
    ├── __init__.py
    ├── subpackage
    │   ├── BUILD
    │   ├── __init__.py
    │   └── test_subfoo.py
    └── test_foo.py

Describe alternatives you've considered

I tried looking for other examples on the web, but either my google-fu is failing me or there aren't any 😞.

Footnotes

1. Note: the bzlmod example appears to do something similar with the libs/my_lib dir, but it's not quite the same because libs/my_lib doesn't need to be pip-installed to run tests. IMO the example also does too much, but that's a separate topic 🙃. ↩

2. typically as an editable package pip install -e ., but a non-editable install also works. ↩

3. Really all that's needed is .../packaging_tutorial/src to be in PYTHONPATH. ↩

[Asmhoa]

Thanks for raising and putting up the example! Super helpful