Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for virtual projects #6585

Merged
merged 2 commits into from
Aug 27, 2024
Merged

Add support for virtual projects #6585

merged 2 commits into from
Aug 27, 2024

Conversation

charliermarsh
Copy link
Member

@charliermarsh charliermarsh commented Aug 24, 2024

Summary

The basic idea here is: any project can either be a package, or not ("virtual").

If a project is virtual, we don't build or install it.

A project is virtual if either of the following are true:

  • tool.uv.virtual = true is set.
  • [build-system] is absent.

The concept of "virtual projects" only applies to workspace member right now; it doesn't apply to path dependencies which are treated like arbitrary Python source trees.

TODOs that should be resolved prior to merging:

  • Documentation
  • How do we reconcile this with "virtual workspace roots" which are a little different -- they omit [project] entirely and don't even have a name?
  • uv init --virtual should create a virtual project rather than a virtual workspace.
  • Running uv sync in a virtual project after uv init --virtual shows Audited 0 packages in 0.01ms, which is awkward. (See: Improve messages for empty solves and installs #6588.)

Closes #6511.


[build-system]
requires = ["setuptools>=42", "wheel"]
build-backend = "setuptools.build_meta"
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately, we do need to set an explicit build backend for our test projects now. This is fine and correct, we were just omitting it before.

@charliermarsh
Copy link
Member Author

This needs documentation.

@charliermarsh
Copy link
Member Author

I wonder if we should use different terminology here to differentiate from a "virtual workspace root".

@charliermarsh
Copy link
Member Author

We might be better off with a concept of "applications" vs. "libraries" here?

@charliermarsh
Copy link
Member Author

Or, even better, we remove virtual workspace roots entirely.

@charliermarsh charliermarsh force-pushed the charlie/virtuals branch 3 times, most recently from fc0f0b6 to 56f78fb Compare August 24, 2024 16:19
@charliermarsh
Copy link
Member Author

I feel pretty good about this apart from the use of "virtual" here and the fact that a "virtual workspace root" is slightly different.

@mkniewallner
Copy link
Contributor

"virtual" seems a bit abstract to me, and not really self-explanatory about what it does.

Some background on this:

From those two, I find Poetry's one to be a better fit, as "distribution" seems like this is only about preventing publishing a package.

What about using something a bit more explicit like project-type that could either be application or package (or library)? I personally find those concepts closer to what people are used to when creating Python projects. Using an enum also make it possible to add more values in the future (although I don't really see what else we could have).

@charliermarsh
Copy link
Member Author

My preferred framing here is application vs. library (in Rust it’s “binary” vs. “library”), but curious to hear from @zanieb on it when they’re back.

@a-reich
Copy link

a-reich commented Aug 25, 2024

IIUC, [project] still must have name and version for virtual projects/applications - does it make sense to make them optional?

@charliermarsh
Copy link
Member Author

No, I still plan to require those. Otherwise, it’s not clear how to identify a given package in the lockfile.

@a-reich
Copy link

a-reich commented Aug 25, 2024

Hmm ok, I guess I misunderstood the mental model, as it hadn’t occurred to me that a project was allowed to depend on an application.

@charliermarsh
Copy link
Member Author

Just depends on the semantics we want to allow!

@philgyford
Copy link

No, I still plan to require those. Otherwise, it’s not clear how to identify a given package in the lockfile.

Fair enough if there’s no technical way around it, but it feels very odd to require it for something like a website.

Even on my solo, personal sites I might commit and deploy several times a day, and having to decide on a version number every time (and remember to change it) would be a real pain.

@z3z1ma
Copy link

z3z1ma commented Aug 25, 2024

Even on my solo, personal sites I might commit and deploy several times a day, and having to decide on a version number every time (and remember to change it) would be a real pain.

You dont need to. Set the fields once in pyproject toml. Since you are not building a package with a build backend, the values are moot. Just use them as metadata for your app.

@sersorrel
Copy link

I'm not sure that the "library"/"application" framing necessarily corresponds well to "should/should not be installable". You want a library to be an installable package, so that it can be depended upon; but you also (sometimes?) want an application to be an installable package, so that you can install and run it (say, pipx install uv).

@minkooseo
Copy link

What about "non package" or "not a package" then? Not sure if it's a good flag name, but it's something we can use to explain what this is about. It's also a term used in #6578.

Coming from the experience of always putting sources at the top, 'git push' and then 'git pull' at the server to serve web traffic, it was quite new to me that I should put code under src/project_name. I learned that I build a package and publish it that way, and I can create a package and install it on the server.

It is still a project for me, so I was searching the doc to understand what's a project then if mine is virtual. (I still don't know)

It is an application and not a library, but it is now clear to me that application can still be an installable package. It may create confusion for people with knowledge.

@charliermarsh
Copy link
Member Author

That's actually what we're using in this PR. The flag is package = false etc.

@zanieb
Copy link
Member

zanieb commented Aug 27, 2024

@minkooseo might be helpful to look at #6689 too, but we're going to push out a bunch of changes around this soon.

@charliermarsh charliermarsh merged commit eb14056 into main Aug 27, 2024
57 checks passed
@charliermarsh charliermarsh deleted the charlie/virtuals branch August 27, 2024 17:42
tmeijn pushed a commit to tmeijn/dotfiles that referenced this pull request Aug 30, 2024
This MR contains the following updates:

| Package | Update | Change |
|---|---|---|
| [astral-sh/uv](https://github.com/astral-sh/uv) | minor | `0.3.5` -> `0.4.0` |

MR created with the help of [el-capitano/tools/renovate-bot](https://gitlab.com/el-capitano/tools/renovate-bot).

**Proposed changes to behavior should be submitted there as MRs.**

---

### Release Notes

<details>
<summary>astral-sh/uv (astral-sh/uv)</summary>

### [`v0.4.0`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#040)

[Compare Source](astral-sh/uv@0.3.5...0.4.0)

This release adds first-class support for Python projects that are not designed as Python packages (e.g., web applications, data science projects, etc.).

In doing so, it includes some breaking changes around uv's handling of projects. Previously, uv required that all projects could be built into distributable Python packages, and installed them into the virtual environment. Projects created by `uv init` always included a `[build-system]` definition and existing projects that did not define a `[build-system]` would use the legacy setuptools build backend by default.

Most users are not developing libraries that need to be packaged and published to PyPI. Instead, they're building applications using web frameworks, or running collections of Python scripts in the project's root directory. In these cases, requiring a `[build-system]` was confusing and error-prone. In this release, uv changes the default behavior to orient around these common use cases.

In summary, the major changes are:

-   uv no longer attempts to package and install projects that do not define a `[build-system]`.
    -   While the project itself will not be installed into the virtual environment, its dependencies will still be included.
    -   The previous behavior can be recovered by setting `package = true` in the `[tool.uv]` section of your `pyproject.toml`.
-   `uv init` no longer creates a `src/` directory or defines a `[build-system]` by default.
    -   The previous behavior can be recovered with `uv init --lib` or `uv init --app --package`.
-   uv allows and recommends including `[project]` definitions in virtual workspace roots.
    -   Previously, the uv required the `[project]` section to be omitted.
-   uv allows disabling packaging of projects, even if they define a `[build-system]`, by setting `package = false` in the `[tool.uv]` section of your `pyproject.toml`.

See the latest documentation on [build systems in projects](http://docs.astral.sh/uv/concepts/projects/#build-systems) for more details.

##### Enhancements

-   Add first-class support for non-packaged projects ([#&#8203;6585](astral-sh/uv#6585))
-   Add `--app` and `--lib` options to `uv init` ([#&#8203;6689](astral-sh/uv#6689))
-   Use `virtual` source label in lockfile for non-packaged dependencies ([#&#8203;6728](astral-sh/uv#6728))
-   Read hash from URL fragment if `--hashes` are omitted  ([#&#8203;6731](astral-sh/uv#6731))
-   Support `{package}@&#8203;{version}` in `uv tool install` ([#&#8203;6762](astral-sh/uv#6762))
-   Publish additional Docker tags without patch version ([#&#8203;6734](astral-sh/uv#6734))

##### Bug fixes

-   Accept either strings or structs for hosts ([#&#8203;6763](astral-sh/uv#6763))
-   Avoid including non-excluded members in parent workspaces ([#&#8203;6735](astral-sh/uv#6735))
-   Avoid reading stale `.egg-info` from mutable sources ([#&#8203;6714](astral-sh/uv#6714))
-   Avoid writing invalid PEP 723 scripts on `tool.uv.sources` ([#&#8203;6706](astral-sh/uv#6706))
-   Compare virtual members when invalidating lockfile ([#&#8203;6754](astral-sh/uv#6754))
-   Do not require workspace members to sync with `--frozen` ([#&#8203;6737](astral-sh/uv#6737))
-   Implement deserialization for trusted host ([#&#8203;6716](astral-sh/uv#6716))
-   Avoid showing duplicate paths in `uv python list` ([#&#8203;6740](astral-sh/uv#6740))
-   Raise an error for unclosed script tags in PEP 723 scripts ([#&#8203;6704](astral-sh/uv#6704))

##### Documentation

-   Add dependabot and renovate documentation page ([#&#8203;6236](astral-sh/uv#6236))
-   Bind to the host to allow connections in FastAPI Docker example ([#&#8203;6753](astral-sh/uv#6753))
-   Fix some broken links ([#&#8203;6705](astral-sh/uv#6705))
-   Update FastAPI guide for virtual projects and use `uv init` to create the `pyproject.toml` ([#&#8203;6752](astral-sh/uv#6752))
-   Update project documentation for the application / library concepts ([#&#8203;6718](astral-sh/uv#6718))
-   Update workspace documentation to remove legacy virtual projects ([#&#8203;6720](astral-sh/uv#6720))

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever MR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this MR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this MR, check this box

---

This MR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy40NDAuNyIsInVwZGF0ZWRJblZlciI6IjM3LjQ0MC43IiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJSZW5vdmF0ZSBCb3QiXX0=-->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add first-class support for virtual projects
8 participants