Set pyproject.toml #3821

bigstones · 2023-11-19T17:42:27Z

What changes are proposed in this pull request?

(Please fill in changes proposed in this fix)
I think it easier to configure the environment with poetry.
I created a pyproject.toml file based on my setup.py file.

How is this patch tested? If it is not, please explain why.

(Details)

$ poetry shell
$ poetry update
$ poetry install
$ python

import fiftyone as fo
import fiftyone.zoo as foz
dataset = foz.load_zoo_dataset("quickstart")
session = fo.launch_app(dataset)

Release Notes

Is this a user-facing change that should be mentioned in the release notes?

No. You can skip the rest of this section.
Yes. Give a description of this change to be included in the release

notes for FiftyOne users.

(Details in 1-2 sentences. You can just refer to another PR with a description

if this PR is part of a larger change.)

If you want to set up a development environment, you can do it easily with poetry.

What areas of FiftyOne does this PR affect?

App: FiftyOne application changes
Build: Build and test infrastructure changes
Core: Core fiftyone Python library changes
Documentation: FiftyOne documentation changes
Other

benjaminpkane · 2023-11-21T23:23:10Z

Hi @bigstones! Agree, poetry is much better and the best practice. If you have the time, the install script may need to be updated. And removing the requirements directory and setup.py in favor of a full the pyproject.toml is also another detail. If not, I can handle after Thanksgiving.

Thanks for the contribution! I'll follow up next week.

bigstones · 2023-11-22T01:44:01Z

Hi @bigstones! Agree, poetry is much better and the best practice. If you the time, the install script may need to be updated. And removing the requirements directory and setup.py in favor of a full the pyproject.toml is also another detail. If not, I can handle after Thanksgiving.

Thanks for the contribution! I'll follow up next week.

Hi @benjaminpkane! I will try to work on it as I have time this week.

Thanks for your reply

codecov · 2023-11-28T01:17:25Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (8c018ed) 15.85% compared to head (6cb00aa) 15.65%.
Report is 1 commits behind head on develop.

❗ Current head 6cb00aa differs from pull request most recent head 867fb25. Consider uploading reports for the commit 867fb25 to get more accurate results

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #3821      +/-   ##
===========================================
- Coverage    15.85%   15.65%   -0.21%     
===========================================
  Files          731      728       -3     
  Lines        81886    80536    -1350     
  Branches      1093     1074      -19     
===========================================
- Hits         12987    12609     -378     
+ Misses       68899    67927     -972

Flag	Coverage Δ
app	`15.65% <ø> (-0.21%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

benjaminpkane · 2023-11-28T22:30:10Z

Thanks for the updates! Changes look great 🚀 I will test tomorrow

brimoor · 2023-11-29T02:49:36Z

pyproject.toml

 [build-system]
-requires = ["importlib-metadata; python_version<'3.8'", "setuptools", "wheel"]


@benjaminpkane we still need this right?

Yes 👍

Will need to update GitHub workflows as well

benjaminpkane · 2023-12-01T21:15:04Z

Will circle back to this after 0.23.0 is released. Pardon the delay! Want to make sure development will not be interrupted before merging

benjaminpkane · 2023-12-06T23:22:50Z

Hi @bigstones. I think we can move to a pyproject.toml, but I'm not sure we'll be able to use poetry as a build system unless the package_dir and dynamic requires settings are supported. I will follow up with adjustments tomorrow.

bigstones · 2023-12-07T13:41:31Z

Hi @bigstones. I think we can move to a pyproject.toml, but I'm not sure we'll be able to use poetry as a build system unless the package_dir and dynamic requires settings are supported. I will follow up with adjustments tomorrow.

Hi @benjaminpkane

I believe Poetry is beneficial for dynamic requires settings due to its support for managing virtual environments. As for the package_dir configuration, I'm aware that Poetry also provides support for it, and I'll look into it a bit more for confirmation.

Thanks

benjaminpkane · 2023-12-07T16:10:52Z

Hi @bigstones. I think we can move to a pyproject.toml, but I'm not sure we'll be able to use poetry as a build system unless the package_dir and dynamic requires settings are supported. I will follow up with adjustments tomorrow.

Hi @benjaminpkane

I believe Poetry is beneficial for dynamic requires settings due to its support for managing virtual environments. As for the package_dir configuration, I'm aware that Poetry also provides support for it, and I'll look into it a bit more for confirmation.

Thanks

Ok, I will wait for your confirmation. The requirement is we need to ensure that one and only one of opencv-python, opencv-contrib-python, opencv-contrib-python-headless, opencv-python-headless is installed when installing a the source distribution.

The package_dir requirement is that tutorials and recipes subpackages are installed with the main fiftyone package.

/Users/ben/venvs/fo/lib/python3.9/site-packages/fiftyone
(fo) ➜  fiftyone ll
total 32
-rw-r--r--   1 ben  staff   828B Dec  7 10:08 __init__.py
-rw-r--r--   1 ben  staff   4.4K Dec  7 10:08 __public__.py
drwxr-xr-x   7 ben  staff   224B Dec  7 10:08 brain
-rw-r--r--   1 ben  staff   3.2K Dec  7 10:08 constants.py
drwxr-xr-x  42 ben  staff   1.3K Dec  7 10:08 core
drwxr-xr-x   4 ben  staff   128B Dec  6 16:08 db
drwxr-xr-x   5 ben  staff   160B Dec  7 10:08 factory
drwxr-xr-x   4 ben  staff   128B Dec  7 10:08 internal
drwxr-xr-x   5 ben  staff   160B Dec  7 10:08 migrations
drwxr-xr-x  13 ben  staff   416B Dec  7 10:08 operators
drwxr-xr-x   8 ben  staff   256B Dec  7 10:08 plugins
drwxr-xr-x   3 ben  staff    96B Dec  7 10:08 recipes
drwxr-xr-x   5 ben  staff   160B Dec  7 10:08 resources
drwxr-xr-x  34 ben  staff   1.1K Dec  7 10:08 server
drwxr-xr-x   6 ben  staff   192B Dec  7 10:08 service
drwxr-xr-x   3 ben  staff    96B Dec  7 10:08 tutorials
drwxr-xr-x   4 ben  staff   128B Dec  7 10:08 types
drwxr-xr-x  50 ben  staff   1.6K Dec  7 10:08 utils
drwxr-xr-x   5 ben  staff   160B Dec  7 10:08 zoo

bigstones · 2023-12-09T15:06:30Z

Hi @bigstones. I think we can move to a pyproject.toml, but I'm not sure we'll be able to use poetry as a build system unless the package_dir and dynamic requires settings are supported. I will follow up with adjustments tomorrow.

Hi @benjaminpkane
I believe Poetry is beneficial for dynamic requires settings due to its support for managing virtual environments. As for the package_dir configuration, I'm aware that Poetry also provides support for it, and I'll look into it a bit more for confirmation.
Thanks

Ok, I will wait for your confirmation. The requirement is we need to ensure that one and only one of opencv-python, opencv-contrib-python, opencv-contrib-python-headless, opencv-python-headless is installed when installing a the source distribution.

The package_dir requirement is that tutorials and recipes subpackages are installed with the main fiftyone package.
/Users/ben/venvs/fo/lib/python3.9/site-packages/fiftyone
(fo) ➜  fiftyone ll
total 32
-rw-r--r--   1 ben  staff   828B Dec  7 10:08 __init__.py
-rw-r--r--   1 ben  staff   4.4K Dec  7 10:08 __public__.py
drwxr-xr-x   7 ben  staff   224B Dec  7 10:08 brain
-rw-r--r--   1 ben  staff   3.2K Dec  7 10:08 constants.py
drwxr-xr-x  42 ben  staff   1.3K Dec  7 10:08 core
drwxr-xr-x   4 ben  staff   128B Dec  6 16:08 db
drwxr-xr-x   5 ben  staff   160B Dec  7 10:08 factory
drwxr-xr-x   4 ben  staff   128B Dec  7 10:08 internal
drwxr-xr-x   5 ben  staff   160B Dec  7 10:08 migrations
drwxr-xr-x  13 ben  staff   416B Dec  7 10:08 operators
drwxr-xr-x   8 ben  staff   256B Dec  7 10:08 plugins
drwxr-xr-x   3 ben  staff    96B Dec  7 10:08 recipes
drwxr-xr-x   5 ben  staff   160B Dec  7 10:08 resources
drwxr-xr-x  34 ben  staff   1.1K Dec  7 10:08 server
drwxr-xr-x   6 ben  staff   192B Dec  7 10:08 service
drwxr-xr-x   3 ben  staff    96B Dec  7 10:08 tutorials
drwxr-xr-x   4 ben  staff   128B Dec  7 10:08 types
drwxr-xr-x  50 ben  staff   1.6K Dec  7 10:08 utils
drwxr-xr-x   5 ben  staff   160B Dec  7 10:08 zoo

Thank you for your patience.

I've reviewed the two points:

It's confirmed that Poetry does not provide dynamic requires, and it seems unlikely to be supported in the future as well. You can find more details in the following link: python-poetry/poetry#8509. Additionally, I verified that installing FiftyOne with the poetry build command results in the installation of opencv-python-headless.

To set the package_dir, it appears that a directory structure change is necessary. Currently, 'recipes' and 'tutorials' are located under 'docs/source,' and it seems that they need to be moved to 'docs/source/fiftyone.' More information can be found in the following links: https://python-poetry.org/docs/pyproject/#packages and python-poetry/poetry#6258.

I've made the code changes and pushed them. If you encounter any issues, please let me know. Thank you.

benjaminpkane · 2023-12-14T16:01:40Z

Thanks @bigstones. It would be ideal if we could use poetry, but without the ability install a correct opencv dependency, I don't think it can be used. We want to be able to support users who choose to install from source so the environment doesn't break because they have conflicting opencv packages. This issue has discussion about that.

Moving to a pyproject.toml with a setuptools backend that retains this behavior would be a welcomed contribution 🙏

bigstones · 2023-12-18T20:46:02Z

Thanks @bigstones. It would be ideal if we could use poetry, but without the ability install a correct opencv dependency, I don't think it can be used. We want to be able to support users who choose to install from source so the environment doesn't break because they have conflicting opencv packages. This issue has discussion about that.

Moving to a pyproject.toml with a setuptools backend that retains this behavior would be a welcomed contribution 🙏

Oh I see, I will modify the project to maintain the necessary files from setup.py while also making them compatible with pyproject.toml.

benjaminpkane · 2023-12-21T18:55:16Z

Hi @bigstones. I've done a fair amount of investigating to see how we could use poetry but it doesn't seem like a good fit right now.

I've pushed a new branch named hatch that includes your pyproject.toml contribution, and adds hatchling for builds. If you merge that into this PR, I think we can get this merged! No rush. cc @findtopher

This new setup has actually reduces the build and source distribution sizes and makes things a lot simpler to manage! 🙇 🚀

Note that when python-poetry/poetry#3332 is resolved, poetry can likely be used to manage a development install of fiftyone.

With hatchling, we are build allow a correct dependency resolution for opencv (see #1494) when the below is used

pip install fiftyone --no-build-isolation --no-binary fiftyone,voxel51-eta

Note that using hatchling for builds will drop support for source installs on Python 3.7

bigstones · 2023-12-25T05:56:25Z

Hi @benjaminpkane , thank you for your answer. I'm glad to learn about the Hatchling library. As it's a library I'm encountering for the first time, it seems necessary to conduct more tests. I think I'll have some time to do this around next year. Wishing you a lot of happiness in the New Year, and I'll be returning soon.
Thank you.

benjaminpkane · 2024-01-10T16:22:32Z

Hi @bigstones. I think I have most of the hatchling work completed. Will merge this and finish it up! Thanks!

Set pyproject.toml

7f4daad

brimoor requested a review from benjaminpkane November 20, 2023 13:36

bigstones added 4 commits November 25, 2023 08:37

Set pyproject.toml based on requirements

0c91404

Modify the installation to use Poetry

42c7f83

Remove the requirements files

207f8f3

fix install bash

6cb00aa

brimoor reviewed Nov 29, 2023

View reviewed changes

bigstones added 3 commits December 9, 2023 23:59

add gitignore poetry.lock .venv .python-version

bcf0cc9

update pyproject.toml for package_dir include

77693dc

move doc/source/* -> doc/source/fiftyone/*

df208bc

bigstones and others added 8 commits December 19, 2023 05:49

apply pyproject.toml from to python-poetry/poetry#8770

c9eee3a

move doc/source/fiftyone/* -> doc/source/*

9335d98

Merge branch 'voxel51:develop' into develop

0b65952

restore requirements

4739d39

install bash with setup py

2a51479

update docs/source/conf.py to latest

00e5a4c

restore settings docs/source/fiftyone -> docs/source

422214a

create install_by_poetry.bash

9a7ae2c

This comment was marked as off-topic.

Sign in to view

restore setup.py docs/source/fiftyone -> docs/source

5d522c9

Merge branch 'voxel51:develop' into develop

867fb25

benjaminpkane changed the base branch from develop to hatch January 10, 2024 16:18

benjaminpkane changed the base branch from hatch to pyproject January 10, 2024 16:20

benjaminpkane merged commit d8eb906 into voxel51:pyproject Jan 10, 2024
3 of 6 checks passed

benjaminpkane mentioned this pull request Jan 17, 2024

Move to pyproject.toml package file #4019

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Set pyproject.toml #3821

Set pyproject.toml #3821

bigstones commented Nov 19, 2023

benjaminpkane commented Nov 21, 2023 •

edited

Loading

bigstones commented Nov 22, 2023

codecov bot commented Nov 28, 2023 •

edited

Loading

benjaminpkane commented Nov 28, 2023

brimoor Nov 29, 2023

benjaminpkane Nov 30, 2023

benjaminpkane commented Dec 1, 2023

benjaminpkane commented Dec 6, 2023 •

edited

Loading

bigstones commented Dec 7, 2023

benjaminpkane commented Dec 7, 2023 •

edited

Loading

bigstones commented Dec 9, 2023

benjaminpkane commented Dec 14, 2023

bigstones commented Dec 18, 2023

This comment was marked as off-topic.

benjaminpkane commented Dec 21, 2023 •

edited

Loading

bigstones commented Dec 25, 2023

benjaminpkane commented Jan 10, 2024

		[build-system]
		requires = ["importlib-metadata; python_version<'3.8'", "setuptools", "wheel"]

Set pyproject.toml #3821

Set pyproject.toml #3821

Conversation

bigstones commented Nov 19, 2023

What changes are proposed in this pull request?

How is this patch tested? If it is not, please explain why.

Release Notes

Is this a user-facing change that should be mentioned in the release notes?

What areas of FiftyOne does this PR affect?

benjaminpkane commented Nov 21, 2023 • edited Loading

bigstones commented Nov 22, 2023

codecov bot commented Nov 28, 2023 • edited Loading

Codecov Report

benjaminpkane commented Nov 28, 2023

brimoor Nov 29, 2023

Choose a reason for hiding this comment

benjaminpkane Nov 30, 2023

Choose a reason for hiding this comment

benjaminpkane commented Dec 1, 2023

benjaminpkane commented Dec 6, 2023 • edited Loading

bigstones commented Dec 7, 2023

benjaminpkane commented Dec 7, 2023 • edited Loading

bigstones commented Dec 9, 2023

benjaminpkane commented Dec 14, 2023

bigstones commented Dec 18, 2023

This comment was marked as off-topic.

benjaminpkane commented Dec 21, 2023 • edited Loading

bigstones commented Dec 25, 2023

benjaminpkane commented Jan 10, 2024

benjaminpkane commented Nov 21, 2023 •

edited

Loading

codecov bot commented Nov 28, 2023 •

edited

Loading

benjaminpkane commented Dec 6, 2023 •

edited

Loading

benjaminpkane commented Dec 7, 2023 •

edited

Loading

benjaminpkane commented Dec 21, 2023 •

edited

Loading