Dev pre-release on PyPI

[ziw-liu]

A pre-release on PyPI can ease testing for downstream projects such as recOrder and microDL, and help us navigate PyPI issues before the 'stable' release. Once we have most of the 0.1.0 features ready we can tag a v0.1.0dev0 and use this issue to track the process.

[AhmetCanSolak]

I would suggest we go with v.0.0.1 when most features are ready @ziw-liu . We can have a shorther release cycle at the beginning but it is more important to not rush things at the beginning of our longer journey.

[ziw-liu]

I would suggest we go with v.0.0.1 when most features are ready @ziw-liu .

From my interpretation of semantic versioning the third digit is used for patches, e.g. when we have a first release of v0.1.0 we can have a maintenance branch called patches/0.1.x and the patched version will be v0.1.1 etc. Whenever a API is changed I think it is advised to increment the second digit, but I'm honestly not sure if v0.0.0 is a valid version number.

---

Editing with some reference:

The first version of a software usually starts at 0.1.0 and increments from there.

^ https://py-pkgs.org/07-releasing-versioning.html#version-numbering

Also see: semantic versioning

[AhmetCanSolak]

My experience is that people get (easily) upset as you break APIs they build on, even if you are only on minor versions. So my suggestion is mostly about being less promising to API users, I don't think anyone would get upset by not following semantic versioning conventions. I suspect we will develop and break many places in the near timeline.

[talonchandler]

I would like to start the process of making recOrder and waveorder depend on iohub.

I'm planning to:

1. Assist with any decisions that need to be made to merge Improve CLI and API reading entry points #92. @ziw-liu can you comment here or on Improve CLI and API reading entry points #92 with your suggestion for the best path forward. I think our collective decision on Monday was to rename imread then merge?
2. Tag the commit with a name like v0.1.0dev0 (I'm open to suggestions) without releasing to PyPI.
3. Remove WaveorderReader from waveorder and depend on the tagged version of iohub via github (Edit: in waveorder #112 we decided to make iohub an [examples] dependency only). After testing, release waveorder==1.0.0.
4. Remove WaveorderReader from recOrder and depend on the tagged version of iohub via github.

I think merging #92 is the current blocker, so I would appreciate progress and discussion on that point.

[mattersoflight]

@ziw-liu as we discussed offline, instead of a dev pre-release on pyPI, let's create a tag. Both @talonchandler and @ieivanov are eager to start depending on iohub in recOrder, waveOrder, mantis. And, I hear that you want to start depending on iohub in microDL.

It makes sense to create a tag and update the installation instructions in README and docs once iohub has the features needed by these four repos. Do we need to merge open PRs (#91 and #97) before the dev release?

[ziw-liu]

@mattersoflight thanks for summarizing the discussion. I think a dev tag is very low stake and I am happy with or without merging #97 (just merged #91).

As for the documentation update I don't know if the upcoming tag will be better for user installation than the HEAD of main since they are both not intended to be stable, and bleeding edge may have more bug fixes.

[mattersoflight]

@talonchandler, @ieivanov can you chime in on how useful #97 is for recOrder and mantis? Upon closer reading, it seems to be important for reading and writing recent data.

Re: documentation, I think it is useful to clarify in the READM that iohub can be installed in a conda environment in the following ways:
a) from the HEAD of main if you want truly bleeding edge code.
b) from a dev tag for a specific feature set that is reasonably tested.
c) by defining the HEAD of the main or on a tag as a dependency in the requirements.txt of your package.

The installation page should be updated to point to README before the PyPI release.

@AhmetCanSolak what do you think of this approach?

[talonchandler]

#97 isn't necessary for recOrder in the near term. It will be convenient to have eventually for the corner case of analyzing 2D+3D MM acquisitions, but I now have workarounds for data acquired on the spindle project.

I don't anticipate 2D+3D acquisitions from the mantis project, although @ieivanov can chime in.

@ziw-liu and I agreed to go ahead with tagging main with v0.1.0dev0, and I've started some very early work on recOrder in this draft PR. These tags can be moved/renamed, so this work is still low stakes.

[ieivanov]

#97 will not be an issue for the mantis, git tags seem like a good idea to me too

[ziw-liu]

Unfortunately, PyPI does not allow packages to depend on a URL. PEP440 states:

Public index servers SHOULD NOT allow the use of direct references in uploaded distributions. Direct references are intended as a tool for software integrators rather than publishers.

For example, when I tried to upload a recOrder-napari pre-release to PyPI depending on the v0.1.0dev1 git tag, I got this error:

ERROR    HTTPError: 400 Bad Request from https://upload.pypi.org/legacy/
         Invalid value for requires_dist. Error: Can't have direct dependency: 'iohub @
         git+https://github.com/czbiohub/[email protected]'

In my opinion a 0.1.0dev* release on PyPI is still low-stake and does not promise anything about API stability, but will allow other PyPI projects we develop to use iohub.

[ziw-liu]

Now on TestPyPI. Seems fine. Cannot actually install because dependencies are not available from the test index. But the page looks reasonable enough.

[ziw-liu]

Pushed v0.1.0dev3 to PyPI. Thanks for everyone's help!