Add support for pinned bundle versions

[dunkmann00]

This adds support for pinning the version of any bundle used. If a pinned version is found in the pyproject.toml that will be used as the bundle version when running circup commands. Otherwise, the behavior remains unchanged and circup will use the latest available bundle version.

I've not added any tests yet, happy to do so if this addition is welcome.

Closes #258

[dunkmann00]

Fixed up the failing tests and added tests for this feature.

[FoamyGuy]

I tested this today and can confirm it seems to work as intended.

I think it's a little awkward to have to configure this with a pyproject.toml file, other options for circup like the path to the target device, and host/port for web workflow are currently accepted only as command line arguments.

I am in favor of all configurations for the utility supporting the same input mechanism rather than the user needing to use different things to input different configuration options. So I would lean towards changing this from using pyproject.toml to instead use a command line argument like --pin-bundle-tag or similar, or perhaps even a new sub-command like circup pin-bundle which would then exist alongside others like circup bundle-add, and circup bundle-remove.

That being said I can see convenience value in taking values from pyproject.toml as well. I would not be opposed if we wanted to implement that as one possible way to input the circup configurations, if we make it so the rest of the configurations beyond just the pinned bundle version can be done that way too so that the user doesn't have to keep track of which configs must be done in which different way.

[FoamyGuy]

This extends the search all the way back to the root of the system (on linux), I think that is a bit of an extreme length to go looking for a pyproject.toml file.

I would be in favor of limiting this to either the direct parent only, or maybe 1 level above that if you feel its warranted.

If very deeply nested project structures beyond that are a concern perhaps we could also just allow the pyproject.toml file path to be specified by the user with a command line argument so that we aren't searching around too many places for it.

[dunkmann00]

If you feel strongly about it, sure, I can make it so that it only checks the current directory and the parent. I went with this as it is how poetry and uv look for a pyproject.toml file. In fact, I just used poetry's implementation.

[FoamyGuy]

I think current directory and parent would be good. I think the further away it gets upward from the current directory it increases the chances of finding an unrelated pyproject.toml file and then having unexpected behavior that is a little tricky to trace down.

[dunkmann00]

the user doesn't have to keep track of which configs must be done in which different way.

This seems fair, and I agree the goal here should be to prioritize simplicity.

I think we might be thinking of what is the purpose of this feature slightly differently. I tried to describe the reason I felt "pinning" would be useful in #258. I realize I forgot to link that issue to this PR so that's my fault. But to summarize, I'm thinking of pinning as the same thing as the requirements.txt, a way to make sure the same versions of libraries are installed each time I run install.

Currently, if you run circup install -r requirements.txt you can get completely different versions of all the libraries you have in requirements.txt. circup will try to download the latest bundle every time. Any changes to libraries within it could cause your code to break. But at the moment, there is nothing you can do about that.

Using pyproject.toml as a place to store this information just seemed like the most natural place for a python project to store information about the project to me. I also don't know how a pin-bundle command would work because I don't think pinning a bundle globally is a good idea, each project might want to use different versions.

I don't know how most people use circup. Maybe they just manually circup install 'library' when they need to and just do that over and over again as is required. I was trying to make the workflow of circup install -r requirements.txt more robust by ensuring you actually get the same exact libraries installed each time. This would make it easier to install a project on multiple boards and know it should work. I would think this might also help if tutorials provide projects that can be cloned/downloaded for the same reason.

As another point, I don't actually feel pyproject.toml is awkward to work with since its now quite ubiquitous with any python project you're going to make or use. I do think it could make sense to add a command that would persist this pinned bundle info, analogous to using freeze for the requirements.txt (or as I think about it, maybe adding a flag to freeze to also freeze the bundle versions?). In the current implementation it would add it to the pyproject.toml project for you. But if we don't want to use pyproject.toml it could add it to some other configuration file I guess. I don't like having to manually enter it as a command line argument because that now requires you to manually store what bundle versions you want to use somewhere and enter it every time (and when sharing the project you have to communicate this information separately).

But I'd love to know what you think @FoamyGuy and how you see "pinning" getting used.

[FoamyGuy]

Okay, I've read up on the issue and your summary above, that does all make good sense to me as well. I do think I fall more into the "manually circup install 'library'..." category with my usage, so I was indeed thinking about it a bit differently from how you described it here and the issue. I don't use circup with requirements.txt much.

I also don't know how a pin-bundle command would work because I don't think pinning a bundle globally is a good idea, each project might want to use different versions.

Yeah that is a good point, pin-bundle won't work out well.

I don't like having to manually enter it as a command line argument because that now requires you to manually store what bundle versions you want to use somewhere and enter it every time (and when sharing the project you have to communicate this information separately).

That is fair, and pyproject.toml does neatly solve all of those problems so I can see it's appeal.

How about keeping the existing behavior but adding --pin-bundle-tag (or maybe even just --bundle-tag) as an optional argument for install subcommand and have it be possible to specify the bundle tag to use that way as an alternative to pyproject.toml. That way this feature can be used more easily in a situation that isn't associated with a specific project or pyproject.toml file.

As an aside to the bundle tags but still to the core of this issue: It should be possible to make circup able to install individual libraries at specific versions also by downloading .py or .mpy files from the release assets of the library repo on github. Then it could behave more like pip with regards to those versions being specified in requirements.txt. Though I can definitely see that the bundle tag solution is less complex, and convenient for it's own sake since we already distribute it that way and people often use all the libraries from a given bundle version that they happen to get when they downloaded it.

[dunkmann00]

How about keeping the existing behavior but adding --pin-bundle-tag (or maybe even just --bundle-tag) as an optional argument for install

I like that idea 👍

As an aside to the bundle tags but still to the core of this issue: It should be possible to make circup able to install individual libraries at specific versions also by downloading .py or .mpy files from the release assets of the library repo on github.

That's interesting. I think that would be really cool, it would completely remove the need for bundle pinning, and maybe for using bundles altogether in circup. But I assumed this wasn't something you/other maintainers would be interested in. Like you said, at least for now I think pinning a bundle is less complex and convenient. But if this is something that you'd consider I would be interested in trying to get that working...in a future PR of course, I think that one might take a bit 😅

[dunkmann00]

Okay, added a --bundle-version option for the command line. I should note it got added to the main group, so it would be used circup --bundle-version "adafruit/Adafruit_CircuitPython_Bundle=20251024" install .... This is because there are actually a bunch of commands that would be affected by pinning. You can see the changes in the commit to see if this is okay with you.

I also added a bundle-freeze command. I thought it could be useful to generate the output needed to copy paste into a pyproject.toml to get the pinning. I think it worked out okay, but let me know what you think about that as well!

[FoamyGuy]

This looks goo to me. Thanks for working on this @dunkmann00! I think this is great functionality to have.

I tested the --bundle-version option successfully. Good call on moving that to the main options instead of inside install.

I did limit the pyproject.toml search to cwd and parent and tested to ensure both places still correctly find the pinned version.

[dunkmann00]

Thanks! Glad to get this in so quickly!

I did limit the pyproject.toml search to cwd and parent and tested to ensure both places still correctly find the pinned version.

Okay, that sounds fine! I would like to note just in case this happens to come up in the future that I did check other python tools and they all checked back up to the root (poetry, uv, ruff, pytest, pylint, black).