-
Notifications
You must be signed in to change notification settings - Fork 23
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Candidates for easy separation from standard library #473
Comments
To follow-up on this - a stronger analysis of relocation impact would include usage by other nimble packages, not simply usage by the compiler package internally..Something like https://forum.nim-lang.org/t/9323 but at the module level { and where a simple table is all that is needed, not a maybe large image }. It is more work to collect this data since usage is not declared in a Even this stronger analysis will not cover "dark code" not in the nimble directory. Problem context can guide that sometimes. As 3 examples, A) It is probably the unwritten plan already, but there should probably be at least a major release cycle like nim-1.8 where the modules are first deprecated only to be moved the next cycle. Then whatever falls between the cracks of the ahead-of-time analysis can perhaps be un-deprecated. The gap between these deprecate-move releases in real-time should hopefully include a lot of word-of-mouth chatter/Forum posts/etc. among users to test any packages they use, and hopefully last at least several months so that more casual users who are also depended upon somehow can respond somehow. Maybe all of this is super-obvious/implicit in The Plan, but I thought I should say something. |
I'm not sure if this is going to be done either, but if it had to be, it would be based off the 1.6 branch with commits from devel cherrypicked in. I only know that it's been established that devel is the 2.0 branch. I was actually going to mention that asyncftpclient has basically no uses on github search (other than Nim repo clones), but it would have been misleading because I've used asyncftpclient in a private repo myself. Though I ended up just copying asyncftpclient into my codebase and making edits on it because it was broken. These modules definitely aren't unused, them being used is just 1 factor in moving them out of the stdlib. It is still good to get awareness/feedback though. I don't think a Nim 1.8 necessarily will help with this as warnings are often ignored, and it might not be maintained too long until 2.0 or just not enough people will use it. A 2.0 preview maybe, but some breaking changes are hard to warn for, and it's easier to put them in 2.0 directly. A forum post asking what people think about modules moving out might be the best thing to do, I just personally don't involve myself with the forum that much. I am fine with the idea of putting warnings in 2.0 and removing things in 2.x instead, especially for the standard library. I believe this is the strategy for |
"Often ignored" != "always ignored" and it seems needlessly harsh to punish whatever subset of users does pay attention to warnings with an abrupt change when Nim has nice mechanisms for SW evolution. So, the 2.0 deprecation/2.x removal sounds better to me than just direct move at 2.0, BUT I believe some people are also pretty attached to SemVer notions (as in 2.x changing from deprecation -> hard errors == big breaking change in a minor rev == bad). So, beats me what the best plan is. I would have said 1.8 deprecated for N >= 3 months except this seems Off Plan from your account, and maybe I am too conservative. Or maybe others were too optimistic in branding devel 2.0 given the status of change-work? @Araq had mentioned "once in a decade" in The Roadmap thread. Anyway, "how used" more broadly than internal does still seem like useful metadata in decision making, even if it is only 1 factor { and someone may have dep tools like I mentioned :-) }. |
It's fine to leave versions of them in the standard library for any amount of time then. The key thing is moving them to nim-lang repos. We can do |
Isn't fancy like that but is what I have for now all imports of nimble packages in packages.json. |
While a fine start and certainly better than nothing, this looks like simple You might be able to get something a bit stronger by just capturing the "CC: ....foo.nim" logging output of compiles and "uniquifying them" within packages/tests for such/etc. While it might be high effort to even get so much old code to compile, we also probably mostly care only about packages that lib/tool consumers might care about. This happens to translate (roughly & conveniently) to "compiling with low effort". In a "not really compiling all the way" sense, |
Ordered usage stats from Aug 2021 #398 (comment) |
My take is that any library that needs any of:
should be split off. They don't necessarily need to be abandoned, but they should be split off into one or more repositories that say "this module is under semantic-versioned development". |
SQL stuff seems out of place for stdlib also: |
I have made a repository for smtp as a start/example, preserving commit history using this. If this is fine to move to nim-lang, I can do this for other modules as well. Edit: parsesql already exists as a nimble package, which appears to be a fork of the stdlib version (without commit history) with new additions. I should be able to add the stdlib commits as well as @bung87's, or we can just use the existing repository without the stdlib commits. bung87's version is also old, there have been some changes since in the stdlib that we would have to add as well with this method. |
I have updated the smtp port a bit to move the "test" out from the module and try to compile it for CI. If this won't be moved to nim-lang anytime soon then I will make the PR to publish it as a package and the link can be changed later. I will likely make a repo for The impure packages are difficult: Nothing has to be done to these modules in the standard library yet. I intend to submit the packages to the registry for people to use though and uphold their maintenance for the time being (port commits from stdlib, accept bugfix/improvement PRs, add collaborators). |
The ideal organize db modules is put them into one nimble package with |
Added |
Rewrote the RFC to be more readable, the move-out demonstrations are now at https://github.com/nim-old-stdlib where there is also a guide on how to do it (edit: moved to https://gist.github.com/metagn/f808a4b398cece6b8361d57f06efd259). Anyone who wants to keep the repos there updated or make their own repos or transfer the repos is free to ask for permissions in the organization. |
It seems that if packages have a vibrant nimble alternative already (for example because they were forked due to lack of maintenance in stdlib), they would make good candidates for removal as well, else they only create confusion and poor UX for newcomers to Nim (that inevitably will have to go through the same disappointment as the ones doing the forking) |
|
Some updates not covered above: The There also might be a new candidate,
At the very least not both of Edit: Seems to have been moved out |
Because the RFC is long, here is the current progress first:
Current progress
asyncftpclient
to nimble packages Nim#20952)smtp
to nimble packages Nim#20953)RFC
From #437:
I am in no position of authority in this community but I thought I would make this RFC to expand on this point as some clarity would be nice.
Why?
There are benefits to moving out certain modules from the standard library, not just for the sake of keeping the core of Nim minimal, but also for the development of the modules themselves. Some of these benefits are:
There are many obvious exceptions, such as when they are closely tied to Nim's language development or when they are too popular to feasibly move out.
How?
There are many ways to go about this but only people with authority can make the decisions. The following are general guidelines:
nim-lang
organization, or seem "official" in some wayBackwards compatibility
Assuming the worst case (these modules are removed and they are not distributed with Nim), all most people would have to do is add these modules to their
.nimble
file/manually runnimble install
. While this is basically nothing for most people there might still be call for including them in some kind of distribution, see #476 for more recent discussion about this.Which modules?
This is the main focus of this RFC.
Some history
There is already a fair amount of past discussion of whether or not to move out certain modules, for example
logging
in #146.A "graveyard" repository was created in the past for some functioning modules in the standard library that were not expected to receive maintenance to be moved to.
cgi
andcookies
were considered to be put in this repository, but this was rejected, as these modules are "used and reasonably free of issues [and] don't cost us much maintenance".oids
andsmtp
were removed at first, but then put back in the standard library, for similar reasons.In recent times,
smtp
has had a PR merged in Nov 2021 that seems like it was made just to make smtp work at all for some servers. However this PR is not on a stable release yet as of Nov 2022, nearly a year since, as a specific effort was not made to backport it.Recently bigints, previously a nimble package, as a compromise to being included in the standard library, has been moved to the
nim-lang
organization and received a slight uptick in productivity. But it seems the only person who can merge PRs is narimiran and activity has staled after a while despite there seemingly being people interested in contributing.Metrics
Current independent stdlib modules
The following modules (among others that are too new (like
std/with
) or just happen to be unused despite being important (likevolatile
)) in the standard library have no (or close to no) dependents in the Nim repo itself and have self-contained tests:tests/gc/growobjcrash
which might be disabled and only usescgi.decodeData
)compiler/sighashes
and a single unrelated test,tests/typerel/tnoargopenarray
(and a Nim in Action test)tests/stdlib/tdb
nimtracker
?tests/async/tlambda
)tests/stylecheck/taccept
)nimgrep
, in teststests/misc/tinvalidnewseq
,tests/manyloc/keineschweine
, Nim in Actionconcurrency_regex.nim
)tests/stdlib/trandom
)mersenne(moved to graveyard)This gives a good blueprint for which modules could be easily moved out.
Usage stats
The following modules that were listed above are the most common based on Nimble usage stats (in parentheses are independent modules that are unlisted above for being too new etc):
None of these strike me as too popular to move out.
Another metric that could be added here is development activity i.e. commits that aren't simple universal refactorings, but this would be somewhat difficult.
Subjective conclusion
This is my assessment of the modules at hand:
Small modules (<500 lines including docs and whitespace)
These require seemingly little maintenance, and have a noticeable amount of users.
These seem to be harmless and it might be annoying to remove them, although they are essentially small Nimble packages. I think there should be no guarantee that these modules are maintained and they should only stay as utility modules. If there was a way to have these in a separate pile from the rest of the standard library, but have them easily accessible anyway, it would be nice.
Medium sized
These may be used in important applications but aren't super common. They occasionally get important patches.
These are the sweetspot when it comes to the friction of moving out IMO, with a niche userbase that probably wants the best version of these modules. For this reason I have made an example demonstration of moved out asyncftpclient and smtp.
parsecsv
was only recently made independent in the Nim repo, with the removal of the nimweb tool. It's not as big or often-patched as the rest of these modules either, and it fits with the rest of theparse
modules in the standard library, so it might have less priority to move out.parsesql
is kind of in the same boat, but it's a bigger module and more prone to needing patches. There is currently a competing version of it on Nimble that has extra features and doesn't have commit history. Completely unsure what to do with this module.Big wrappers
These are relatively commonly used, but may not be that hard to move out.
These are a bit more effort due to their size and number but moving them out would make the codebase a lot cleaner and wouldn't affect a very large number of users.
Others
Some modules I haven't mentioned yet but I think I've seen discussion about are json and http/ssl modules. These would be the biggest challenge and I'm guessing there's no plans to move them out until an entirely new design is conceived.
The text was updated successfully, but these errors were encountered: