-
Notifications
You must be signed in to change notification settings - Fork 188
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
New package develop mode and lock file support. #913
Conversation
7a9b4ac
to
140b858
Compare
I'm not sure how to specify the link in |
dd83f14
to
ebdb777
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Here are some minor suggestions/fixes for the readme.
(The rest will be reviewed later)
Just attempted to compile this on Nim 1.4.8 and got:
Is this in a Nim fork of yours? |
Yes. You need the changes from the |
Sorry but the pull request is not ready for merge. After introducing parallel package downloads via the
I also have to update the documentation. |
src/nimblepkg/checksums.nim
Outdated
while true: | ||
var bytesRead = readChars(file, buffer) | ||
if bytesRead == 0: break | ||
checksum.update(buffer[0..<bytesRead]) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Slow copy
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What is the alternative?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Well, I made it with the toOpenArray
procedure to avoid the copy.
@@ -349,7 +345,7 @@ proc readPackageInfo(result: var PackageInfo, nf: NimbleFile, options: Options, | |||
return | |||
|
|||
result = initPackageInfo(nf) | |||
let minimalInfo = getNameVersion(nf) | |||
result.isLink = not nf.startsWith(options.getPkgsDir) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks flaky
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Any suggestions on how to avoid it?
For reference, what we posted in Telegram. So here are our top-level asks:
These are our main tasks for you to resolve, we can change the rest of the details ourselves. On a positive note, we really like the |
There is a description of what every flag does. They are needed for manipulating the
Sync files are necessary for implementing the develop packages repositories synchronization functionality. Look at the
Zah, told me to use the
I was specifically asked by Zah to add this feature and to be enabled by default. I think that this is nice optimization because tarball downloads are faster.
Well, I think that the lock files, the new develop mode, the tarball downloads, and the parallel downloads are 4 distinct features and it was possible to implement them as 4 independent pull requests, but I was specifically asked by Zah to implement all of them with the lock files. The functionalities now are too intertwined and it will be a nightmare if I try to separate them. |
New RFC for nimble development roadmap nim-lang/RFCs#398 talks about some features like So far the feature list seems quite large - far beyond of what original specification and includes things like
|
By using the |
@bobeff I think you can use |
@dom96 Yes, I understood this but if in the future if we try to implement parallel downloads when not using a lock file it will be not possible for the recursive algorithm without |
All tuples from the same type which are found in more than one place in the code are type aliased and their usage is replaced with the corresponding alias. Related to nim-lang#127
- The package sha1 checksum is added to the `PackageInfo` object when it is being initialized either by calculating it or by getting it from the package directory name in the local package cache. - When package is being written in the local cache its sha1 checksum is added to the directory name alongside its name and version. - Reverse dependencies format is changed to contain also the package sha1 checksum. Conversion between the old and the new formats is implemented for backward compatibility. All functionality related to the reverse dependencies is fixed to work properly with the new format. - The indexing of the `nimbledata.json` fields is changed to use values from an enum for their names to avoid typing mistakes. - All unit tests are fixed to pass with the new changes and new unit tests are added where it is needed. Related to nim-lang#127
Fix `processDeps` procedure to not return a dependency multiple times when it is a common dependency of more than one package. Related to nim-lang#127
The `cd` template from `tools.nim` did not restore the previous directory in the case when return from the function is made in the body. The defer statement is used to fix this. Related to nim-lang#127
The parallel package downloads are removed, because the `asynctools` library for asynchronous processes, that is used is not production-ready. It has several hard-to-find bugs causing а crashing of Nimble. First, they must be fixed before using it. Related to nim-lang#127
I removed the parallel downloads feature from the pull request, because the |
The CI gives a |
We didn't see similar problems when cloning via |
Downloading of Nimble packages as tarballs when working with GitHub is now disabled by default. The option `--no-tarballs` for disabling it is removed and a new option `--tarballs` for explicitly enabling the feature is added instead. Related to nim-lang#127
Pass the requested branch to the `git` or `hg` command instead of the "branch" word. Related to nim-lang#127
I made the tarballs download feature disabled by default. But now a problem with calculating checksums of symlink files in the repository has appeared under Windows. I will investigate it tomorrow. |
Sometimes when downloading a package some symbolic links files cannot be open on Windows when Nimble is run as administrator. For this reason, add exception handling to the `open` procedure to avoid the crash and display a warning message that the file content will not be count in the calculation of the package's checksum. Related to nim-lang#127
https://forum.nim-lang.org/t/8404#54193 I have experienced the same message/error. |
follow up #19821 dights cannot clash with letters 'Z' and 'O' For `threading-0.2.0-288108d1dfa34d5ade5ce4d922af51909c83cebf` Before: raiseNilAccess__OOZOOZOnimbleZpkgs50Zthreading4548O50O4845505656494856d49dfa5152d53ade53ce52d575050af5349574857c5651cebfZthreadingZsmartptrs_u4 After: raiseNilAccess__OOZOOZOnimbleZpkgs2Zthreading450O2O045288108d1dfa34d5ade5ce4d922af51909c83cebfZthreadingZsmartptrs_u4 <del> nimble or something might use `git rev-parse --short HEAD` to shorten the length of package version names ref nim-lang/nimble#913 </del>
This pull request contains a new package develop mode and lock files support. It depends on this pull request in Nim. The current minimal supported version of Git is bumped to
2.22
from2019-06-07
.In order to avoid repetition see the following sections from the documentation for more details:
This pull request contains also a significant amount of code refactoring. One of the big refactoring changes is splitting of
tester.nim
in modules by suites. You could look at the very detailed commit messages for more particulars.