bikeshed all the things

[StefanKarpinski]

File Names

• dependencies.toml => deps.toml or Deps.toml to match [deps] sections
• compatibility.toml => compat.toml or Compat.toml to match the Compat.toml file we'll have soon in project directories
• package.toml seems fine but if we want to capitalize then Package.toml
• versions.toml seems fine but if we want to capitalize then Versions.toml

Capitalization

Since we have Project.toml and Manifest.toml we'll want the compatibility file to be called Compat.toml. Do we want the file in registries to match and be called Compat.toml – note that it does not have the same format as the Compat.toml in a project directory, so maybe these should have different names entirely or maybe the capitalization difference is ok. If we do want to call the file in registries Compat.toml then I suppose we should capitalize the other ones as well, so Deps.toml, Compat.toml, Package.toml and Versions.toml. Perhaps capitalizing them all is better since it feels more consistent with the files in project directories. But then Deps.toml doesn't match [deps]. Perhaps that's fine or maybe we want to capitalize the section name as well?

TOML Key Hierarchy

Currently we have git-tree-sha1 and that's it. However, we'll soon add archive / archive-url and archive-sha256. We will, in the future want to have other ways of identifying source trees, including git tree hashes using whatever cryptographically secure hashing algorithm SHA-1 is replaced with and potentially other non-git source tree hashing algorithms (examples are surprisingly hard to find). So we may want to have a bit more structure to these keys. Since TOML added support for inline dotted keys, there's no syntactic cost to creating a semantic hierarchy for these keys. So, for example, we could have this kind of scheme:

tree.git.sha1 = "6876ad48d788265a2af7fc51f6aaee750f0e0b6d"
tree.git.sha224 = "164a9e5878ad31fff16ba79bea852fdf76995db986febcf38c560a35"
tree.mercurial.sha1 = "416c0aef5a427aebfdb05b84408e767033fb374a"

archive.url = "https://github.com/JuliaData/DataFrames.jl/archive/v0.11.5.tar.gz"
archive.sha1 = "8a41f626b1df49ee493669f0f2dd98a007352a46"
archive.sha256 = "37adb80e5929b1babf9db7e6f6374dee0f298a2dc40eaa03d194e1e65b94d890"

This way all the stuff that describes the source tree is under the tree key and all the stuff describing the archive – where to get it, what hashes the archive file should have before you try to unpack it – are under the archive key.

[StefanKarpinski]

Here's what my gut is telling me about names:

Files in projects

• Project.toml or JuliaProject.toml – project metadata and top-level dependency UUIDs
• Manifest.toml or JuliaManifest.toml – full dependency graph with versions (optional)
• Compat.toml or JuliaCompat.toml – compatibility with other packages (optional, TBD)

Files in registries

• ../Registries.toml – a list of registries in priority order (TBD)
• Registry.toml – description about of the registry itself, what packages it contains and where
• A/ACME/Package.toml – metadata about the ACME package
• A/ACME/Versions.toml – what tagged versions exist and identifying information
• A/ACME/Deps.toml – top-level dependencies of across versions, what UUID each name refers to
• A/ACME/Compat.toml – what versions of other packages various versions are compatible with

TOML Key Hierarchies

I think we should do this with dotted keys as described above.

Sections

Stick with lowercase [deps] the way we have now. I don't think the difference between capitalization for a section versus a file is a big deal.

Potential format for Registries.toml

registries = [
  "Personal",
  "OrgPrivate",
  "OrgPublic",
  "Curated",
  "Uncurated",
]

The earlier in the list a registry appears the higher precedence it take, which only affects the meaning of a version name when there is a disagreement between the different registries. We may want to do something about combining or overriding dependencies and compatibility as well.

[StefanKarpinski]

Depot Layout

Another thing to bikeshed. Currently we have:

• environments – named global environments
• registries – where all the registries visible to this repo go
• packages – where all the installed versions of packages go
• upstream – where clones of upstream repositories go
  • maybe call this clones instead?
• logs – where logs of various activities go (package manager and otherwise)
  • it has been proposed that ~/.julia_history be moved to ~/.julia/logs/repl_history/YYYY/MM.jl and shared by month for easier management.
  • currently we have ~/.julia/logs/usage.toml to record usages of manifest files; since we want to have logs of other things in this directory, perhaps it should be called manifest_usage.toml instead; this should probably be sharded in the same way as history files if we do that so it would be ~/.julia/logs/manifest_usage/YYYY/MM.toml.

Should these be uppercase? Other file names seem to be going that way but I like them lowercase, perhaps because they're generic locations for collections of things.

We will need a few more of these directories in the future:

• libraries – this would be where BinDeps2 downloaded libraries go
• compiled – this is currently called libs/v#.# but that needs to change
  • e.g. ~/.julia/compiled/v1.3/ACME/wqGL.ji
• config – this would be where various configuration files go
  • e.g. ~/.julia/config/startup.jl, the proposed new location of ~/.juliarc.jl

[JeffreySarnoff]

re: FileName.toml vs JuliaFileName.toml
these files reside in some superstructure that identifies Julia, so there is no need to restate the "Julia" affinity imo

[JeffreySarnoff]

re: CapitalizedNames vs. lowercasednames
All the structure-participating|elucidating filenames that have the extension "toml" are better Capitalized.toml as the general public will be using myconfig.toml styled names.

[JeffreySarnoff]

etc:
Compat is clear enough. What are Deps, things that must exist, for direct and indirect required scaffolding? Dependencies or Dependents?

[StefanKarpinski]

re: FileName.toml vs JuliaFileName.toml
these files reside in some superstructure that identifies Julia, so there is no need to restate the "Julia" affinity imo

The two names exist because one may want to have a single repo which contains both Julia code and some other code and Project.toml presumes to address the whole thing. For Julia packages of course, that will not be the case. Of course, there's a case to be made that we could wait and see if there's actually any demand for this.

re: CapitalizedNames vs. lowercasednames
All the structure-participating|elucidating filenames that have the extension "toml" are better Capitalized.toml as the general public will be using myconfig.toml styled names.

👍

Compat is clear enough. What are Deps, things that must exist, for direct and indirect required scaffolding? Dependencies or Dependents?

The items in the Deps.toml file and [deps] section don't actually need to exist, they are just all of the names that the project knows about as direct dependencies. I.e. these are the names X which, if the code contains import X, it will have some idea of what to do. Currently there is no way to declare any of these to be optional or platform-specific or test-specific, but there will be, at which point calling them dependencies maybe be a bit sloppy. Yet people do seem to use the terms "optional dependency" and "platform dependency" and "test dependency" so it still seems apt. In a sense they are the things upon which the contained code may depend, not those upon which it definitely does.

[JeffreySarnoff]

They are contingent dependencies.
I call the roles that interdepending subsets bring forward "Modalities".

I very much like where this is headed.

[StefanKarpinski]

I decided to go ahead with all of this except for the tree.git.sha1 part – having a deep hierarchy there was just too annoying, and it required making changes to the TOML parser. If we want to, in the future we can migrate this name by supporting both for a while but for now it's just too fiddly.

[JeffreySarnoff]

🆗

[KristofferC]

Anything left to do here?

[StefanKarpinski]

Nah, seems good now.

[StefanKarpinski]

Actually, there is one thing to do: make the [compatibility] section match the de facto decision that was made here due to nearly complete lack of feedback. The official name of the section is [compat] because no one ever said anything to the contrary.