Use a More Correct (TM) directory for local data

[emberian]

No description provided.

[emberian]

This conforms with the XDG base directory specification. It's possible we want to do something else on OS X, which is more idiomatic?

[tomjakubowski]

(pulled from an orphaned comment)

The equivalent of $XDG_DATA_HOME on OS X is ~/Library. Not many CLI applications conform to this, though, tending to just dump stuff in dotfiles/directories directly under ~.

[o11c]

For Windows it would be CSIDL_LOCAL_APPDATA

[alexcrichton]

Can you elaborate on the reasons for this change? Other package managers like npm and bundler have files in my home directory, so there's certainly precedent for this.

[emberian]

On Linux at least, storing data directly in the home directory and not in an XDG location is considered poor form. There is certainly precedent for it, but I'd argue that every application that does it is wrong, not just according to the spec, but using the basic namespacing argument.

[o11c]

@alexcrichton The existence of badly-behaved programs is no justification to do the same. It is incorrect for programs to assume that they are important enough for a new top-level directory in HOME. That would be like every program installing itself into /rust, /cargo, etc.

XDG specifies the semantics of (by default) ~/.local/share, ~/.config/, and ~/.cache, which is quite a useful improvement over figuring out "how important is this junk".

Other subdirectories such as ~/.local/lib/ and ~/.local/bin/ are not part of XDG yet (probably just because no one has submitted a patch), but it is generally assumed that ~/.local/ is a sort of parallel to the FHS /usr/local/ hierarchy; in particular, it is often expected that ./configure --prefix=$HOME/.local will DTRT, though note that ~/.config is sort of similar to /etc/ and ~/.local/share is writable (taking on some of the characteristics of /var/)

The only actual standard that I'm aware of is PEP 370, which may be useful to determine the best locations on non-Linux platforms.

Given that there are some standards, it is better to follow them than to make something up from scratch, and additionally work on finishing up. (Particularly, some of the open questions from the last Fedora flamewar on the subject would be resolved by making XDG_BIN_HOME a thing, so that it can be overridden instead of being hard-coded)

[o11c]

Related: rust-lang/rust#12725 and somewhat rust-lang/rust#5219 and rust-lang/rust#6425

[emberian]

Updated to handle Windows and OSX correctly, with links to relevant documentation.

[emberian]

The OSX travis builder seems quite broken -- it can't find rustc!

[jhasse]

I think it should be XDG_CONFIG_HOME instead of XDG_DATA_HOME. If I understood it correctly XDG_DATA_HOME represents /usr/local/share which contains read-only data a program might need to run. As cargo checks out git repositories at runtime I would classify those files as "config" data suited for ~/.config/cargo.

[o11c]

XDG_DATA_HOME represents /usr/local/share which contains read-only data

Close, but wrong.

From the spec: There is a single base directory relative to which user-specific data files should be written. This directory is defined by the environment variable $XDG_DATA_HOME.

(Note that in order to implement cargo install it will be necessary to exceed the spec by writing to ~/.local/lib and ~/.local/bin, but since those are a de facto standard, there shouldn't be a problem).

I've quickly written out a rust-xdg library, but I'm thinking about 1. renaming it to reflect its hopefully-cross-platform nature soon, 2. changing the API so that programs can only access one application-specific subdirectory instead of just anything.

[o11c]

@cmr Note that you are required to ignore the environment variable if it's not an absolute path, please don't join it to home.

[errordeveloper]

Looks like this is some sort global cache of git repositories? Having a global one is totally an optimisation, it should be per-project for the start, which is the safest way.

I'd vote on closing this, it's making this feature more complicated, and it wasn't is not quite right already. Instead there could be just a simple environment variable (or a config option or flag) that user can set, but it would default to the current working directory.

I also think this PR was a big time bikeshed, sorry @cmr.

[emberian]

@errordeveloper it is not any less safe to have a per-project "cache", and this is already how cargo works. It does not affect the feature in any way besides storing everything in a more correct location than ~/.cargo

[errordeveloper]

Are those by any chance bare clones which are then used as reference for per-project clones? (That would be my optimisation)

[jhasse]

@o11c How does the spec say that I'm wrong? It doesn't say what is meant by "user-specific data files". Therefore I assume the same rules as for /usr/share and /usr/local/share (since the only thing different is the prefix, /usr, /usr/local and ~/.local).

The rules for /usr/share are read-only data files: http://www.pathname.com/fhs/pub/fhs-2.3.html

[o11c]

@jhasse because despitet the similarities FHS does not specify ~/.local/, XDG does. And XDG says that ~/.local/share is for writable data. (Usually it's read-mostly, but write if you're going to read it later).

That said, since this information is nonessential and could be recovered, there's also a good argument for XDG_CACHE_HOME.

[jhasse]

Where does XDG say that local/share is for writable (during runtime) data?

[lucy]

The XDG basedir spec doesn't say anything about being analogous to FHS, and XDG_DATA_HOME is already widely used for runtime writable data.

[jhasse]

Okay I give in. But IMHO the spec should be clearer about this and also it's kind of confusing (to me at least) that this has a different role as /usr/share.

[SimonSapin]

Since git repositories and checkouts are "non-essential" (they can be downloaded again and recreated), I thought they’d go into $XDG_CACHE_HOME rather than $XDG_DATA_HOME.

[alexchandel]

👍 👍 👍 Why hasn't this landed yet? There's no reason whatsoever for Cargo to be dumping garbage into my home directory. This isn't 1970, Cargo shouldn't imitate the few hardcoded disappointments that do (.ssh, .bashrc, etc). If ~/.config/git is enough for git, then ~/.cache/cargo is enough for Cargo.

[alexchandel]

Moreover for the OS X case, ~/Library/cargo is NOT an appropriate place for cargo's files.

Either ~/.cache/cargo or ~/Library/io.crates.cargo is appropriate for cache files, and either ~/.local/share/cargo or ~/Library/Application Data/io.crates.cargo is appropriate for persistent data files. However, the ~/Library locations should be reserved for Cocoa .apps, so Cargo's best option on OS X is ~/.cache/cargo.

[tomjakubowski]

However, the ~/Library locations should be reserved for Cocoa .apps, so Cargo's best option on OS X is ~/.cache/cargo.

@alexchandel do you have a source for that? I don't have my Mac handy but IIRC homebrew and some other CLI applications use ~/Library.

[alexcrichton]

If we would like to move forward with this, I would be in favor of a less radical approach than one taken here. It is easier to convey "look in ~/.cargo for information" than "look up your local XDG spec to see where you're supposed to look", and I also believe that generally most users don't necessarily mind if the spec is "violated".

Would the following be palatable?

• Configuration is queried for in both the .cargo/config location
• Configuration is also queried (merged) fro the xdg location
• Add a configuration key for ~/.cargo/git (to move to an "appropriate location").
• Add configuration keys for other such locations cargo requires so they can be moved.

This would allow those that would like to respect the XDG spec to have their config file in the right location with configuration to keep it all in the right location while also allowing other users to have an easily discoverable location for this information.

[SimonSapin]

Speaking only for myself: I’m mildly annoyed that Cargo puts temporary files at a non-standard location, but not enough to bother writing a configuration file and maintaining it when Cargo evolves.

Also, I don’t understand why having a configuration key for where to put things is preferable to just looking up an environment variable.

Users don’t need to look at the XDG spec if they haven’t overridden the environment variables themselves. It’s not that complicated: Configuration is in ~/.config/<program name>/, ~/.cache can safely be nuked (or just excluded in backups).

And it’s really not radical. Lots of programs already use this convention: ls -1 ~/.config|wc -l is 52 on my system.

[alexcrichton]

I'm under the impression that if I'm anywhere but linux then the configuration is not in ~/.config, so I don't believe that it's "just that simple."

I would also be ok with environment variables, but can you point to any other package managers that use this sort of scheme? All package managers I know of store data in ~/.<name>.

[SimonSapin]

I don’t know what the conventions are on other systems, I’m only stating what they are on Linux.

I don’t see why package managers would be special, compared to other programs, regarding where to have configuration files and cache files.

[tomjakubowski]

~/.cargo/config should also be configurable at $XDG_CONFIG_HOME/cargo/config, as Git does with ~/.gitconfig.

[Earnestly]

Along with git, other low-level projects such as JACK and util-linux also follow the XDG Base Directory Specification.

Other Operating Systems also have equivalents such as Application Data and Local Application Data which has existed since Windows 98. OSX provides HOME/Library (barring /Library and /System/Library which are OSX-only).

This problem has been so prevalent on places like Windows and OSX that developer guides where written such as "Don't Pollute User Space" from Apple (Atwood blog about it) to people such as Raymond Chen and his Stop the madness: Subdirectories of My Documents article.

Unlike Windows and OSX, Linux has not had a simple guide for developers to use which gives control back to the users to define the hierarchical structure in their home directory until the XDG Base Directory Specification.

Before this we (Linux users) relied on ad-hoc environment variables that some programs exposed such as GNUPGHOME or SLRNHOME with some people going so far as you create LD_PRELOAD hacks such as libetc and even FUSE filesystems such as rewritefs in a vein attempt to combat the ever increasing complexity of HOME.

Not to mention that "hiding" dotfiles was a originally a bug which has since become a regretable feature (in my opinion).

Does cargo really think HOME/.cargo is acceptable these days?

[errordeveloper]

@tomjakubowski why don't you just symlink it for now... like in in the old days ;) It will take people here a while to decide, as it seems.

[mcpherrinm]

Cargo checking out git repos into .cargo is a pretty painful: I have small & slow NFS homedirs on several machines I use, and not respecting $XDG_CACHE_DIR and friends means I have to manually ln -s .cargo $XDG_CACHE_DIR/cargo

While this is not the most typical configuration, I really hope we can conform to the XDG spec, at least on Linux. My ~/.cargo is currently 1.2GB, which is more than double the size of my homedir.

[o11c]

Can someone please make a PR on https://github.com/o11c/rust-xdg to use the proper directories on Windows?

I'd also like to add support for Documents/Videos/Pictures/Music eventually, and also give it a nicer API, but it's certainly functional already.

[Earnestly]

@o11c That is quite a nice implementation, I'm glad you actually check for absolute paths as well! (Many seem to miss that.)

[o11c]

@Earnestly it's amazing how much easier it is to follow the spec when you sit down and read the spec.

[gyscos]

What's the current status regarding XDG-directories support?

Issue #1148 is fixed, and if I set $CARGO_HOME to $HOME/.config/cargo, it is correctly used without creating a .cargo inside. But when $CARGO_HOME is not set, we don't seem to use the XDG directories by default; we don't even seem to look at it at all.

[SimonSapin]

What's the current status regarding XDG-directories support?

The status is that it’s not supported, as you found out. This PR being shown as "merged" is a bug.

[Earnestly]

Is there a new bug report about getting this supported? Snowflake environment
variables are nice and all but when you have 40+, not so much.

[SimonSapin]

@Earnestly I could not find an open issue. Please feel free to file one.