Misleading language in image-layout.md

[runcom]

Three issues:

First:

Object names in the refs and blobs subdirectories MUST NOT include characters outside of the set of "A" to "Z", "a" to "z", "0" to "9", the hyphen -, the dot ., and the underscore _.

This sentence is wrong because I can have colons : into the refs subdirectory to say ./refs/vbatts-app:latest

Second:

Hash algorithm identifiers containing the colon : will be converted to the hyphen -. For example sha256:5b will map to the layout blobs/sha256-5b.

these two sentences don't play nice together. What are we talking about? refs naming? both? I can have ./refs/sha256:5b58496 but what does that mean that the name will map to blobs/sha256-5b58496? AFAICT the content of that refs which is a descriptor will give us the location in the blobs dirs, not the ref file name.

Third:

$ cd example.com/app/
$ find .

The above (with app/) isn't making clear that an image-layout can carry more than just one app with:

./refs/vbatts-app:latest
./refs/alpine:1.0
./refs/ubuntu:sha256-5bfdjsldf

/cc @vbatts @stevvooe @philips

[runcom]

I believe we need to split the refs and blobs naming paragraph into two pieces - one for refs and one for blobs because clearly the two have different naming restrictions.

[wking]

On Thu, Jul 21, 2016 at 06:44:19AM -0700, Antonio Murdaca wrote:

This sentence is wrong because I can have colons : into the refs
subdirectory to say ./refs/vbatts-app:latest

Colons are not portable, and we map them to hyphens 1. If folks
want more characters in the ref names, I suggest we have Unicode names
that are encoded to UTF-8 and then to filesystem-safe base64 2.

Hash algorithm identifiers containing the colon : will be converted to the hyphen -. For example sha256:5b will map to the layout blobs/sha256-5b.

these two sentences don't play nice together. What are we talking
about? refs naming? both?

This is just about the blob filename. Splitting it into its own
paragraph would help clarify that.

AFAICT the content of that refs which is a descriptor will give us
the location in the blobs dirs, not the ref file name.

This is correct.

$ cd example.com/app/
$ find .

The above (with app/) isn't making clear that an image-layout can carry more than just one app with:

./refs/vbatts-app:latest
./refs/alpine:1.0
./refs/ubuntu:sha256-5bfdjsldf

I'm fine adding different app examples here, although I'd recommend
either avoiding colons or base64-encoding the ref names.

[runcom]

 Hash algorithm identifiers containing the colon : will be converted to the hyphen -. For example sha256:5b will map to the layout blobs/sha256-5b.

these two sentences don't play nice together. What are we talking
about? refs naming? both?
This is just about the blob filename. Splitting it into its own
paragraph would help clarify that.

it's still misleading - what does that actually mean? what are we talking about? blobs naming? who does the conversion? is it really needed to say that? can't we just say blobs are named ALGO-SUM?
The second sentence is also misleading, where is the case someone come up with `sha256:5b? I guess from a manifest blob, yes, correct, but that sentence right there isn't saying it explicitly, I'm just guessing it.

Not sure about the colon story and base64 - I'd love maintainers to chime in

[wking]

On Thu, Jul 21, 2016 at 08:34:36AM -0700, Antonio Murdaca wrote:

Hash algorithm identifiers containing the colon : will be
converted to the hyphen -. For example sha256:5b will map to the
layout blobs/sha256-5b.

these two sentences don't play nice together. What are we
talking about? refs naming? both?

This is just about the blob filename. Splitting it into its own
paragraph would help clarify that.

it's still misleading - what does that actually mean? what are we
talking about? blobs naming? who does the conversion?

The CAS consumer says “get me ‘sha256:5b…’”, and the CAS engine says
“with image-layout 1.0.0, ‘sha256:5b…’ will be stored in
‘blobs/sha256-5b…’” and fetches and returns that content 1.

is it really needed to say that? can't we just say blobs are named
ALGO-SUM?

You could do that too. I don't think it really matters.

The second sentence is also misleading, where is the case someone
come up with `sha256:5b? I guess from a manifest blob, yes, correct,
but that sentence right there isn't saying it explicitly, I'm just
guessing it.

Who cares where sha256:5b… comes from? Both sentences are just “If
you have a blob who's SHA-256 hash has a hex digest 5b…, the name for
that blob is sha256:5b… and the image-layout 1.0.0 path for that blob
is blobs/sha256-5b…”. It doesn't matter if it's a manifest, or a
layer, or a PNG, or whatever.

Consumers will know what sha256:5b… should be from the context of
wherever they got the name (e.g. from a descriptor that also carries
mediaType, or from an email saying “I've just cut a new release…” or
“Here's a cute cat picture”).

[runcom]

What's your saying is ok with with as I understand most of the concept and stuff you've highlighted - still I think that's not really clear in those sentences

[wking]

On Thu, Jul 21, 2016 at 09:28:49AM -0700, Antonio Murdaca wrote:

… still I think that's not really clear in those sentences

I expect PRs are welcome ;).

[stevvooe]

Colons should not be allowed.

The example refs are completely invalid. The following:

./refs/vbatts-app:latest
./refs/alpine:1.0
./refs/ubuntu:sha256-5bfdjsldf

Should be:

./refs/latest
./refs/1.0
./refs/sha256-5bfdjsldf

Otherwise, we completely break windows compatibility, as colons are used for alternate data streams.

[runcom]

So there's no way to have multiple images (images as in Ubuntu, Alpine, Debian, Fedora) in a single image layout?!

[wking]

On Thu, Jul 21, 2016 at 03:44:51PM -0700, Stephen Day wrote:

Colons should not be allowed.
…
Otherwise, we completely break windows compatibility, as colons are
used for alternate data streams.

This particular bit (or at least, one solution to it) has been spun
off into #174.

[stevvooe]

So there's no way to have multiple images (images as in Ubuntu, Alpine, Debian, Fedora) in a single image layout?!

Rather extreme conclusion and tone. Maybe try taking a deep breath before replying? 🐹

The refs simply aren't a tool for naming. They work more like tags and are separated in a similar manner to git. refs provides root references, from which an a set of blobs can be accessed. I'm not even sure that refs will have a one to one mapping with tags (the above example was great).

The path name of a ref must be compatible with as many filesystem and archive formats as possible.

Let's look at alternative example to better understand the future role of refs (not a proposal):

refs/images -> sha256:xxxx, type=some naming thing
blobs/sha:xxxx ->
  {
    "images": [
       { # a descriptor!
         "name": "alpine:latest",
         "digest": "..."
       }
    ]
  }

This part of the specification still needs to be worked out carefully. There are several considerations:

1. Names need to be compatible across image layout, dns and other discovery systems.
2. Image Layout needs to support several images from possibly different repositories.
3. The trust model for naming must be clarified (how does one trust the names in a tar file?). File names aren't a very good authority if you can't verify the integrity of the underlying storage.

[wking]

On Thu, Jul 21, 2016 at 04:38:55PM -0700, Stephen Day wrote:

1. Image Layout needs to support several images from possibly
   different repositories.

This seems trivial, since you can drop all the blobs into the same
image-layout bucket, and then pick whatever you like as the refs
pointing into those blobs. The only possible issue I see is if you
want to specify a standard ref-name template for a particular image
name, and I think #174 covers any possible concerns there.

[wking]

On Thu, Jul 21, 2016 at 04:38:55PM -0700, Stephen Day wrote:

1. The trust model for naming must be clarified…

Spun off into #176.