Maps with no keys #569

gkellogg · 2018-01-17T02:02:11Z

Allows indexing on @none, which does not result in adding the index to the indexed object when expanding.
Allows compaction to use @language, @index, @id, and @type even if there is no corresponding member.

… containers. Changes behavior for id maps to use `@none` instead of a blank node identifier. Fixes #480

…action to use concatenated/sorted values from `@container`.

davidlehn · 2018-01-17T22:59:40Z

spec/latest/json-ld-api/index.html

+                <li>Append the values <code>@index</code> and <code>@index@set</code>
+                  to <em>containers</em>.</li>
+              </ol>
+            </li>
              append <code>@graph</code>, <code>@id</code>, <code>@index</code> and <code>@set</code> to <em>containers</em>.</li>


Was this line supposed to be removed due to additions above?

davidlehn · 2018-01-17T23:02:48Z

spec/latest/json-ld-api/index.html

-                  is <code>json-ld-1.1</code>, append <code>@id</code>, <code>@type</code>,
-                  and <code>@index</code> to <em>containers</em>.</li>
+                  <span class="changed">and append <code>@id</code>, <code>@id@set</code>,
+                    <code>@type</code>, and <code>@type@set</code></span>,


Should this be ordered as @set@type? (Also, does that appear anywhere else in doc?)

Ah, yes, lexicographically.

gkellogg · 2018-01-17T23:25:14Z

Note that that IRI Compaction introduces a number of new things to add to @containers, due to the various permutations, and change to allow things within an index key to be indexed using @none.

davidlehn · 2018-01-18T00:12:07Z

spec/latest/json-ld/index.html

+
+  <p class="changed">The special index <code>@none</code> is used for indexing
+    data which does not have an associated index, which is useful to maintain
+    a normalized represeiontation.</p>


davidlehn · 2018-01-18T00:14:43Z

spec/latest/json-ld/index.html

+
+  <p class="changed">The special index <code>@none</code> is used for indexing
+    data which does not have a language, which is useful to maintain
+    a normalized represeiontation.</p>


davidlehn · 2018-01-18T00:14:54Z

spec/latest/json-ld/index.html

+
+  <p class="changed">The special index <code>@none</code> is used for indexing
+    <a>node objects</a> which do not have an <code>@id</code>, which is useful to maintain
+    a normalized represeiontation. The <code>@none</code> index may also be


davidlehn · 2018-01-18T00:15:04Z

spec/latest/json-ld/index.html

+
+  <p>As with <a>id maps</a>, the special index <code>@none</code> is used for indexing
+    <a>named graphs</a> which do not have an <code>@id</code>, which is useful to maintain
+    a normalized represeiontation. The <code>@none</code> index may also be


davidlehn · 2018-01-18T00:15:13Z

spec/latest/json-ld/index.html

@@ -3517,6 +3674,48 @@ <h2>Node Type Indexing</h2>
  -->
  </pre>

+  <p class="changed">The special index <code>@none</code> is used for indexing
+    <a>node objects</a> which do not have an <code>@type</code>, which is useful to maintain
+    a normalized represeiontation. The <code>@none</code> index may also be


davidlehn · 2018-01-18T00:21:12Z

spec/latest/json-ld/index.html

+  <p>As with <a>id maps</a>, the special index <code>@none</code> is used for indexing
+    <a>named graphs</a> which do not have an <code>@id</code>, which is useful to maintain
+    a normalized represeiontation. The <code>@none</code> index may also be
+    a term which expands to <code>@none</code>, such as the term <em>none</em>


Copy & paste issue: the "none" term isn't used in this example.

davidlehn · 2018-01-18T00:38:17Z

You almost touched every file with this commit.
What happens when you have multiple things that would compact to @none? Do they all go under @none key? Or is it an error? Should be tests one way or another and maybe spec text about it (I may have missed it).

gkellogg · 2018-01-18T03:37:12Z

What happens when you have multiple things that would compact to @none.

In general, this is just fine. Two node objects without an @id would be indexed using @none, and this would round-trip. Same for index, language, and type maps.

There may be an interesting case with unamed/indexed graphs, which we should probably explore. Two anonymous graphs would seem to merge into a single graph, and would not round-trip, where you would normally consider that each anonymous graph was, in fact, named with a missing BNode. Of course, this can be controlled in the source by giving the graph an explicit blank node identifier.

I think we should explore with some test cases, and address with a note, but I don't think we need different behavior.

azaroth42

Overall looks good, just a couple of questions.

azaroth42 · 2018-01-18T16:17:49Z

spec/latest/json-ld/index.html

@@ -4250,7 +4448,7 @@ <h2>Language Maps</h2>
        or an array containing both <code>@language</code> and <code>@set</code>
      </span>. The keys of a
      <a>language map</a> MUST be <a>strings</a> representing
-      [[BCP47]] language codes and the values MUST be any of the following types:</p>
+      [[BCP47]] language codes, or the <a>keyword</a> <code>@none</code>, and the values MUST be any of the following types:</p>



..., the keyword @none or an alias that expands to it, and ... ?

azaroth42 · 2018-01-18T16:19:36Z

spec/latest/json-ld-api/index.html

-                          <span class="ednote">From <a href="#issue-480">Issue 480</a>
-                          the <a>language tag</a> <code>@none</code> represents no language.</span></li>
+                          <span class="changed">If <em>language</em> is <code>@none</code>,
+                            or expands to <code>@none</code>, do not set the <code>@language</code> member.</span>


Do we need explicit mention that the underlying RDF datatype is different? (xsd:string versus rdf:langString)

Actually, @none only appears in the map, it is removed on expansion, so there is no underlying RDF datatype. This is why the id maps were changed to not use blank node identifiers, as it aids round-tripping, and they never make it into either expansion or RDF generation.

Right, I agree that @none is json-ld flavored sugar... but the RDF Concepts spec asserts different datatypes based on the presence or lack of a language tag. At least that's my understanding of https://www.w3.org/TR/rdf11-concepts/#section-Graph-Literal

If there's a language tag, then the data type is necessarily rdf:langString. And if there isn't, then it's not. If there isn't another data type, it's an xsd:string.

Which we can be completely silent about, but I'd like us to have thought about that silence :)

Language maps have a proscribed form, where the key is a string, and the value is either a string or an array of strings, so the expansion of a map value of @none, would be like {"@value": "non-languaged-tagged value"}, thus no datatype or language. The Object to RDF Conversion algorithm renders this as a literal without datatype or language, which, as you note, is semantically equivalent to having the datatype xsd:string.

Do you think this needs some exposition in either document?

azaroth42 · 2018-01-18T16:22:55Z

I agree that having an array of objects indexed to @none is fine. An unnamed named graph that gets put into an index seems like a pathological edge case, that anyone who knows what they're doing would instinctively avoid anyway.

…, as well as the bare keyword `@none`. Fix some minor issues in the API defining values of `@nest`.

…es on merging multiple unindexed or unnamed graphs.

gkellogg · 2018-01-19T00:12:28Z

@azaroth42 and @davidlehn your concerns have been addressed.

niklasl · 2018-01-22T16:29:05Z

Is this feature compatible with an approach possible in 1.0, where you'd use two distinct terms to separate language-keyed values and non-language-tagged ones? Like:

{
  "@context": {
   "rdfs": "http://www.w3.org/2000/01/rdf-schema#",
   "xsd": "http://www.w3.org/2001/XMLSchema#",
   "labelByLang": {"@id": "rdfs:label", "@container": "@language"},
   "labelString": {"@id": "rdfs:label", "@language": null}
  },
  "labelByLang": {"en": "thing", "sv": "sak"},
  "labelString": "stuff"
}```

gkellogg · 2018-01-22T17:12:48Z

We may need more tests for this, and the IRI Compaction language may need to be updated. In most cases, preference in term selection is given to the most specific match; there are tests for this, but we may not have sufficient overlap. The language in 2.9.3 may need to be moved until after @none is added in 2.10 to ensure this.

I'll look through to make sure we have adequate test coverage for these various cases.

On a personal note, I have long thought the term selection logic to be over complicated in trying to maintain such variation in matching, which is not the typical case. In 99% of cases (I would guess) there is a single term matching an IRI, and that would be the one to use. Allowing for matching from among a set of possible IRIs complicates the algorithm, and is one aspect of performance issues for large datasets.

…@language` and `@language@set` when value has neither `@language` nor `@type`. Also move `@index` and `@index@set` for values without `@index` to below `@none`.

gkellogg · 2018-01-22T19:33:38Z

CG resolved to merge: https://json-ld.org/minutes/2018-01-22/index.html#resolution-4

niklasl · 2018-01-22T20:18:46Z

If further work on this is to be done (apart from ensuring the explicit none-term case works), could we consider this to be an opt-in feature for map containers; e.g. by declaring "@container": ["@language", "@none"]? (Although I understand the desire to get away from complex term selection rules, unless we're to evict some of the existing mechanics, we need to support it going forward.)

The "none-term" case is important, we've used it for the National Library data in Sweden (since, in library data in general, lacking language tags is the norm, alas.)

…org#569.

gkellogg · 2018-01-22T22:51:06Z

@niklasl Can you provide an example which illustrates your concern. I'd hate to have to add @none as a container value, and it's not clear how that would fit anyway, as @none is effectively no container.

You can inhibit the behavior by defining another term which would be used for terms which you don't want to be in a language map, as the test case added does.

As I mentioned on the call, we could also vary based on processingMode/version, and only allow @none in a language map if version is 1.1 (or processingMode is json-ld-1.1). However, the ability to define a term that is matched with a higher priority than language maps for values without a language would seem to adequately address this issue for me.

…org#569.

niklasl · 2018-01-23T13:50:45Z

@gkellogg Indeed, if the behaviour is inhibited to handle the example I provided above, the critical part of the issue is solved, since anyone using language-maps in 1.1 can declare a "companion" term for the none-case.

Apart from the reasonably undesired full IRI fallback, there is still one case which works in 1.0 that isn't compatible with this IIUC. If we use @vocab, in 1.0, this will round-trip:

{
  "@context": {
   "@vocab": "http://www.w3.org/2000/01/rdf-schema#",
   "labelByLang": {"@id": "label", "@container": "@language"}
  },
  "labelByLang": {"en": "thing", "sv": "sak"},
  "label": "stuff"
}

Whereas with none-maps in 1.1, we now get only "labelByLang": {"en": "thing", "sv": "sak", "@none": "stuff"}.

The new behaviour is reasonably a better default (and I am very grateful for your work on implementing it!). It is technically is a breaking change though (as shown in this and the above example). I think we need to document it in tests, and probably add the processingMode >= 1.1 check as you suggest. And also add clear instructions for how to upgrade to 1.1, by stating that you now have to use a companion term for language-maps in order to single out the non-language-tagged values using distinct terms.

I'm not keen on my opt-in either, I just wanted to mention it in case we end up not being able to justify the break. (And if so, while @none isn't a reasonable container value (in that you cannot use @container: @none), technically, it could be, just a pointless one (a map with one key (@none) where all values for the term are put...). Hopefully that is moot of course.)

gkellogg · 2018-01-23T17:16:21Z

See #571.

…org#569.

gkellogg added 2 commits January 15, 2018 15:06

Rename scope tests that were confused for map tests.

ba63075

Adds indexing on @none, or an alias of @none for all types of map…

42c89ee

… containers. Changes behavior for id maps to use `@none` instead of a blank node identifier. Fixes #480

gkellogg added test-suite api syntax labels Jan 17, 2018

gkellogg added this to the JSON-LD 1.1 milestone Jan 17, 2018

gkellogg self-assigned this Jan 17, 2018

gkellogg requested review from niklasl, davidlehn, dlongley and azaroth42 January 17, 2018 02:02

Improve Inverse Context creation and setting _containers_ in IRI Comp…

dccd5a0

…action to use concatenated/sorted values from `@container`.

davidlehn reviewed Jan 17, 2018

View reviewed changes

Remove errant line and change @type@set to @set@type.

f6c72ca

davidlehn reviewed Jan 18, 2018

View reviewed changes

Fix spelling and copy/paste issues.

3e19d7b

azaroth42 reviewed Jan 18, 2018

View reviewed changes

gkellogg added 2 commits January 18, 2018 15:54

Clarify in the grammar that in a map, an alias of @none may be used…

b1cc41b

…, as well as the bare keyword `@none`. Fix some minor issues in the API defining values of `@nest`.

Describe @none in the @graph @index case, and note potential issu…

82f0526

…es on merging multiple unindexed or unnamed graphs.

gkellogg mentioned this pull request Jan 20, 2018

Language maps on literals with no language don't play well together #480

Closed

Update container prioritization in IRI Compaction of @none over `…

f8ecc02

…@language` and `@language@set` when value has neither `@language` nor `@type`. Also move `@index` and `@index@set` for values without `@index` to below `@none`.

gkellogg merged commit 749a243 into master Jan 22, 2018

gkellogg deleted the maps-with-no-keys branch January 22, 2018 21:31

gkellogg added a commit to digitalbazaar/jsonld.js that referenced this pull request Jan 22, 2018

Index maps on @none or an alias for json-ld/json-ld.org#569.

35028f1

gkellogg added a commit to digitalbazaar/jsonld.js that referenced this pull request Jan 22, 2018

Improve container ordering for term selection due to json-ld/json-ld.…

ae5fd8d

…org#569.

gkellogg added a commit to digitalbazaar/jsonld.js that referenced this pull request Jan 22, 2018

Index maps on @none or an alias for json-ld/json-ld.org#569.

4aa55b3

gkellogg added a commit to digitalbazaar/jsonld.js that referenced this pull request Jan 22, 2018

Improve container ordering for term selection due to json-ld/json-ld.…

09032fb

…org#569.

gkellogg mentioned this pull request Jan 23, 2018

Limit use of @none in maps to 1.1 mode #571

Closed

gkellogg added a commit to digitalbazaar/jsonld.js that referenced this pull request Jan 25, 2018

Index maps on @none or an alias for json-ld/json-ld.org#569.

c9b7a11

gkellogg added a commit to digitalbazaar/jsonld.js that referenced this pull request Jan 25, 2018

Improve container ordering for term selection due to json-ld/json-ld.…

a04e615

…org#569.

gkellogg added a commit to digitalbazaar/jsonld.js that referenced this pull request Jan 25, 2018

Index maps on @none or an alias for json-ld/json-ld.org#569.

f1d4307

gkellogg added a commit to digitalbazaar/jsonld.js that referenced this pull request Jan 25, 2018

Improve container ordering for term selection due to json-ld/json-ld.…

5b24185

…org#569.

gkellogg added a commit to digitalbazaar/jsonld.js that referenced this pull request Jan 26, 2018

Index maps on @none or an alias for json-ld/json-ld.org#569.

abf0cc2

gkellogg added a commit to digitalbazaar/jsonld.js that referenced this pull request Jan 26, 2018

Improve container ordering for term selection due to json-ld/json-ld.…

a268b42

…org#569.

gkellogg added a commit to digitalbazaar/jsonld.js that referenced this pull request Feb 10, 2018

Index maps on @none or an alias for json-ld/json-ld.org#569.

54e86f5

gkellogg added a commit to digitalbazaar/jsonld.js that referenced this pull request Feb 10, 2018

Improve container ordering for term selection due to json-ld/json-ld.…

71146a9

…org#569.

davidlehn pushed a commit to digitalbazaar/jsonld.js that referenced this pull request Feb 28, 2018

Index maps on @none or an alias for json-ld/json-ld.org#569.

5ee8969

davidlehn pushed a commit to digitalbazaar/jsonld.js that referenced this pull request Feb 28, 2018

Improve container ordering for term selection due to json-ld/json-ld.…

2e0bc2f

…org#569.

davidlehn pushed a commit to digitalbazaar/jsonld.js that referenced this pull request Feb 28, 2018

Index maps on @none or an alias for json-ld/json-ld.org#569.

e6c606d

davidlehn pushed a commit to digitalbazaar/jsonld.js that referenced this pull request Feb 28, 2018

Improve container ordering for term selection due to json-ld/json-ld.…

5def7d7

…org#569.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Maps with no keys #569

Maps with no keys #569

gkellogg commented Jan 17, 2018

davidlehn Jan 17, 2018

davidlehn Jan 17, 2018

gkellogg Jan 17, 2018

gkellogg commented Jan 17, 2018 •

edited by davidlehn

Loading

davidlehn Jan 18, 2018

davidlehn Jan 18, 2018

davidlehn Jan 18, 2018

davidlehn Jan 18, 2018

davidlehn Jan 18, 2018

davidlehn Jan 18, 2018

davidlehn commented Jan 18, 2018

gkellogg commented Jan 18, 2018 •

edited

Loading

azaroth42 left a comment

azaroth42 Jan 18, 2018

azaroth42 Jan 18, 2018

gkellogg Jan 18, 2018

azaroth42 Jan 19, 2018

gkellogg Jan 20, 2018

azaroth42 commented Jan 18, 2018

gkellogg commented Jan 19, 2018

niklasl commented Jan 22, 2018

gkellogg commented Jan 22, 2018

gkellogg commented Jan 22, 2018

niklasl commented Jan 22, 2018

gkellogg commented Jan 22, 2018

niklasl commented Jan 23, 2018

gkellogg commented Jan 23, 2018

Maps with no keys #569

Maps with no keys #569

Conversation

gkellogg commented Jan 17, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gkellogg commented Jan 17, 2018 • edited by davidlehn Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

davidlehn commented Jan 18, 2018

gkellogg commented Jan 18, 2018 • edited Loading

azaroth42 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

azaroth42 commented Jan 18, 2018

gkellogg commented Jan 19, 2018

niklasl commented Jan 22, 2018

gkellogg commented Jan 22, 2018

gkellogg commented Jan 22, 2018

niklasl commented Jan 22, 2018

gkellogg commented Jan 22, 2018

niklasl commented Jan 23, 2018

gkellogg commented Jan 23, 2018

gkellogg commented Jan 17, 2018 •

edited by davidlehn

Loading

gkellogg commented Jan 18, 2018 •

edited

Loading