Add support of Value Variables for uid_in.

[MichelDiz]

Experience Report

What you wanted to do

{

  getJeunet as var(func: eq(name@fr, "Jean-Pierre Jeunet")) 

  caro(func: eq(name@en, "Marc Caro")) {
    name@en
    director.film @filter(uid_in(~director.film, val(getJeunet) )) {
      name@en
    }
  }
}

If I use @filter(uid_in(~director.film, val(getJeunet))) it gets the error ": Wrong variable type encountered for var(getJeunet) 2.". I've tried other ways too just to test.

This could avoid problems with UID changes in Nodes. And be used in documentation.

Why that wasn't great, with examples

Most functions accept Query Variables

BTW

uid_in works fine with DQL Variables.

Some discussion on this https://discuss.dgraph.io/t/uid-in-cant-use-variables/2772
https://discuss.dgraph.io/t/improve-uid-in-with-value-variable/5266

[campoy]

This query could also be written cascade and normalize

{
  caro(func: eq(name@en, "Marc Caro"))
    @cascade @normalize {
    director.film {
      film: name@en
      ~director.film @filter(eq(name@en, "Jean-Pierre Jeunet")) {
          name@en
      }
    }
  }
}

If enough people request this we could consider it, but for now I think we can just keep it for later.

[MichelDiz]

Adding just one more example (sample in the end):

In my dataset, I would like to return only those friends who have (or Not in some cases) the same party in common "with me" (Who is the point/target of the query).

BTW, some cases could be multiple params - like "same party" + "Same School" + "Same generation" (Like "baby boomers") and so on.

The straight forward query would be like this below - However, I need to already know the uid from my party that I'm a member of. That's not ideal. Not all users will deal with UIDs directly in their end applications.

{
  q(func: eq(name, "Jane Doe")) {
    uid
    name
    memberOf {
      uid
      Party
    }
    friend @filter(NOT uid_in(memberOf, 0xa1257)) { #this case is "not part of some know party"
    uid
    name
    }
  }
}

To resolve this query along with the current procedures, we would need to create one more block var to take my (Jane Doe) party and then keep "memberOf" as reverse. Then it would be possible to take all (which could be numerous) users who are in the same party (or equivalent) and use their UIDs to filter on the final query.

So in this case, this approach would be extremely not recommended. The above one being the best.

{
  Jane as var (func: eq(name, "Jane Doe")) {
    uid
    memberOf {
      Party
      ~memberOf  {
        uid
        Fellas as name
      }
    }
  }

q(func: uid(Jane)) {
    uid
    name
    memberOf {
      uid
      Party
    }
    friend @filter(NOT uid(Fellas)) {
      uid
      name
    }
    }
  }

Using var and Val we can solve this

{
  q(func: eq(name, "Jane Doe")) {
    uid
    name
    memberOf {
      myparty as uid #Extracting the party to user in the uid_in func
      Party
    }
    friend @filter(NOT uid_in(memberOf, val(myparty))) { # Easily applying a filter without multiple round trips
    uid
    name
    }
  }
}

The result:

{
  "data": {
    "q": [
      {
        "uid": "0xa1255",
        "name": "Jane Doe",
        "memberOf": [
          {
            "uid": "0xa1257",
            "Party": "Republican Party"
          }
        ],
        "friend": [
          {
            "uid": "0xa1256",
            "name": "Lucas Lee"
          },
          {
            "uid": "0xa125d",
            "name": "Julia Doe"
          }
        ]
      }
    ]
  }
}

{
	"set": [{
			"uid": "_:Jane",
			"address": {
				"addressLocality": "Colorado Springs",
				"addressRegion": "CO",
				"postalCode": "80840",
				"streetAddress": "100 Main Street"
			},
			"colleague": {
				"uid": "_:JohnColleague",
				"name": "JohnColleague"
			},
			"email": "[email protected]",
			"image": "janedoe.jpg",
			"jobTitle": "Research Assistant",
			"name": "Jane Doe",
			"alumniOf": "Dartmouth",
			"birthPlace": "Philadelphia, PA",
			"birthDate": "1979-10-12",
			"height": "72 inches",
			"gender": "female",
			"friend": [{
					"uid": "_:Lucas"
				},
				{
					"uid": "_:Julia"
				},
				{
					"uid": "_:Jin"
				}
			],
			"memberOf": {
				"uid": "_:Republican",
				"Party": "Republican Party"
			},
			"nationality": "Albanian",
			"telephone": "(123) 456-6789",
			"url": "http://www.example.com"
		},
		{
			"uid": "_:Jin",
			"address": {
				"addressLocality": "Colorado Springs",
				"addressRegion": "CO",
				"postalCode": "80840",
				"streetAddress": "100 Main Street"
			},
			"colleague": {
				"uid": "_:JameColleague",
				"name": "JameColleague"
			},
			"email": "[email protected]",
			"image": "Jin.jpg",
			"jobTitle": "Research Assistant",
			"name": "Jin Ang",
			"alumniOf": "Dartmouth",
			"birthPlace": "Hangzhou, Zhejiang",
			"birthDate": "1978-10-12",
			"height": "72 inches",
			"gender": "male",
			"friend": [{
					"uid": "_:Lucas"
				},
				{
					"uid": "_:Julia"
				},
				{
					"uid": "_:Jane"
				}
			],
			"memberOf": {
				"uid": "_:Republican",
				"Party": "Republican Party"
			},
			"nationality": "Chinese",
			"telephone": "(555) 555-55555",
			"url": "http://www.example.com"
		},
		{
			"uid": "_:Lucas",
			"address": {
				"addressLocality": "Colorado Springs",
				"addressRegion": "CO",
				"postalCode": "80840",
				"streetAddress": "100 Main Street"
			},
			"colleague": {
				"uid": "_:JameColleague",
				"name": "JameColleague"
			},
			"email": "[email protected]",
			"image": "Lucas.jpg",
			"jobTitle": "Research Assistant",
			"name": "Lucas Lee",
			"alumniOf": "Dartmouth",
			"birthPlace": "San Francisco, CA",
			"birthDate": "1982-11-01",
			"height": "72 inches",
			"gender": "male",
			"friend": [{
					"uid": "_:Jin"
				},
				{
					"uid": "_:Julia"
				},
				{
					"uid": "_:Jane"
				}
			],
			"memberOf": {
				"uid": "_:Democrats",
				"Party": "Democrats"
			},
			"nationality": "Noth American",
			"telephone": "(123) 456-6789",
			"url": "http://www.example.com"
		},
		{
			"uid": "_:Julia",
			"address": {
				"addressLocality": "San Diego",
				"addressRegion": "CA",
				"postalCode": "92111",
				"streetAddress": "Argyle St"
			},
			"colleague": {
				"uid": "_:JameColleague",
				"name": "JameColleague"
			},
			"email": "[email protected]",
			"image": "Julia.jpg",
			"jobTitle": "Research Assistant",
			"name": "Julia Doe",
			"alumniOf": "Dartmouth",
			"birthPlace": "Philadelphia, PA",
			"birthDate": "1979-10-12",
			"height": "72 inches",
			"gender": "female",
			"friend": [{
					"uid": "_:Jin"
				},
				{
					"uid": "_:Lucas"
				},
				{
					"uid": "_:Jane"
				}
			],
			"memberOf": {
				"uid": "_:Democrats",
				"Party": "Democrats"
			},
			"nationality": "Albanian",
			"telephone": "(123) 456-6789",
			"url": "http://www.example.com"
		}
	]
}

[MichelDiz]

We could also do this with 2 var blocks + 1 query block:

{
  Jane as var(func: eq(name, "Jane Doe"))  {
    myparty as  memberOf
  }
    
  var(func: uid(Jane)) {
    friend {
      memberOf @filter(uid(myparty))  {
         Fellas as  ~memberOf 
      }
    }
  }

q(func: uid(Jane)) {
    uid
    name
    memberOf {
      uid
      Party
    }
    friend @filter(NOT uid(Fellas)) {
      uid
      name
    }
    }
  }

BTW - There is another important point to mention.

As far as I know, it would not be possible to get a value via Val in the same block (memberOf to => friend edge in the same block). This could also be something to work on, but as it seems complicated. That would be the proper form in the latter case.

{
  Jane as var(func: eq(name, "Jane Doe"))  {
    myparty as  memberOf #Extracting the party to user in the uid_in func
  }
  
  q(func: uid(Jane))  {
    uid
    name
    memberOf {
      uid 
      Party
    }
    friend @filter(NOT uid_in(memberOf, val(myparty))) { # Easily applying a filter without multiple round trips
    uid
    name
    }
  }
}

And another important fact to point out. Is that this query above would only be safe in the context of a direct query where returns only one entity. However, if the query has multiple entities (like "show me all users and apply this rule") this approach would not be safe. It would capture all parties of all people and they would all be filtered out. Returning null for the edge friend.

[MichelDiz]

Just a Note

I've noticed that the value variable works fine in uid_in if I convert the UID to number using a hexadecimal converter (Which cannot be done on the fly in the query). The issue is related to the lack of the possibility of conversion of UID to string. That is, it would be necessary for Value Variable to convert the UID to number and/or string.

{
  var(func: uid(0x0)){
    a as math(637870)
  }
 
  caro(func: eq(name@en, "Marc Caro")) {
    uid
    name@en
    director.film @filter(uid_in(~director.film, val(a)) ) {
      name@en
    }
  }

}

{
  "data": {
    "caro": [
      {
        "uid": "0x108cd1",
        "name@en": "Marc Caro",
        "director.film": [
          {
            "name@en": "The City of Lost Children"
          },
          {
            "name@en": "Delicatessen"
          },
          {
            "name@en": "The Bunker of the Last Gunshots"
          },
          {
            "name@en": "L'évasion"
          }
        ]
      }
    ]
  }
}

[lych4o]

I have encountered a similar problem.
With the schema like:

type person {
    locatedIn: [city]
    name
    friend: [person]
}
type city {
    locatedIn: [country]    
    name
}
type country {
    name
}
name: string .
locatedIn: [uid] @reverse.
friend: [uid] @reverse.

How do I filter out friends of a person who do not live in a country?
If uid_in could receive a list of uid or query varibles, I would like to do like this:

var(func: eq(name, 'America')) {
    cities as ~locatedIn { }
}
query(func: uid(*person_uid*)) {
    friend @filter(uid_in(locatedIn, cities)) { #Or val(cities)
        name
    }
}

Even with the way mentioned in your reply in Oct 1, it still does not support a list of uid.
Also, even this problem can be achieved by uid_in, it is still too wastefull since it is not need to compare all the cities with the out edge of friend.
What should I do to filter out friends of a person who do not live in a country?

[MichelDiz]

@lych4o https://discuss.dgraph.io/t/problem-about-node-filter-according-to-another-node-with-distance-of-2/5615/3

[seanlaff]

Really struggling without this feature since I can't use cascade to do filtering (some of my predicates are nullable, and if I use cascade, it excludes all the nodes containing predicates with nulls in the response).

This is especially painful when trying to do a conditional upsert where I want to set fields only if they are null.

My data model looks like this. I have Hosts and VMs- but VMs can move between hosts throughout time, so I also have a vm_info node that sits between hosts and vms that tracks start/end time.

Ex. Host1 had VM1 running on it from M-F, but VM2 running on it T-W and F-S. That means I'll have 3 vm_info nodes attached to the Host1 node.

Since the graph is updated in realtime, the endtime predicate on vm_info may be null.

If I want to get the vm_info nodes that have an edge to Host1 and an edge to VM2, I can't do this right now without overfetching (potentially a lot).

@cascade will prevent any endtime: null vm_info nodes being returned (which I'm most interested in since I want to add endTimes). uid_in supporting values would a welcome addition.

[all-seeing-code]

`` it gets the error ": Wrong variable type encountered for var(getJeunet) 2.". I've tried other wa

The error message has changed since the issue was reported. It now throws "Function 'uid_in' requires 1 arguments, but got 0 ([])", for the below query:

{
  getJeunet as var(func: eq(name@fr, "Jean-Pierre Jeunet")) 
  caro(func: eq(name@en, "Marc Caro")) {
    name@en
    director.film @filter(uid_in(~director.film, val(getJeunet) )) {
      name@en
    }
  }
}

[all-seeing-code]

Using var and Val we can solve this

{
  q(func: eq(name, "Jane Doe")) {
    uid
    name
    memberOf {
      myparty as uid #Extracting the party to user in the uid_in func
      Party
    }
    friend @filter(NOT uid_in(memberOf, val(myparty))) { # Easily applying a filter without multiple round trips
    uid
    name
    }
  }
}

val keyword doesn't return uid therefore it will not work becaue uid_in expects a <predicate> and an uid as noted here in docs. What we really want is uid of myparty which can be extracted using uid(myparty). The issue with this is that in nested functions we allow only few functions to be nested (val/count/len) as seen here. We would have to extend that to include uid function and also ensure that it is applied only in cases where we are sure that uid function will return only 1 value and not a list.

As a side note the below query works because val is forced to store exact uid in form of a hexadecimal number and therefore val(a) returns a uid which is passed to uid_in

Just a Note

I've noticed that the value variable works fine in uid_in if I convert the UID to number using a hexadecimal converter (Which cannot be done on the fly in the query). The issue is related to the lack of the possibility of conversion of UID to string. That is, it would be necessary for Value Variable to convert the UID to number and/or string.

{
  var(func: uid(0x0)){
    a as math(637870)
  }
 
  caro(func: eq(name@en, "Marc Caro")) {
    uid
    name@en
    director.film @filter(uid_in(~director.film, val(a)) ) {
      name@en
    }
  }

}

Tagging @ashish-goswami @pawanrawal and @MichelDiz.

[all-seeing-code]

Really struggling without this feature since I can't use cascade to do filtering (some of my predicates are nullable, and if I use cascade, it excludes all the nodes containing predicates with nulls in the response).

This is especially painful when trying to do a conditional upsert where I want to set fields only if they are null.

My data model looks like this. I have Hosts and VMs- but VMs can move between hosts throughout time, so I also have a vm_info node that sits between hosts and vms that tracks start/end time.

Ex. Host1 had VM1 running on it from M-F, but VM2 running on it T-W and F-S. That means I'll have 3 vm_info nodes attached to the Host1 node.

Since the graph is updated in realtime, the endtime predicate on vm_info may be null.

If I want to get the vm_info nodes that have an edge to Host1 and an edge to VM2, I can't do this right now without overfetching (potentially a lot).

@cascade will prevent any endtime: null vm_info nodes being returned (which I'm most interested in since I want to add endTimes). uid_in supporting values would a welcome addition.

@seanlaff Can you give me the query which you are running to by-pass the issue right now? Also can you give me a version of query which you want to run to avoid pulling unnecessary data? I am wondering if we could have a better way to solve your problem with the existing supported variables/functions.

[seanlaff]

@anurags92 I ended up solving it like this

query {
	project as var(func: eq(xid, "{{ .ProjectID }}"))
	fromNode as var(func: eq(xid, "{{ .FromID }}"))
	toNode as var(func: eq(xid, "{{ .ToID }}")) @cascade { has_project @filter(uid(project)) }
	var(func: uid(fromNode)) @cascade {
		has_project @filter(uid(project))
		a: fallsWithin as {{ .Relationship }} @cascade @filter(lt(start, "{{ .EndDate }}") AND gt(end, "{{ .EndDate }}")) {
			{{ .Relationship }} @filter(uid(toNode))
		}
		b: noEndStartLT as {{ .Relationship }} @cascade @filter(NOT has(end) AND lt(start, "{{ .EndDate }}")) {
			{{ .Relationship }} @filter(uid(toNode))
		}
	}
}

Followed by a bunch of conditional mutations, i.e @if(gt(len(fallsWithin), 0)) etc. Basically I handle the different "branches" at the query level rather than in the mutations.

I think the most painful part is that I cannot easily batch these kind of requests.

[seanlaff]

This PR looks interesting, in regards to my cascade filtering issue #5607

[all-seeing-code]

@seanlaff Merged this PR. Take a look if this helps you: #5320

[seanlaff]

@anurags92 Awesome!