-
Notifications
You must be signed in to change notification settings - Fork 1.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix(DQL): optimize query for has function with offset. (#7727) #8431
Conversation
|
@anurags92 this needs to clean up the hidden comments. |
return | ||
// for `has` function when there is no filtering and ordering, we fetch | ||
// correct paginated results so no need to apply pagination here. | ||
if !(len(sg.Filters) == 0 && sg.SrcFunc != nil && sg.SrcFunc.Name == "has") { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there a case possible where this if evaluates to false but we still need to perform pagination like before?
} | ||
} | ||
return int32(count) | ||
// make offset = 0, if there is need to fetch all the results. | ||
return int32(count), 0 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This should probably be:
return int32(count), int32(sg.Params.Offset)
There is a case where one might want all the results after skipping the first few (denoted by the Offset).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@akon-dey Actually no, I think that in this case worker/task.go needs pb.query.Offset
to in fact be 0 in order to function correctly.
This PR has a failing test, which needs to be properly understood. I am marking this as draft for now. |
Fixes DGRAPH-574. This PR reduces the latency of the `has` function slightly. For example the given query:- ``` { me(func: has(actor.film), first: 10, offset: 300000){ name@en } } ``` when run on the `21million` dataset takes around `80` milliseconds to execute as compared to earlier around `93` milliseconds. Machine Specs:- ``` Model Name: MacBook Pro Model Identifier: MacBookPro16,1 Processor Name: 6-Core Intel Core i7 Processor Speed: 2.6 GHz Number of Processors: 1 Total Number of Cores: 6 L2 Cache (per Core): 256 KB L3 Cache: 12 MB Memory: 16 GB ``` Note:- This does not optimize for the `has` function on predicates with the `@lang` tag.
6c8d35a
to
5e6f44c
Compare
I don't see a significant performance improvement on my end.
Benchmark results:
Benchmark results:
Here's the code I used to get the results. It expects a local dgraph running with |
This was fixed. It had a test which required a different set of changes. They are being brought in via: #8471 |
@anurags92 the description have some hidden comments. Please, remove it. |
On a larger dataset (sf10).
Code to repro: https://go.dev/play/p/Mv6Y6YOG_BR |
Fixes DGRAPH-574.
This PR reduces the latency of the
has
function slightly. For example the given query:-when run on the
21million
dataset takes around80
milliseconds to execute as compared to earlier around93
milliseconds.Machine Specs:-
Note:-
This does not optimize for the
has
function on predicates with the@lang
tag.(cherry picked from commit 9ba15b7)