-
Notifications
You must be signed in to change notification settings - Fork 279
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Question: Example of doing pagination (especially on nested resources that are paginated) #69
Comments
This is a good question, and it remains an open question for me. (FWIW, basic pagination is documented at https://github.com/shurcooL/githubv4#pagination, but it doesn't cover pagination across more than one resource that this issue is asking about.) So far in the few places where I need to do pagination across more than one resource type per query, I've been able to get by with only doing pagination at the top level, where it's most important. For the inner levels, I just request the maximum single page size with a "TODO" to figure out a good approach, and that's getting me by for now (enough that this isn't a high priority for me). You can see an example here: It seems to me that it could be implemented by doing nested iteration. That is, paginate across all entries in the inner-most resource type first, then go to next higher level page, and repeat until all pages are visited. Something like:
If someone finds good example code, I think this would be worth documenting to save others the effort of figuring it out. One more thought: this question is applicable to GraphQL as a whole, so perhaps people have found solutions for this in other languages that may be helpful to find. |
@dmitshur thats really surprising to hear - seems like this would be a pretty typical thing with complex graphql APIs. As it stands right now i'm thinking i'm going to have to make a choice between 2 options both of which require a 2nd API call that is JIT (just in time) at its usage: Adding an additional function on the client where you pass in the top level resource you want to fill something like OR adding a function instead of using a field on that struct IE Definitely seems like a rock and a hard place to implement nested pagination with this framework. /cry |
If anyone stumble across this - this is how i've implemented it in our OpsLevel go client
|
@dmitshur is there any way a field can be included on query struct but ignored when this library generates the graphql query - similar to the json tag Something like type StuffQuery struct {
StuffPages []StuffPage `graphql:"stuff"`
Stuff []Stuff `graphql:",ignored"`
} This would allow me to get the raw data back from the server into the field |
After some digging it appears not - https://github.com/shurcooL/graphql/blob/master/internal/jsonutil/graphql.go#L257 seems like thats the area which would need to be refactored to support this. Seems like this would be a good feature to add to support writing easy to access structures of paginated data. Thoughts @dmitshur? |
Following this thread on Twitter, I came up with the following: https://gist.github.com/myitcv/f7da6cb073de3c48c9f88b6b95225b4c The overall goal was to extract:
for a given GitHub repository, minimising the number of calls to the GitHub GraphQL endpoint as far as possible. This felt like incredibly hard work, not to mention error-prone. I would welcome any thoughts on a better approach. |
I'm working with a graphQL API that has a top level resource that has several fields in it that are paginated themselves - something like
and its unclear to me how to properly handle pagination of these nested
tools
andtags
resources. I understand how thepageInfo
object'sendCursor
andhasNextPage
fields work and how to use them with the argument toafter
to query the next page of data but its unclear to me the proper pattern to use in go code when using this graphql framework.I would love if anyone has links to example code query-ing nested pagination or blog posts talking about how to do pagination with this framework.
Thanks! Love the framework so far!
The text was updated successfully, but these errors were encountered: