Skip to content
This repository was archived by the owner on Jul 24, 2024. It is now read-only.

Support pagination for retrieving runtime programs #170

Merged
rathishcholarajan merged 22 commits intoQiskit:runtime-release-q4from
kt474:runtime-program-pagination
Nov 8, 2021
Merged

Support pagination for retrieving runtime programs #170
rathishcholarajan merged 22 commits intoQiskit:runtime-release-q4from
kt474:runtime-program-pagination

Conversation

@kt474
Copy link
Copy Markdown
Contributor

@kt474 kt474 commented Oct 20, 2021 •
edited
Loading

Summary

fixes #26

Details and comments

kt474
kt474 commented Oct 20, 2021
View reviewed changes
Comment thread qiskit_ibm/api/rest/runtime.py Outdated
@kt474 kt474 changed the title [WIP] Support pagination for retrieving runtime programs Support pagination for retrieving runtime programs Oct 20, 2021
rathishcholarajan
rathishcholarajan suggested changes Oct 25, 2021
View reviewed changes
Copy link
Copy Markdown
Member

@rathishcholarajan rathishcholarajan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pprint_programs() also needs a limit and offset. And when limit and offset are passed we should automatically refresh.

@jyu00 Do we need the refresh option anymore or should we refresh by default? I find myself setting it always just to make sure everything is up to date.

@kt474 kt474 mentioned this pull request Oct 26, 2021
Merged
@rathishcholarajan
Copy link
Copy Markdown
Member

rathishcholarajan commented Oct 26, 2021

@kt474 Let's call it skip instead of offset to be consistent with the jobs terminology.

@kt474
Copy link
Copy Markdown
Contributor Author

kt474 commented Oct 27, 2021 •
edited
Loading

Say there are 10 jobs and then something like programs(limit = 4, skip = 4) is called where 2 programs are skipped and 4 are retrieved - so there are 4 in the cache. If a user then calls something like programs(limit = 2, skip = 0) the original 2 programs would not be returned and the first 2 programs from the cache would be returned instead.

A better solution would be to automatically refresh when limit and skip are passed

rathishcholarajan
rathishcholarajan suggested changes Oct 28, 2021
View reviewed changes
Comment thread releasenotes/notes/runtime-program-pagination-8d599ae984a5ce33.yaml Outdated
Comment thread qiskit_ibm/runtime/ibm_runtime_service.py Outdated
Comment thread releasenotes/notes/runtime-program-pagination-8d599ae984a5ce33.yaml Outdated
Comment thread qiskit_ibm/runtime/ibm_runtime_service.py
Comment thread qiskit_ibm/runtime/ibm_runtime_service.py Outdated
Comment thread test/ibm/runtime/test_runtime.py
kt474 and others added 4 commits October 28, 2021 15:58
@kt474 @rathishcholarajan
Update releasenotes/notes/runtime-program-pagination-8d599ae984a5ce33…
77199e9 
….yaml

Co-authored-by: Rathish Cholarajan <rathishc24@gmail.com>
@kt474 @rathishcholarajan
Update releasenotes/notes/runtime-program-pagination-8d599ae984a5ce33…
ac4c506 
….yaml

Co-authored-by: Rathish Cholarajan <rathishc24@gmail.com>
@kt474
add to test case
d692aee
@kt474
refactor refresh logic
4987738
@rathishcholarajan rathishcholarajan added backport potential The bug might be minimal and/or important enough to be port to stable Changelog: New Feature Include in the Added section of the changelog labels Nov 2, 2021
@kt474 kt474 mentioned this pull request Nov 2, 2021
Closed
rathishcholarajan
rathishcholarajan suggested changes Nov 5, 2021
View reviewed changes
Copy link
Copy Markdown
Member

@rathishcholarajan rathishcholarajan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kt474 Could you write some integration tests?

@jyu00
Copy link
Copy Markdown
Collaborator

jyu00 commented Nov 6, 2021

I know the limit on the server side is 20, but can we be nicer to our users? Like if someone specify 50, just loop under the cover and get all 50, instead of making the user do it.

jyu00
jyu00 reviewed Nov 6, 2021
View reviewed changes
Comment thread qiskit_ibm/runtime/ibm_runtime_service.py Outdated
@jyu00
Copy link
Copy Markdown
Collaborator

jyu00 commented Nov 6, 2021

@rathishcholarajan I think we should try to get this into the release this week. Now that the server limits the results to 20, there is no way for users to go beyond 20 with the existing programs() method.

rathishcholarajan
rathishcholarajan reviewed Nov 8, 2021
View reviewed changes
Comment thread qiskit_ibm/api/clients/runtime.py Outdated
@rathishcholarajan rathishcholarajan changed the base branch from main to runtime-release-q4 November 8, 2021 02:45
rathishcholarajan
rathishcholarajan suggested changes Nov 8, 2021
View reviewed changes
Comment thread qiskit_ibm/api/clients/runtime.py Outdated
Comment thread qiskit_ibm/api/rest/runtime.py Outdated
Comment thread qiskit_ibm/api/rest/runtime.py Outdated
Comment thread qiskit_ibm/api/clients/runtime.py Outdated
Comment thread qiskit_ibm/api/rest/runtime.py Outdated
Comment thread qiskit_ibm/runtime/ibm_runtime_service.py Outdated
@rathishcholarajan rathishcholarajan merged commit 4522d9b into Qiskit:runtime-release-q4 Nov 8, 2021
@rathishcholarajan rathishcholarajan mentioned this pull request Nov 8, 2021
Merged
@kt474 kt474 deleted the runtime-program-pagination branch November 8, 2021 14:35
rathishcholarajan added a commit that referenced this pull request Nov 9, 2021
@rathishcholarajan @renier
@jyu00 @kt474
Runtime release Q4 (#165)
8dffda3 
* Remove version field from runtime program (#152)

* Remove version field from runtime program

* Add release note

* Rename isPublic to is_public when creating or reading runtime programs (#155)

* Update programId to program_id when running program (#139)

This needs to change in the program upload body request in order to
meet the IBM Cloud API guidance.

Co-authored-by: Jessie Yu <jessieyu@us.ibm.com>

* Add support to view program update date (#160)

* Upload runtime program using 'data' field (#157)

* Read programs from "programs" array in response (#161)

* Remove version field from runtime program (#152)

* Remove version field from runtime program

* Add release note

* Rename isPublic to is_public when creating or reading runtime programs (#155)

* Update programId to program_id when running program (#139)

This needs to change in the program upload body request in order to
meet the IBM Cloud API guidance.

Co-authored-by: Jessie Yu <jessieyu@us.ibm.com>

* Add support to view program update date (#160)

* Upload runtime program using 'data' field (#157)

* Read programs from "programs" array in response (#161)

* Pass program as base64 string to update (#168)

* Accept JSON schema as program metadata (#158)

* Accept JSON schema as program metadata

* Update qiskit_ibm/runtime/ibm_runtime_service.py

Co-authored-by: Jessie Yu <jessieyu@us.ibm.com>

* Apply suggestions from code review

* Apply suggestions from code review

Co-authored-by: Jessie Yu <jessieyu@us.ibm.com>

Co-authored-by: Jessie Yu <jessieyu@us.ibm.com>

* Pass program params as object (#171)

* Fix integration tests

* Use count to reduce one last extra call to API (#172)

* Allow updating runtime metadata in place (#188)

* update runtime metadata

* return if no data

* fix mypy

* Update releasenotes/notes/update-runtime-metadata-d2ddbcfc0d034530.yaml

Co-authored-by: Rathish Cholarajan <rathishc24@gmail.com>

* Remove version field from runtime program (#152)

* Remove version field from runtime program

* Add release note

* Rename isPublic to is_public when creating or reading runtime programs (#155)

* Update programId to program_id when running program (#139)

This needs to change in the program upload body request in order to
meet the IBM Cloud API guidance.

Co-authored-by: Jessie Yu <jessieyu@us.ibm.com>

* Add support to view program update date (#160)

* Upload runtime program using 'data' field (#157)

* Read programs from "programs" array in response (#161)

* Pass program as base64 string to update (#168)

* Accept JSON schema as program metadata (#158)

* Accept JSON schema as program metadata

* Update qiskit_ibm/runtime/ibm_runtime_service.py

Co-authored-by: Jessie Yu <jessieyu@us.ibm.com>

* Apply suggestions from code review

* Apply suggestions from code review

Co-authored-by: Jessie Yu <jessieyu@us.ibm.com>

Co-authored-by: Jessie Yu <jessieyu@us.ibm.com>

* Pass program params as object (#171)

* Fix integration tests

* Use count to reduce one last extra call to API (#172)

* Allow updating runtime metadata in place (#188)

* update runtime metadata

* return if no data

* fix mypy

* Update releasenotes/notes/update-runtime-metadata-d2ddbcfc0d034530.yaml

Co-authored-by: Rathish Cholarajan <rathishc24@gmail.com>

* Allow filtering runtime jobs by program ID (#193)

* Allow filtering runtime jobs by program ID

* Fix lint

* Allow runtime program authors to retrieve program data (#174)

* retrieve program data

* refetch once if no program data

* remove unused import

* refresh program on data property

* fix lint

* Update qiskit_ibm/runtime/runtime_program.py

Co-authored-by: Rathish Cholarajan <rathishc24@gmail.com>

* Update qiskit_ibm/runtime/runtime_program.py

Co-authored-by: Rathish Cholarajan <rathishc24@gmail.com>

* Update qiskit_ibm/runtime/runtime_program.py

Co-authored-by: Rathish Cholarajan <rathishc24@gmail.com>

* add test case

* add test case

* add default data constant

* add _validate_program method

* Update test/ibm/runtime/test_runtime.py

Co-authored-by: Rathish Cholarajan <rathishc24@gmail.com>

* Update cache after updating program (#196)

* Allow filtering runtime jobs by provider (#197)

* add provider param

* split provider into hub/group/project

* add reno

* wip add test case

* fix lint/docs

* Update qiskit_ibm/runtime/ibm_runtime_service.py

Co-authored-by: Rathish Cholarajan <rathishc24@gmail.com>

* Update releasenotes/notes/filter-jobs-by-provider-dead04faaf223840.yaml

Co-authored-by: Rathish Cholarajan <rathishc24@gmail.com>

* refactor test cases

* remove print

* add integration test

Co-authored-by: Rathish Cholarajan <rathishc24@gmail.com>

* Support pagination for retrieving runtime programs  (#170)

* wip add limit/offset params

* add reno

* refactor & update test case

* offset -> skip, implement refresh logic

* refresh when skip/limit not default

* Update releasenotes/notes/runtime-program-pagination-8d599ae984a5ce33.yaml

Co-authored-by: Rathish Cholarajan <rathishc24@gmail.com>

* Update releasenotes/notes/runtime-program-pagination-8d599ae984a5ce33.yaml

Co-authored-by: Rathish Cholarajan <rathishc24@gmail.com>

* add  to test case

* refactor refresh logic

* refactor

* fix lint

* add integration test

* update doc string

* Update qiskit_ibm/api/clients/runtime.py

Co-authored-by: Rathish Cholarajan <rathishc24@gmail.com>

* cleanup merge

* Apply suggestions from code review

* fix lint refactor

* Fetch all programs upfront 20 at a time and store in cache

For subsequent requests paginate and return from cache

* Fix integration test

Co-authored-by: Rathish Cholarajan <rathishc24@gmail.com>
Co-authored-by: Rathish Cholarajan <Rathish.C@ibm.com>

Co-authored-by: Renier Morales <renier@users.noreply.github.com>
Co-authored-by: Jessie Yu <jessieyu@us.ibm.com>
Co-authored-by: Kevin Tian <kevin.tian@ibm.com>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

2 more reviewers

@jyu00 jyu00 jyu00 left review comments

@rathishcholarajan rathishcholarajan rathishcholarajan approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

backport potential The bug might be minimal and/or important enough to be port to stable Changelog: New Feature Include in the Added section of the changelog

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Support runtime programs pagination

3 participants

@kt474 @rathishcholarajan @jyu00