You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently the pagination API uses page-based pagination without sorting by time of creation/modification. This leads to incomplete results when iterated over the pages in dynamic systems with multiple modifications happening at the same time when results are fetched.
For example, if during the process of iterating over the pages of environments (api/v1/environment) an additional environment is created, either:
the new environment will be omitted, or
a different random environment will be omitted
this is because the environments are sorted by namespace and name. For a more concrete example see the details below.
Say we have five environments A, B, D, E, F, and page size equals 2.
User performs two actions:
requested creating environment called "C", and then
opens the form with a list of all environments
If the database commit happens after the second page is fetched, this could lead to the form getting replies:
A, B - first page (ok)
D, E - second page (ok at the time)
E, F - third page - oops!
If many users are creating environments, the admin would be randomly missing some environments.
If the results were sorted by date of environment was created (/last modified if environments can be renamed), then this is not a problem, because the replies could be (assuming A, B, D, E, F were created in this order):
Coming back to this, I think we can most easily fix this by simply sorting paginated results by time. I'm sure it's possible to do with the other proposed methods, but this just seems simplest to me. @trallard and @krassowski do you have preferences about this?
This seems like an easy fix, so once I have your input I'll happily fix this.
Feature description
Currently the pagination API uses page-based pagination without sorting by time of creation/modification. This leads to incomplete results when iterated over the pages in dynamic systems with multiple modifications happening at the same time when results are fetched.
For example, if during the process of iterating over the pages of environments (
api/v1/environment
) an additional environment is created, either:this is because the environments are sorted by namespace and name. For a more concrete example see the details below.
Say we have five environments A, B, D, E, F, and page size equals 2.
User performs two actions:
If the database commit happens after the second page is fetched, this could lead to the form getting replies:
If many users are creating environments, the admin would be randomly missing some environments.
If the results were sorted by date of environment was created (/last modified if environments can be renamed), then this is not a problem, because the replies could be (assuming A, B, D, E, F were created in this order):
Originally posted in nebari-dev/nebari#2599 (comment)
Instead, one of pagination implementations which guarantees data completeness should be used across conda-store REST API:
Value and/or benefit
No randomly missing items (e.g. environments) in paginated replies.
Anything else?
No response
The text was updated successfully, but these errors were encountered: