Switch ListRoles to the paginated API

[gzdunek]

Contributes to https://github.com/gravitational/customer-sensitive-requests/issues/182 (switches ListRoles in Web UI to the paginated API).

e-counterpart: https://github.com/gravitational/teleport.e/pull/3881

Instead of loading all roles with a single call to GetRoles, we now use the paginated API.
The user roles are used in two kinds of UI elements in the Web UI:

• Resource tables (roles screen, locks screen) - I switched these to serviserside tables (I must admit that it was more difficult than I thought, I hope I used the servierside props correctly).
  • One note: since there is a short delay between updating the resource and having the updated value propagate to the cache, we can't refetch the data immediately after updating the role. Because of that, I have to update the changed role locally, on the browser side (so we have a mix of server side and browser side pagination).
• Select inputs - these were switched to SelectFieldAsync that loads the options asynchronously on demand. SelectFieldAsync passes a search string to the loadOptions function so we can filter the roles on the server side.

I recommend review commit-by-commit.
I'd also appreciate if you could test these changes locally, as I'm not fully familiar with all the places I have changed 🙏

Changelog: Use a paginated API for listing roles to improve efficiency

[avatus]

Everything looks great so far, and I did test locally. Check the comment I left and let me know what you think.

[avatus]

We need to be able to handle requests from an old webclient (in the case of two different proxies running together) which would still expect a response that is only an array of items, rather than this paginated object.

Without having to resort to a larger refactor that can version this endpoint, we can just use the existence of one of these new parameters to determine between old/new clients. For example "if limit exists, send new response. otherwise, send only uiRoles"

We did something similar here

[avatus]

Also, make sure the TODO i left in useKeyBasedPagination would fix this from the frontend side as well please (new webclient being able to handle "old" response types).

[gzdunek]

Oh, I completely forgot that we can have proxies running different versions.
I added a check on the parameters to determine what response type should be returned. It's not beautiful but works.

Also, make sure the TODO i left in useKeyBasedPagination would fix this from the frontend side as well please

The thing I could do to allow new webclient being able to handle "old" response types, would be to wrap the response in the service layer if I detect that I didn't receive the paginated object. But I'm not sure if it makes any sense, things like limit or search would not work at all.

I don't know how to handle this, to be honest. If I have a server-side table, how can I make it work with both paginated and non-paginated responses?
Maybe instead of modifying the existing endpoint I should add a new one? At least the request would fail with 404 error instead of something like res.map is not a function.
EDIT: I could also throw an error in resource.ts saying ...please update your Teleport Proxy to version ... if I detect a non-paginated response.

[avatus]

I don't know how to handle this, to be honest. If I have a server-side table, how can I make it work with both paginated and non-paginated responses?

The worst case scenario is we receive a very large dataset from the "old" api. I don't have any answer for this yet. An upcoming project will be to add a properly versioned API. I don't know exactly how it would work yet for newer web clients receiving 404s but my guess is there would have to be a fallback to a previous version or just accept the 404. I want to discuss this with the team with an RFD once we get the greenlight for the project.

[gzdunek]

Actually, we can support the "new webclient being able to handle old response types" case too.
I added a compatibility layer in resource.ts: if we detect that we received a non-paginated response, we can "paginate" it locally, in the browser. I added support for search, limit and startKey.
Let me know what you think about this.

[avatus]

i LOVE this. amazing work!!

[avatus]

Thanks for the backward compatibility change. Everything looks fine to me and works locally. I would appreciate if other reviewers could click around locally as well with things they are more familiar w.r.t roles to double check this one.

[kimlisa]

looks good for the most part, i'm mostly concerned by the static 50 limit ok after playing more with it, i got how it works 👍

[public-teleport-github-review-bot]

@gzdunek See the table below for backport results.

Branch	Result
branch/v14	Failed
branch/v15	Create PR

[kimlisa]

i had a question on this. i am also introducing a new listing, and i was going to follow the same pattern as access request listing (which is infinite scrolling by use of useKeyBasedPagination) then realized the role listing here is manual fetching (click next).

i was curious why we preferred the manual fetching over the infinite scrolling?

[avatus]

I wont speak for @gzdunek , but I chose infinite scrolling over "next" because design wanted it (access requests) to be infinite scrolling. no other reason tbh.

I think it's probably case by case even if we do want a more standard approach. I know notifications are also infinite scrolling

[gzdunek]

Actually I didn't consider switching that screen to infinite scrolling, because I only wanted to change the API from non-paginated to the paginated one.
Also, I thought that "next" fetching was that "more standard approach" for resource tables 😄 (and tbh I didn't know that we now want to prefer infinite scrolling over "next").
I think you may ask Kenny about your case.