Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prevent API groups from growing stale #264

Merged
merged 4 commits into from
Feb 12, 2019

Conversation

fabianvf
Copy link
Member

@fabianvf fabianvf commented Feb 5, 2019

@willthames @asetty if you'd like to take a look

@openshift-ci-robot openshift-ci-robot added the size/S Denotes a PR that changes 10-29 lines, ignoring generated files. label Feb 5, 2019
@willthames
Copy link
Contributor

What is the difference between the existing refresh mechanism and the one resulting from this change? It's not obvious to me how the refresh is triggered.

Copy link
Contributor

@asetty asetty left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, just one comment

if request_resources:
resources = self.get_resources_for_api_version(DISCOVERY_PREFIX, group['name'], version, preferred)
new_group[version] = ResourceGroup(preferred, resources=resources)
groups[DISCOVERY_PREFIX][group['name']] = new_group
self._cache['resources'] = groups
self._cache['resources'].update(groups)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wouldn't doing the dictionary update every iteration of the loop be slower than just setting self._cache['resources'] at the end?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

woops, typo, that was supposed to be one level out

@fabianvf
Copy link
Member Author

fabianvf commented Feb 6, 2019

This makes the api_groups refresh when accessed, so that it's always the most up to date list of resources available in the cluster. Before it would only refresh when you attempted to access a resource that did not exist in the cache. Since updating api_groups is just a single API call (with the lazy discoverer at least), I thought this behavior would be more useful than requiring you to manually refresh the cache or miss a resource to get the new groups to show up. We hit this in one of our operators, where it created a CRD and later verified that the API group existed- because api_groups didn't refresh, the check failed.

@djzager
Copy link
Member

djzager commented Feb 6, 2019

I think I tested this correctly:

#!/usr/bin/env python
import yaml
import pprint
from kubernetes import client, config
from openshift.dynamic import DynamicClient

k8s_client = config.new_client_from_config()
dyn_client = DynamicClient(k8s_client)
api_groups = dyn_client.resources.api_groups
for res in api_groups:
    print(res)
➜  openshift-restclient-python git:(pr/264) ✗ ./example.py

certificates.k8s.io
events.k8s.io
authorization.k8s.io
apiextensions.k8s.io
autoscaling
apps
batch
apiregistration.k8s.io
networking.k8s.io
extensions
storage.k8s.io
policy
coordination.k8s.io
rbac.authorization.k8s.io
scheduling.k8s.io
admissionregistration.k8s.io
authentication.k8s.io

Then I create my CRD

➜  openshift-restclient-python git:(pr/264) ✗ ./example.py

examples.djzager.io
certificates.k8s.io
events.k8s.io
authorization.k8s.io
apiextensions.k8s.io
autoscaling
apps
batch
apiregistration.k8s.io
networking.k8s.io
extensions
storage.k8s.io
policy
coordination.k8s.io
rbac.authorization.k8s.io
scheduling.k8s.io
admissionregistration.k8s.io
authentication.k8s.io

@djzager
Copy link
Member

djzager commented Feb 6, 2019

And I can show it failing on master:

(osrcp) ➜  openshift-restclient-python git:(master) ✗ ./example.py
Traceback (most recent call last):
  File "./example.py", line 10, in <module>
    for res in api_groups:
TypeError: 'instancemethod' object is not iterable

@fabianvf
Copy link
Member Author

fabianvf commented Feb 8, 2019

@willthames did that clear up your reservations? Any opposition to merge?

@fabianvf fabianvf merged commit 31c96ef into openshift:master Feb 12, 2019
willthames pushed a commit to willthames/openshift-restclient-python that referenced this pull request Nov 22, 2019
* Make api_groups a property

* Prevent api_groups from growing stale by re-requesting resources every time

* Add back v1.List

* Typo
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
size/S Denotes a PR that changes 10-29 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants