Skip to content

Cache CUAHSI GetServicesInBox2 results #1932

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
kdeloach opened this issue Jun 1, 2017 · 2 comments
Closed

Cache CUAHSI GetServicesInBox2 results #1932

kdeloach opened this issue Jun 1, 2017 · 2 comments
Assignees

Comments

@kdeloach
Copy link
Contributor

kdeloach commented Jun 1, 2017

BTW, for future consideration: it occurred to me that currently there are only 94 "data services" in WDC (ie, the maximum total that can be returned by GetServicesInBox2), and those change rarely. Maybe a future strategy for better search responsiveness could be to get all services in the US lower 48 (based on a GetServicesInBox2 query) and cache them once a day, then just use that cache together with the actual query results from GetSitesInBox2.

Ref: #1931 (comment)

@rajadain
Copy link
Member

Commenting to subscribe to this thread and get email notifications.

@rajadain rajadain added the 1 label Jul 17, 2017
@rajadain
Copy link
Member

rajadain commented Jul 17, 2017

This can be done fairly easily along these lines:

diff --git a/src/mmw/apps/bigcz/clients/cuahsi/search.py b/src/mmw/apps/bigcz/clients/cuahsi/search.py
index 08622b4..955f8cd 100644
--- a/src/mmw/apps/bigcz/clients/cuahsi/search.py
+++ b/src/mmw/apps/bigcz/clients/cuahsi/search.py
@@ -124,8 +124,16 @@ def group_series_by_location(series):
 
 
 def make_request(request, **kwargs):
+    key = 'bigcz_{}_{}'.format(request.__name__,
+                               hash(frozenset(kwargs.items())))
+    cached = cache.get(key)
+    if cached:
+        return cached
+
     try:
-        return request(**kwargs)
+        response = request(**kwargs)
+        cache.set(key, response)
+        return response
     except URLError, e:
         if isinstance(e.reason, timeout):
             raise RequestTimedOutError()

Will need to ensure that this expires at appropriate times, because this is not an immutable operation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants