Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adjust BigCZ max area from 1500 -> 8000 #2409

Closed
wants to merge 1 commit into from

Conversation

kellyi
Copy link
Contributor

@kellyi kellyi commented Oct 19, 2017

Overview

This PR increases the max analyzable area for BigCZ from 1500 to 8000.

Connects #2388

Demo

screen shot 2017-10-19 at 5 05 58 pm

&

screen shot 2017-10-19 at 5 06 11 pm

Testing Instructions

  • get this branch, bundle, then visit bigcz
  • try to analyze an aoi larger than 1500 but below 8000 sq km & verify that it works
  • try to analyze an aoi larger than 8000 sq km & verify that you encounter the guard

@rajadain
Copy link
Member

I'll take a deeper look soon, but the main reason for going with 1500 for the limit was WDC searches timing out. It may be that we need to increase the timeout for those somewhere, or constrain the results otherwise (like limiting to the last five years for example) to make it finish in time, and pair that with this to be viable. I'll know more once I've taken a look.

@rajadain
Copy link
Member

I tested this, but WDC results choke on even a 2000 sq km area of interest. Rather surprisingly, the bottleneck isn't the actual SOAP call, but this line:

response = recursive_asdict(request(**kwargs))

which serializes the results so they can be cached. This caching code is in a spot that does not know about the response's internal structure, so cannot make sophisticated decisions about it.

This requires a multi-part solution:

  1. Move the caching (and recursive_asdict) out of make_request, into the level above. Continue caching GetServicesInBox2 for 1 week.
  2. For GetSeriesCatalogForBox2, make the request, then sort the SeriesRecords by descending order of date. Then, pick an adequate cap n, and choose the top n records, and serialize those and return to the client.

Since the above approach discards results after the first n, we may need to add some custom pagination logic. But this should be deferred to a later date, since we may be introducing search in #2389, allowing the user to refine their results that way.

@rajadain
Copy link
Member

Another alternative is to limit the results to be within the last 5 years if the area of interest is more than 1500 sq km. This may be usable in conjunction with the work above.

@rajadain
Copy link
Member

Closing in favor of #2418.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants