Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update how spatial schemas are managed; update zone schema to add spatial index #1492

Merged
merged 3 commits into from
Jul 31, 2024

Conversation

greg-does-weather
Copy link
Collaborator

What does this PR do? 🛠️

This PR modifies how we manage our spatial data schemas and data. The various spatial-data/sources file are all updated accordingly.

  • source modules export a single object that identifies the involved table, a list of schema upgrades, and a function for loading data
    • the keys for the schema upgrades is a schema version
    • the values for schema upgrades is a function that makes any necessary schema mutations and returns true if source data should also be reloaded or false if the schema modification is sufficient on its own
  • the meta module is updated accordingly
  • a new schema module is added to handle migrating schemas
  • the main load-shapefile.js file is modified to be source-agnostic; it can rely on each source plus the schema migration module to handle themselves

Overall, this PR should make zone-based spatial queries a little faster, but it should also make it easier and faster to make schema-only updates to spatial data.

Copy link
Collaborator

@eric-gade eric-gade left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Works 👍 and I like the design. You're welcome, future us.

Code Review Checklist

This is an automated comment on every pull request requiring a review. A checked item is either an assertion that I have tested this item or an indication that I have verified it does not apply to this pull request.

The Basics

  • Checks are passing
  • I read the code
  • I ran the code
  • (if applicable) Post deploy steps are run
  • (if applicable) I validated the change on the deployed version in

Documentation

  • changes to “how we do things” are documented in READMEs
  • all new functions and methods are commented using plain language
  • any new modules added documented in modules.md

Security

  • security false positives are documented
  • data from external sources is cleaned and clearly marked

Reliability

  • error handling exists for unusual or missing values
  • interactions with external systems are wrapped in try/except
  • functionality is tested with unit or integration tests
  • dependency updates in composer.json also got changed in composer-lock.json

Infrastructure

  • all changes are auditable and documented via a script
  • it is clear who can and should run the script
  • (if applicable) diagrams have been updated or added in PlantUML

Accessibility

  • New pages have been added to cypress-axe file so that they will be tested with our automated accessibility testing
  • Meets WCAG 2.0 AA or 2.1 AA for Section 508 compliance
    • Site is keyboard accessible. All interactions can be accessed with a keyboard
    • Site is free of keyboard traps. The keyboard focus is never trapped in a loop
    • All form inputs have explicit labels
    • Form instructions are associated with inputs
    • All relevant images use an img tag
    • All images have appropriate alt attributes
    • Multimedia is tagged. All multimedia has appropriate captioning and audio description
    • Text has sufficient color contrast. All text has a contrast ratio of 4.5:1 with the background
    • Site never loses focus. Focus is always visible when moving through the page with the keyboard
    • Tab order is logical
    • Tables are coded properly. Tables have proper headers and column attributes
    • Headings are nested properly. Heading elements are nested in a logical way
    • Language is set. The language for the page is set
    • CSS is not required to use the page. The page makes sense with or without CSS
    • Links are unique and contextual. All links can be understood taken alone, e.g., ‘Read more - about 508’
    • Page titles are descriptive

Device Matrix

  • firefox/gecko (renders correctly and user interactions work)
  • chrome/chromium/edge (renders correctly and user interactions work)
  • safari/webkit (renders correctly and user interactions work)
  • web page is readable and usable
    • at 480px (mobile)
    • at 640px (tablet)
    • at 1024px (desktop)

@greg-does-weather greg-does-weather merged commit 73f632a into main Jul 31, 2024
17 checks passed
@greg-does-weather greg-does-weather deleted the mgwalker/1490-spatial-table-keys branch July 31, 2024 16:45
@greg-does-weather
Copy link
Collaborator Author

I've run this migration against my own environment and it went smoothly. Running it on staging next, and if that goes well, I'll run it on beta.

@loganmcdonald-noaa Can you update design?

git checkout main
git pull
cf login --sso -a api.fr.cloud.gov
./scripts/load-spatial-data.sh design

@eric-gade Can you update your environment?

git checkout main
git pull
cf login --sso -a api.fr.cloud.gov
./scripts/load-spatial-data.sh eric

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Zones table is missing spatial key
2 participants