Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Load spatial data #750

Merged
merged 8 commits into from
Feb 7, 2024
Merged
Show file tree
Hide file tree
Changes from 7 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .eslintrc.js
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ module.exports = {
parserOptions: { ecmaVersion: 2024 },
overrides: [
{
files: ["tests/**/*.js"],
files: ["./*.js", "tests/**/*.js"],
rules: {
"no-console": 0,
},
Expand Down
9 changes: 9 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
spatial-data/c_05mr24.prj filter=lfs diff=lfs merge=lfs -text
spatial-data/c_05mr24.shp filter=lfs diff=lfs merge=lfs -text
spatial-data/c_05mr24.shx filter=lfs diff=lfs merge=lfs -text
spatial-data/s_05mr24.dbf filter=lfs diff=lfs merge=lfs -text
spatial-data/s_05mr24.prj filter=lfs diff=lfs merge=lfs -text
spatial-data/s_05mr24.shx filter=lfs diff=lfs merge=lfs -text
spatial-data/c_05mr24.dbf filter=lfs diff=lfs merge=lfs -text
spatial-data/s_05mr24.shp filter=lfs diff=lfs merge=lfs -text
spatial-data/cities500.txt filter=lfs diff=lfs merge=lfs -text
2 changes: 0 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
/.editorconfig
/.gitattributes

# Ignore dependencies that are managed with Composer.
# Generally you should only ignore the root vendor directory. It's important
Expand Down Expand Up @@ -94,4 +93,3 @@ credentials-*.json

# Test screenshots
/tests/**/screenshots/**

2 changes: 2 additions & 0 deletions docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@ services:
MYSQL_USER: drupal
MYSQL_PASSWORD: drupal
MYSQL_ROOT_PASSWORD: drupal
ports:
- 3306:3306

api-proxy:
build:
Expand Down
16 changes: 10 additions & 6 deletions docs/dev/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,23 +9,27 @@ caches, importing/exporting, configuration, etc.
To get up and running quickly with Docker, ensure you have Docker installed and
then:

1. Clone this repository into a new directory and `cd` into it.
2. Copy the file at `web/sites/example.settings.dev.php` to
1. [Install git LFS](spatial-data.md). We use LFS to store large geospatial
files. If you clone the repo before installing LFS, you can get your repo
up-to-date by running `git lfs pull` after installing.
2. Clone this repository into a new directory and `cd` into it.
3. Copy the file at `web/sites/example.settings.dev.php` to
`web/sites/settings.dev.php` and make any necessary changes.
> [!NOTE]
> As of this writing, no changes are necessary.
3. Run `make zap` from the command line.
4. Run `make zap` from the command line.
This command will first attempt to destroy any weather.gov 2.0 containers you
have, so you may notice some warnings on the command line. It is safe to
ignore those warnings. After the warnings, it will build all of the necessary
containers, create a Drupal site with our configuration, and populate the
site with our content.
4. Run `npm ci` from the command line.
5. Run `npm ci` from the command line.
This installs our Javascript and Sass code linters and formatters, as well as
Cypress, which is necessary for end-to-end and accessibility testing.
5. Browse to [http://localhost:8080](http://localhost:8080) in your broswer. You
6. [Load necessary geospatial data](spatial-data.md) into your database.
7. Browse to [http://localhost:8080](http://localhost:8080) in your broswer. You
should see our front page! Congrats!
6. Browse to [http://localhost:8080/user/login](http://localhost:8080/user/login)
8. Browse to [http://localhost:8080/user/login](http://localhost:8080/user/login)
to log in. Your username is `admin` and your password is `root`. Then you can
do stuff!

Expand Down
43 changes: 43 additions & 0 deletions docs/dev/spatial-data.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
# Spatial data

We store certain geospatial data in our own database and use it for some basic
geospatial queries. In order for the site to work correctly locally, the data
needs to be loaded first. The spatial data is stored in our repo using
[git LFS](https://www.atlassian.com/git/tutorials/git-lfs).

Copy link
Member

@loganmcdonald-noaa loganmcdonald-noaa Feb 7, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might go without saying but - make sure the docker container is running and healthy?

1. Install and enable git LFS.

- **Linux**: [download instructions](https://docs.github.com/en/repositories/working-with-files/managing-large-files/installing-git-large-file-storage?platform=linux)
- **Windows**: [download instructions](https://docs.github.com/en/repositories/working-with-files/managing-large-files/installing-git-large-file-storage?platform=windows)
- **macOS**: the easiest way is to use Homebrew and run
```sh
brew install git-lfs
```
Alternatively, there are manual [download instructions](https://docs.github.com/en/repositories/working-with-files/managing-large-files/installing-git-large-file-storage?platform=mac).

2. Enable git LFS on your local machine by running

```sh
git lfs install
```

3. Load the spatial data into your local database by running
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe for the future not-usses (or us, when we've long forgotten), we can reiterate that you need to zap the application locally here and also git-lfs pull if needed. Or at least just to say, make sure the containers are running locally before doing this step.

Copy link
Collaborator Author

@greg-does-weather greg-does-weather Feb 7, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Went ahead and updated the doc, but also added spatial loading into the make zap target.

```sh
node load-shapefiles.js
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have to also have the dependencies from this script locally since this is running locally and not in the container? i.e. mariadb, shapefiles, etc.

```

## Loading data into cloud environments

Ensure you have the spatial data in your project in the `spatial-data` directory.
If not, run `git lfs pull` to fetch it. Then, log into cloud.gov:

```sh
cf login --sso -a api.fr.cloud.gov
```

And then run this utility script, indicating which environment you want to load
data into.

```sh
./scripts/load-spatial-data.sh <environment>
```
Loading