Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions episodes/01-intro-raster-data.md
Original file line number Diff line number Diff line change
Expand Up @@ -128,7 +128,7 @@ of changes in resolution.
### Raster Data Format for this Workshop

Raster data can come in many different formats. For this workshop, we will use
one of the most common formats for raster data, i.e. the GeoTIFF format, which has the extension `.tif`.
one of the most common formats for raster data, i.e. the GeoTIFF format, which has the extension `.tif`.
A `.tif` file stores metadata or attributes about the file as embedded `tif tags`. For instance, your camera
might store a tag that describes the make and model of the camera or the date
the photo was taken when it saves a `.tif`. A GeoTIFF is a standard `.tif` image
Expand All @@ -149,13 +149,13 @@ from a GeoTIFF file.
## More Resources on the `.tif` format

* [GeoTIFF on Wikipedia](https://en.wikipedia.org/wiki/GeoTIFF)
* [OSGEO TIFF documentation](https://trac.osgeo.org/geotiff/)
* [Open Geospatial Consortium (OGC) GeoTIFF standard](https://www.ogc.org/standards/geotiff/)
:::

### Multi-band Raster Data

A raster can contain one or more bands. One type of multi-band raster
dataset that is familiar to many of us is a color image. A basic color
dataset that is familiar to many of us is a color image. A basic color
image often consists of three bands: red, green, and blue (RGB). Each
band represents light reflected from the red, green or blue portions of
the electromagnetic spectrum. The pixel brightness for each band, when
Expand Down
2 changes: 1 addition & 1 deletion episodes/03-crs.md
Original file line number Diff line number Diff line change
Expand Up @@ -91,14 +91,14 @@

## Which projection should I use?

A well know projection is the [Mercator projection](https://en.wikipedia.org/wiki/Mercator_projection) introduced by the Flemisch cartographer Gerardus Mercator in the 16th Century. This is a so-called cilindrical projection, meaning that a virtual cilinder is placed around the globe to flatten it. This type of projections are relatively accurate near to the equator, but towards the poles blows things up (more info on cylindrical projections [here](https://gisgeography.com/cylindrical-projection/). The main advantage of the Mercator projection is that it is very suitable for navigation purposes since it always shows North as *up* and South and as *down* - in the 17th century this projection was essential for sailors to navigate the oceans.

Check warning on line 94 in episodes/03-crs.md

View workflow job for this annotation

GitHub Actions / Build markdown source files if valid

[uninformative link text]: [here](https://gisgeography.com/cylindrical-projection/)

To decide if a projection is right for your data, answer these questions:

* What is the area of minimal distortion?
* What aspect of the data does it preserve?

[Peter Dana from the University of Colorado at Boulder](https://foote.geography.uconn.edu/gcraft/notes/mapproj/mapproj_f.html) and the [Department of Geo-Information Processing](https://kartoweb.itc.nl/geometrics/Map%20projections/mappro.html) have a good discussion of these aspects of projections. Online tools like [Projection Wizard](https://projectionwizard.org/) can also help you discover projections that might be a good fit for your data.
The [Department of Geo-Information Processing](https://kartoweb.itc.nl/geometrics/Map%20projections/mappro.html) has a good discussion of these aspects of projections. Online tools like [Projection Wizard](https://projectionwizard.org/) and the [Worldmapgenerator](https://www.worldmapgenerator.com) can also help you explore projections and discover what might be a good fit for your data.

:::callout
## Data Tip
Expand Down
6 changes: 2 additions & 4 deletions episodes/04-geo-landscape.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,8 +27,7 @@ The [Open Source Geospatial Foundation (OSGEO)](https://www.osgeo.org/) supports

* [QGIS](https://www.qgis.org/en/site/) is a professional GIS application that is
built on top of and proud to be itself Free and Open Source Software (FOSS). QGIS is
written in Python and C++, has a python console interface, allows to develop plugins and has several interfaces written in R including
[RQGIS](https://cran.r-project.org/package=RQGIS).
written in Python and C++, has a python console interface and allows one to develop plugins.
* [GRASS GIS](https://grass.osgeo.org/), commonly referred to as GRASS
(Geographic Resources Analysis Support System), is a FOSS-GIS software suite used for
geospatial data management and analysis, image processing, graphics and maps
Expand Down Expand Up @@ -66,7 +65,7 @@ The [Open Source Geospatial Foundation (OSGEO)](https://www.osgeo.org/) supports
through their [DevLabs](https://developers.arcgis.com/). ArcGIS software can be
installed using
[Chef Cookbooks from Github](https://github.com/Esri/arcgis-cookbook). In addition, ESRI offers the [ArcPy Python library](https://pro.arcgis.com/en/pro-app/latest/arcpy/get-started/what-is-arcpy-.htm) as part of an ArcGIS Pro licence, allowing to translate operations in the ArcGIS Pro GUI to Python scripts.
* Pitney Bowes produce [MapInfo Professional](https://www.pitneybowes.com/us/location-intelligence/geographic-information-systems/mapinfo-pro.html),
* Precisely produces [MapInfo Professional](https://www.precisely.com/product/precisely-mapinfo/mapinfo-pro), formerly developed by Pitney Bowes Software,
which was one of the earliest desktop GIS programs on the market.
* [Hexagon Geospatial Power Portfolio](https://www.hexagongeospatial.com/products/products)
includes many geospatial tools including ERDAS Imagine, powerful software for remote sensing.
Expand All @@ -90,7 +89,6 @@ The [Open Source Geospatial Foundation (OSGEO)](https://www.osgeo.org/) supports
Private companies have released SDK platforms for large scale GIS analysis:

* [Kepler.gl](https://kepler.gl/#/) is Uber's toolkit for handling large datasets (i.e. Uber's data archive).
* [Boundless Geospatial](https://boundlessgeo.com/) is built upon OSGEO software for enterprise solutions.

Publicly funded open-source platforms for large scale GIS analysis:

Expand Down
2 changes: 1 addition & 1 deletion episodes/05-access-data.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@
In this episode we will explore how to access open satellite data using Python. In particular, we will
consider [the Sentinel-2 data collection that is hosted on Amazon Web Services (AWS)](https://registry.opendata.aws/sentinel-2-l2a-cogs).
This dataset consists of multi-band optical images acquired by the constellation of two satellites from
[the Sentinel-2 mission](https://sentinel.esa.int/web/sentinel/missions/sentinel-2) and it is continuously updated with
[the Sentinel-2 mission](https://dataspace.copernicus.eu/data-collections/copernicus-sentinel-data/sentinel-2) and it is continuously updated with
new images.

## Search for satellite imagery
Expand Down Expand Up @@ -71,7 +71,7 @@
::: challenge
## Exercise: Discover a STAC catalog
Let's take a moment to explore the Earth Search STAC catalog, which is the catalog indexing the Sentinel-2 collection
that is hosted on AWS. We can interactively browse this catalog using the STAC browser at [this link](https://radiantearth.github.io/stac-browser/#/external/earth-search.aws.element84.com/v1).

Check warning on line 74 in episodes/05-access-data.md

View workflow job for this annotation

GitHub Actions / Build markdown source files if valid

[uninformative link text]: [this link](https://radiantearth.github.io/stac-browser/#/external/earth-search.aws.element84.com/v1)

1. Open the link in your web browser. Which (sub-)catalogs are available?
2. Open the Sentinel-2 Level 2A collection, and select one item from the list. Each item corresponds to a satellite
Expand Down Expand Up @@ -160,7 +160,7 @@
COGs typically include multiple lower-resolution versions of the original image, called "overviews", which can also be
accessed independently. By providing this "pyramidal" structure, users that are not interested in the details provided
by a high-resolution raster can directly access the lower-resolution versions of the same image, significantly saving
on the downloading time. More information on the COG format can be found [here](https://www.cogeo.org).

Check warning on line 163 in episodes/05-access-data.md

View workflow job for this annotation

GitHub Actions / Build markdown source files if valid

[uninformative link text]: [here](https://www.cogeo.org)
:::

In order to get data for a specific location you can add longitude latitude coordinates (World Geodetic System 1984 EPSG:4326) in your request.
Expand Down Expand Up @@ -566,7 +566,7 @@
The authentication procedure for dataset with restricted access might differ depending on the data provider. For the
NASA CMR, follow these steps in order to access data using Python:

* Create a NASA Earthdata login account [here](https://urs.earthdata.nasa.gov);

Check warning on line 569 in episodes/05-access-data.md

View workflow job for this annotation

GitHub Actions / Build markdown source files if valid

[uninformative link text]: [here](https://urs.earthdata.nasa.gov)
* Set up a netrc file with your credentials, e.g. by using [this script](https://git.earthdata.nasa.gov/projects/LPDUR/repos/daac_data_download_python/browse/EarthdataLoginSetup.py);
* Define the following environment variables:

Expand Down
2 changes: 1 addition & 1 deletion episodes/06-raster-intro.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@
For your convenience, we included the scene of interest among the datasets that you have already downloaded when following [the setup instructions](../learners/setup.md) (the raster data files should be in the `data/sentinel2` directory). You should, however, be able to download the same datasets "on-the-fly" using the JSON metadata file that was created in [the previous episode](05-access-data.md) (the file `rhodes_sentinel-2.json`).


If you choose to work with the provided data (which is advised in case you are working offline or have a slow/unstable network connection) you can skip the remaining part of the block and continue with the following section: [Load a Raster and View Attributes](#Load-a-Raster-and-View-Attributes).

Check warning on line 52 in episodes/06-raster-intro.md

View workflow job for this annotation

GitHub Actions / Build markdown source files if valid

[missing anchor]: [Load a Raster and View Attributes](#Load-a-Raster-and-View-Attributes)

If you want instead to experiment with downloading the data on-the-fly, you need to load the file `rhodes_sentinel-2.json`, which contains information on where and how to access the target images from the remote repository:

Expand Down Expand Up @@ -113,7 +113,7 @@

The output tells us that we are looking at an `xarray.DataArray`, with `1` band, `10980` rows, and `10980` columns. We can also see the number of pixel values in the `DataArray`, and the type of those pixel values, which is unsigned integer (or `uint16`). The `DataArray` also stores different values for the coordinates of the `DataArray`. When using `rioxarray`, the term coordinates refers to spatial coordinates like `x` and `y` but also the `band` coordinate. Each of these sequences of values has its own data type, like `float64` for the spatial coordinates and `int64` for the `band` coordinate.

This `DataArray` object also has a couple of attributes that are accessed like `.rio.crs`, `.rio.nodata`, and `.rio.bounds()` (in jupyter you can browse through these attributes by using `tab` for auto completion or have a look at the documentation [here](https://corteva.github.io/rioxarray/stable/rioxarray.html#rioxarray-rio-accessors)), which contains the metadata for the file we opened. Note that many of the metadata are accessed as attributes without `()`, however since `bounds()` is a method (i.e. a function in an object) it requires these parentheses this is also the case for `.rio.resolution()`.

Check warning on line 116 in episodes/06-raster-intro.md

View workflow job for this annotation

GitHub Actions / Build markdown source files if valid

[uninformative link text]: [here](https://corteva.github.io/rioxarray/stable/rioxarray.html#rioxarray-rio-accessors)

```python
print(rhodes_red.rio.crs)
Expand Down Expand Up @@ -191,7 +191,7 @@
```
![Raster plot 80 x 80 meter resolution with rioxarray](fig/E06/rhodes_red_80_B04.png){alt="raster plot with defualt setting"}

This plot shows the satellite measurement of the band `red` for Rhodes before the wildfire. According to the [Sentinel-2 documentaion](https://sentinels.copernicus.eu/web/sentinel/technical-guides/sentinel-2-msi/msi-instrument), this is a band with the central wavelength of 665nm. It has a spatial resolution of 10m. Note that the `band=1` in the image title refers to the ordering of all the bands in the `DataArray`, not the Sentinel-2 band number `04` that we saw in the pystac search results.
This plot shows the satellite measurement of the band `red` for Rhodes before the wildfire. According to the [Sentinel-2 documentaion](https://sentiwiki.copernicus.eu/web/s2-mission#S2-Mission-MSI-Instrument), this is a band with the central wavelength of 665nm. It has a spatial resolution of 10m. Note that the `band=1` in the image title refers to the ordering of all the bands in the `DataArray`, not the Sentinel-2 band number `04` that we saw in the pystac search results.


:::callout
Expand All @@ -213,7 +213,7 @@

![Raster plot using vmin 100 and vmax 2000](fig/E06/rhodes_red_80_B04_vmin100_vmax2000.png){alt="raster plot with robust setting"}

More options can be consulted [here](https://docs.xarray.dev/en/v2024.02.0/generated/xarray.plot.imshow.html). You will notice that these parameters are part of the `imshow` method from the plot function. Since plot originates from matplotlib and is so widely used, your python environment helps you to interpret the parameters without having to specify the method. It is a service to help you, but can be confusing when teaching it. We will explain more about this below.

Check warning on line 216 in episodes/06-raster-intro.md

View workflow job for this annotation

GitHub Actions / Build markdown source files if valid

[uninformative link text]: [here](https://docs.xarray.dev/en/v2024.02.0/generated/xarray.plot.imshow.html)

:::

Expand Down
4 changes: 2 additions & 2 deletions episodes/08-crop-raster-data.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,13 +24,13 @@

In this episode, we will work with both raster and vector data.

As *raster data*, we will use satellite images from the search that we have carried out in [the episode: "Access satellite imagery using Python"](05-access-data.md) as well as Digital Elevation Model (DEM) data from the [Copernicus DEM GLO-30 dataset](https://spacedata.copernicus.eu/collections/copernicus-digital-elevation-model).
As *raster data*, we will use satellite images from the search that we have carried out in [the episode: "Access satellite imagery using Python"](05-access-data.md) as well as Digital Elevation Model (DEM) data from the [Copernicus DEM GLO-30 dataset](https://dataspace.copernicus.eu/explore-data/data-collections/copernicus-contributing-missions/collections-description/COP-DEM).

For the satellite images, we have searched for Sentinel-2 scenes of Rhodes from July 1st to August 31st 2023 that have less than 1% cloud coverage. The search resulted in 11 scenes. We focus here on the most recent scene (August 27th), since that would show the situation after the wildfire, and use this as an example to demonstrate raster data cropping.

For your convenience, we have included the scene of interest among the datasets that you have already downloaded when following [the setup instructions](../learners/setup.md). You should, however, be able to download the satellite images "on-the-fly" using the JSON metadata file that was created in [the previous episode](05-access-data.md) (the file `rhodes_sentinel-2.json`).

If you choose to work with the provided data (which is advised in case you are working offline or have a slow/unstable network connection) you can skip the remaining part of the block and continue with the following section: [Align the CRS of the raster and the vector data](#Align-the-CRS-of-the-raster-and-the-vector-data).

Check warning on line 33 in episodes/08-crop-raster-data.md

View workflow job for this annotation

GitHub Actions / Build markdown source files if valid

[missing anchor]: [Align the CRS of the raster and the vector data](#Align-the-CRS-of-the-raster-and-the-vector-data)

If you want instead to experiment with downloading the data on-the-fly, you need to load the file `rhodes_sentinel-2.json`, which contains information on where and how to access the target satellite images from the remote repository:

Expand Down Expand Up @@ -169,7 +169,7 @@
:::challenge
## Exercise: Clip the red band for Rhodes

Now that you have seen how clip a raster using a polygon, we want you to do this for the red band of the satellite image. Use the shape of Rhodes from GADM and clip the red band with it. Furthermore, make sure to transform the no data values to not-a-number (NaN) values.
Now that you have seen how clip a raster using a polygon, we want you to do this for the red band of the satellite image. Use the shape of Rhodes from GADM and clip the red band with it. Furthermore, make sure to transform the no data values to not-a-number (NaN) values.

::::solution
```python
Expand Down
10 changes: 5 additions & 5 deletions episodes/12-data-cube.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@ exercises: 0
---

:::questions
- Can I mosaic tiled raster datasets when my area of interest spans multiple files?
- Can I stack raster datasets that cover the same area along the time dimension in order to explore temporal changes of some quantities?
- Can I mosaic tiled raster datasets when my area of interest spans multiple files?
- Can I stack raster datasets that cover the same area along the time dimension in order to explore temporal changes of some quantities?
:::

:::objectives
Expand All @@ -15,9 +15,9 @@ exercises: 0

## Introduction

In the previous episodes we worked with satellite images with a fixed boundary on how they have been collected, however in many cases you would want to have an image that covers your area of interest which often does not align with boundaries of the collected images. If the phenomena you are interested in covers two images you could manually mosaic them, but sometimes you are interested in multiple images that overlapping.
In the previous episodes we worked with satellite images with a fixed boundary on how they have been collected, however in many cases you would want to have an image that covers your area of interest which often does not align with boundaries of the collected images. If the phenomena you are interested in covers two images you could manually mosaic them, but sometimes you are interested in multiple images that overlapping.

ODC-STAC offers functionality that allows you to get a mosaic-ed image based on the a bounding box or a polygon containing the area of interest. In this lesson we show how [odc-stac](https://odc-stac.readthedocs.io/en/latest/?badge=latest) can be employed to re-tile and stack satellite images in what are sometimes referred to as "data cubes".
ODC-STAC offers functionality that allows you to get a mosaic-ed image based on the a bounding box or a polygon containing the area of interest. In this lesson we show how [odc-stac](https://odc-stac.readthedocs.io/en/latest/?badge=latest) can be employed to re-tile and stack satellite images in what are sometimes referred to as "data cubes".

## Create a data cube with ODC-STAC

Expand Down Expand Up @@ -97,7 +97,7 @@ Data variables: (12/18)

## Working with the data cube

Like we did in the previous episode, let us calculate the NDVI for our study area. To do so we need to focus on the variables: the red band (`red`), the near infrared band (`nir`) and the scene classification map (`scl`). We will use the former two to calculated the NDVI for the AoI. The latter, we use as [a classification mask](https://sentinels.copernicus.eu/web/sentinel/technical-guides/sentinel-2-msi/level-2a/algorithm-overview) provided together with Sentinel-2 L2A products. In this mask, each pixel is classified according to a set of labels (see Figure 3 in [classification mask documentation](https://sentinels.copernicus.eu/web/sentinel/technical-guides/sentinel-2-msi/level-2a/algorithm-overview) ).
Like we did in the previous episode, let us calculate the NDVI for our study area. To do so we need to focus on the variables: the red band (`red`), the near infrared band (`nir`) and the scene classification map (`scl`). We will use the former two to calculated the NDVI for the AoI. The latter, we use as [a classification mask](https://sentiwiki.copernicus.eu/web/s2-processing#S2Processing-SceneClassification(SC)S2-Processing-Scene-Classificationtrue) provided together with Sentinel-2 L2A products. In this mask, each pixel is classified according to a set of labels (see Figure 2 in [classification mask documentation](https://sentiwiki.copernicus.eu/web/s2-processing#S2Processing-SceneClassification(SC)S2-Processing-Scene-Classificationtrue)).

First we define the bands that we are interested in:

Expand Down
6 changes: 3 additions & 3 deletions index.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ As a case study for this lesson we will focus on wildfires. According to [the IP

The data used in this lesson includes optical satellite images from [the Copernicus Sentinel-2 mission][sentinel-2] and topographical data from [OpenStreetMap (OSM)][osm]. These datasets are real-world open data sets that entail sufficient complexity to teach many aspects of data analysis and management. The datasets have been selected to allow participants to focus on the core ideas and skills being taught while offering the chance to encounter common challenges with geospatial data. Furthermore, we have selected datasets which are available anywhere on Earth.

During this lesson we will setup an analysis pipeline which identifies scorched areas based on bands of satellite images collected after the disaster in July 2023. Next, we will calculate the [Normalized Difference Vegetation Index (NDVI)](https://en.wikipedia.org/wiki/Normalized_difference_vegetation_index) to assess the vegetation cover of the areas before and after the wildfire. To investigate the affected built-up areas and main roads, we will use OSM vector data and compare them with the previously identified scorched areas.
During this lesson we will setup an analysis pipeline which identifies scorched areas based on bands of satellite images collected after the disaster in July 2023. Next, we will calculate the [Normalized Difference Vegetation Index (NDVI)](https://en.wikipedia.org/wiki/Normalized_difference_vegetation_index) to assess the vegetation cover of the areas before and after the wildfire. To investigate the affected built-up areas and main roads, we will use OSM vector data and compare them with the previously identified scorched areas.

To most effectively use this material, make sure to [download the data](learners/setup.md#data-sets) and follow [the software setup instructions](learners/setup.md#software-setup) before working through the lesson (this especially accounts for learners that follow this lesson in a workshop).

Expand All @@ -24,9 +24,9 @@ The main python libraries that are used in this lesson are:

- [geopandas](https://geopandas.org/en/stable/)
- [rioxarray](https://github.com/corteva/rioxarray)
- [xarray-spatial](https://xarray-spatial.org/)
- [xarray-spatial](https://xarray-spatial.readthedocs.io)
- [dask](https://www.dask.org/)
- [pystac-client](https://pystac-client.readthedocs.io/)

[sentinel-2]: https://sentinel.esa.int/web/sentinel/missions/sentinel-2
[sentinel-2]: https://dataspace.copernicus.eu/data-collections/copernicus-sentinel-data/sentinel-2
[osm]: https://www.openstreetmap.org/#map=14/45.2935/18.7986
Loading