Lake Ice Detection from Sentinel-1 SAR with Deep Learning

This repository contains the source code (tensorflow) and pre-trained model corresponding to the paper:

Lake Ice Detection from Sentinel-1 SAR with Deep Learning (presented at the ISPRS Congress, 2020, virtual conference)
by Manu Tom, Roberto Aguilar, Pascal Imhof, Silvan Leinss, Emmanuel Baltsavias and Konrad Schindler

This work is part of the Lake Ice Project (Phase 2) funded by MeteoSwiss in the GCOS Switzerland framework. Here is the link to Phase 1 of the same project.

Note that:

• Our Sentinel-1 SAR semantic segmentation system is based on Deeplab v3+.
• Rasters corresponding to Sentinel-1 SAR were downloaded from Google Earth Engine (GEE)

Pre-trained model

Download the Deeplab v3+ network weights (~105MB) pre-trained on our SAR dataset.

Dependencies

The required libraries can be installed by running

pip install -r requirements.txt

General syntax for running our model

• cd models/research/deeplab
• bash train_and_val_sar_ice.sh <dataset_name> <experiment_name> <network_type> <validation_mode>

More details on <dataset_name>:

A sample dataset "2016_17_2017_18_vv_vh_sils_silvaplana_stmoritz_2016_17" is provided in:

models/research/deeplab/datasets/sar_ice

Its name follows this convention:

<winter1>_<winter2>_<band1>_<band2><lake1>_<lake2>_<lakeN>_<validationset>

It was generated with the script

data_scripting/preprocessing_sar.ipynb

and is split in the following folders:

• dataset_2016_17_2017_18_vv_vh_sils_silvaplana_stmoritz_2016_17
  • ImageSets: training and validation file list
  • JPEG images: RGB composite with SAR images, VV -> red, VH -> green
  • SegmentationClass: pixel-wise ground truth. non-frozen -> red, frozen -> blue, background -> white
• tfrecord_2016_17_2017_18_vv_vh_sils_silvaplana_stmoritz_2016_17

Options for <network_type>:

mobilenet_v2, Xception-65

Options for <validation_mode>:

V (Testing only), TV (Training and testing)

To reproduce our results of leave one winter out experiment (train on w2017-18 and test on w2016-17) without training the network:

• Download the dataset from GEE platform (we cannot provide the dataset due to copyright reasons) and extract it in: models/research/deeplab/datasets (Note that, we processed all the available data from the beiginning of September till end of May in each winter)
• Download the Deeplab v3+ model (~105MB) pre-trained on our SAR dataset.

Run:

bash train_and_val_sar_ice.sh 2016_17_2017_18_vv_vh_sils_silvaplana_stmoritz_2016_17 2016_17_regionsils mobilenet_v2 V

Folder structure: - Input weights shall be placed in train subfolder - Output visualization images will be stored in vis subfolder

Takes ~30 min on GPU Quadro M1200

Results are stored in models/research/deeplab/datasets/sar_ice/exp_2016_17_regionsils

Note: An entry with the name and size of 2016_17_2017_18_vv_vh_sils_silvaplana_stmoritz_2016_17 dataset has been already added to the models/research/deeplab/datasets/segmentation_dataset.py file. In case of adding a new dataset, an entry needs to be included in this file.

To train a model

bash train_and_val_sar_ice.sh 2016_17_2017_18_vv_vh_sils_silvaplana_stmoritz_2016_17 2016_17_regionsils mobilenet_v2 TV 0.001 40000 1 8 129

---

Training parameters:

• Dataset: 2016_17_2017_18_vv_vh_sils_silvaplana_stmoritz_2016_17
• Output folder (same as experiment name): 2016_17_regionsils
• Network: mobilenet_v2
• TV: validation mode
• Initial learning rate: 0.001
• Number of steps: 40000
  • This hyperparameter needs to be tuned when adding a new lake
• Atrous rate: 1 -> it generates a [1,2,3] value
• Batch size: 8
• Patch size: 129 -> it generates a 129x129 pixels window

Takes ~8 hours on GeForceGTX1080Ti

Downloading from Google Earth Engine

An example code to download the images of lake Sils from winter 2017-18 (with VV polarisation) is available on: https://code.earthengine.google.com/?scriptPath=users%2Frobertoaguilar%2Flakeice%3Asentinel1. You will need to create a GEE account.

• To download several images, run the script: data_scripting/gee_browser.js on the browser once GEE tasks are ready (after running the downloading script).

Citation

Kindly cite our paper, if you use this project in your research:

@article{tom_aguilar_2020:isprs,
      author = {Manu Tom and Roberto Aguilar and Pascal Imhof and Silvan Leinss and Emmanuel Baltsavias and Konrad Schindler},
      title = {Lake Ice Detection from Sentinel-1 SAR with Deep Learning},
      journal = {ISPRS Ann. Photogramm. Remote Sens. Spatial Inf. Sci.},
      year = {2020},
      volume = {V-3-2020},
      pages = {409--416},
}

Useful links

1. Multi-sensor lake ice monitoring with machine (deep) learning. Project github page.
2. Lake ice monitoring with webcams and crowd-sourced images with Deep-U-Lab. Github repo (tensorflow code, pre-trained model).
3. Tom, M., Suetterlin, M., Bouffard, D., Rothermel, M., Wunderle, S., Baltsavias, E., 2019. Integrated monitoring of ice in selected Swiss lakes. Final Project Report
4. Tom et al. 2020, Ice monitoring in Swiss lakes from optical satellites and webcams using machine learning, MDPI Remote Sensing, 12(21), 3555
5. . Tom, M., Wu, T., Baltsavias, E., Schindler, K., 2021. Recent Ice Trends in Swiss Mountain Lakes: 20-year Analysis of MODIS Imagery. Paper
6. . Tom, M., Jiang, Y., Baltsavias, E., Schindler, K., 2021. Learning a Sensor-invariant Embedding of Satellite Data: A Case Study for Lake Ice Monitoring. Paper
7. . Tom, M., Suetterlin, M., Bouffard, D., Rothermel, M., Wunderle, S., Baltsavias, E., 2019. Integrated monitoring of ice in selected Swiss lakes. Final Project Report
8. Tom, M., Baltsavias, E., Schindler, K., 2020. Integrated Lake Ice Monitoring and Generation of Sustainable, Reliable, Long Time-Series. Final Project Report

Licence

MIT License

Copyright (c) 2020 ETH Zurich

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Author: Manu Tom