Skip to content

Open-source code for the paper "Representation Learning for Remote Sensing: An Unsupervised Sensor Fusion Approach ".

License

Notifications You must be signed in to change notification settings

descarteslabs/contrastive_sensor_fusion

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Contrastive Sensor Fusion

Code implementing Contrastive Sensor Fusion, an approach for unsupervised learning of multi-sensor representations targeted at remote sensing imagery.

Check out our paper, "Representation Learning for Remote Sensing: An Unsupervised Sensor Fusion Approach".

t-SNE embedding of representations

Datasets

Our OSM-labeled dataset can be found at https://storage.cloud.google.com/public-published-datasets/osm_example_dataset.zip?folder=true&organizationId=272688069953

Configuration and Scripts

Training and experiments are all handled through runnable scripts in csf/scripts/. To see the full configuration a script accepts, run python3 csf/scripts/some_script.py --helpfull. To pass a set of configuration defined in a file, run python3 csf/scripts/scripy.py --flagfile=csf/parameters/some_config_file.cfg.

We handle configuration and flags with abseil. All configuration can be changed with command-line arguments or config files, but we've included the settings we used for the paper in csf/parameters/.

Training

To train the model, use csf/scripts/train_unsupervised.py. The training configuration used for the paper is in csf/parameters/training.cfg.

Experiments

There are a number of experiments available to run, all configured with csf/parameters/experiments.cfg. Experiments using a trained model should be passed --checkpoint=some_checkpoint. --experiment_bands configures which bands are passed.

Experiments:

  • csf/scripts/visualize_dataset.py: Visualizes a dataset and the view-creation process. Does not require a trained model.
  • csf/scripts/plot_representations.py: Makes PCA and t-SNE plots of a model's representation of the OSM imagery.
  • csf/scripts/plot_salient_images.py: Finds key directions in representation space and the images that maximally activate them.
  • csf/scripts/run_nearest_neighbor_experiment.py: Runs the nearest-neighbor experiments discussed in our paper.

Citation

TODO

License

This code is distributed under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License (CC BY-NC-SA 4.0) license, which we provide a plaintext version of.

About

Open-source code for the paper "Representation Learning for Remote Sensing: An Unsupervised Sensor Fusion Approach ".

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages