SatMAE: Pre-training Transformers for Temporal and Multi-Spectral Satellite Imagery
Authors:
Yezhen Cong,
Samar Khanna,
Chenlin Meng,
Patrick Liu,
Erik Rozi,
Yutong He,
Marshall Burke,
David B. Lobell,
Stefano Ermon
Abstract:
Unsupervised pre-training methods for large vision models have shown to enhance performance on downstream supervised tasks. Develo** similar techniques for satellite imagery presents significant opportunities as unlabelled data is plentiful and the inherent temporal and multi-spectral structure provides avenues to further improve existing pre-training strategies. In this paper, we present SatMAE…
▽ More
Unsupervised pre-training methods for large vision models have shown to enhance performance on downstream supervised tasks. Develo** similar techniques for satellite imagery presents significant opportunities as unlabelled data is plentiful and the inherent temporal and multi-spectral structure provides avenues to further improve existing pre-training strategies. In this paper, we present SatMAE, a pre-training framework for temporal or multi-spectral satellite imagery based on Masked Autoencoder (MAE). To leverage temporal information, we include a temporal embedding along with independently masking image patches across time. In addition, we demonstrate that encoding multi-spectral data as groups of bands with distinct spectral positional encodings is beneficial. Our approach yields strong improvements over previous state-of-the-art techniques, both in terms of supervised learning performance on benchmark datasets (up to $\uparrow$ 7%), and transfer learning performance on downstream remote sensing tasks, including land cover classification (up to $\uparrow$ 14%) and semantic segmentation. Code and data are available on the project website: https://sustainlab-group.github.io/SatMAE/
△ Less
Submitted 15 January, 2023; v1 submitted 16 July, 2022;
originally announced July 2022.
SustainBench: Benchmarks for Monitoring the Sustainable Development Goals with Machine Learning
Authors:
Christopher Yeh,
Chenlin Meng,
Sherrie Wang,
Anne Driscoll,
Erik Rozi,
Patrick Liu,
Jihyeon Lee,
Marshall Burke,
David B. Lobell,
Stefano Ermon
Abstract:
Progress toward the United Nations Sustainable Development Goals (SDGs) has been hindered by a lack of data on key environmental and socioeconomic indicators, which historically have come from ground surveys with sparse temporal and spatial coverage. Recent advances in machine learning have made it possible to utilize abundant, frequently-updated, and globally available data, such as from satellit…
▽ More
Progress toward the United Nations Sustainable Development Goals (SDGs) has been hindered by a lack of data on key environmental and socioeconomic indicators, which historically have come from ground surveys with sparse temporal and spatial coverage. Recent advances in machine learning have made it possible to utilize abundant, frequently-updated, and globally available data, such as from satellites or social media, to provide insights into progress toward SDGs. Despite promising early results, approaches to using such data for SDG measurement thus far have largely evaluated on different datasets or used inconsistent evaluation metrics, making it hard to understand whether performance is improving and where additional research would be most fruitful. Furthermore, processing satellite and ground survey data requires domain knowledge that many in the machine learning community lack. In this paper, we introduce SustainBench, a collection of 15 benchmark tasks across 7 SDGs, including tasks related to economic development, agriculture, health, education, water and sanitation, climate action, and life on land. Datasets for 11 of the 15 tasks are released publicly for the first time. Our goals for SustainBench are to (1) lower the barriers to entry for the machine learning community to contribute to measuring and achieving the SDGs; (2) provide standard benchmarks for evaluating machine learning models on tasks across a variety of SDGs; and (3) encourage the development of novel machine learning methods where improved model performance facilitates progress towards the SDGs.
△ Less
Submitted 8 November, 2021;
originally announced November 2021.
