We introduce Landsat-Bench: a new benchmark for Geospatial Foundation Models (GFM) pretrained on Landsat imagery. Our benchmark contains three Landsat variants (denoted with the suffix “—L”.) of existing remote sensing datasets: EuroSAT-L, BigEarthNet-L & LC100-L.

Figure 1. Comparisons of the datasets in Landsat-Bench and their original forms. ML and MT indicate multi-label and multi-temporal datasets, respectively.

• [2025.06]: We've benchmarked additional models (Prithvi v1, Prithvi v2, DOFA, and Satlas)! Prithvi-EO-2.0 600M is the new SOTA on EuroSAT-L! More benchmarks to come soon!
• [2025.06]: Landsat-Bench has been accepted for presentation at the ICML TerraBytes Workshop!
• [2025.05]: Datasets have been released on HuggingFace! EuroSAT-L, BigEarthNet-L, LC100-L

pip install .

Datasets can be loaded using the landsatbench package like below:

import kornia.augmentation as K
from landsatbench.datamodule import LandsatDatamodule
from landsatbench.datasets.eurosat import dataset_statistics

dm = LandsatDataModule(name="eurosat", root=root, batch_size=16, num_workers=4, download=False)
dm.setup(stage="fit")

transforms = K.ImageSequential(
    K.Normalize(mean=dataset_statistics["mean"], std=dataset_statistics["std"]),
    K.Resize(224),
    keepdim=True,
)

train_dataloader = dm.train_dataloader()
test_dataloader = dm.test_dataloader()

for batch in train_dataloader:
    images, labels = batch["image"], batch["label"]
    images = transforms(images)
    ...

dm.setup(stage="test")
test_dataloader = dm.test_dataloader()
for batch in test_dataloader:
    images, labels = batch["image"], batch["label"]
    images = transforms(images)
    ...

To evaluate the performance of models on the Landsat-Bench datasets, we provide a set of embedding generation and evaluation scripts in the eval directory. These scripts can be used to compute metrics using KNN and Linear Probing evaluations directly on the embeddings of each dataset.

Precomputed embeddings for the 9 baseline methods are made available on HuggingFace here.

Our paper evaluates the performance of 9 pretrained model baselines on Landsat-Bench. We report both KNN (k=5) and Linear Probe (LP) results on extracted features of each method. For EuroSAT-L classification we report overall accuracy (OA). For BigEarthNet-L and LC100-L multilabel classification we report (micro) mean average precision (mAP). * denotes the model was pretrained on the SSL4EO-L dataset using the weights made available in TorchGeo.

To recreate the datasets please install the dev dependencies pip install .[dev] and use the following scripts in the data directory:

• The extract* scripts to obtain the metadata from the original datasets and save as parquet.

• The download* scripts to download the Landsat 8 collection imagery from the USGS and/or Planetary Computer STAC APIs.

If you use this dataset in your work please cite our paper:

@misc{corley2025landsatbenchdatasetsbenchmarkslandsat,
      title={Landsat-Bench: Datasets and Benchmarks for Landsat Foundation Models}, 
      author={Isaac Corley and Lakshay Sharma and Ruth Crasto},
      year={2025},
      eprint={2506.08780},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2506.08780}, 
}