Infra-Bench CLS

A global, open-source benchmark for critical infrastructure classification with Earth observation foundation models.

Justin Guthrie1, Edward Oughton1, Konrad Wessels1, Matthew Rice1, Isaac Corley2

1George Mason University 2Taylor Geospatial

What it is

Critical infrastructure location data is often incomplete and unevenly distributed globally, especially in developing regions. Earth observation foundation models are proposed as a way to understand the built environment more efficiently, yet they remain largely untested on the facility-scale assets that underpin a range of societal and economic functions.

Infra-Bench CLS tests them on 18,756 Sentinel-1 SAR and Sentinel-2 multispectral images of facility-scale assets at 10 m resolution, covering seven continents, with asset locations drawn from 10 OpenStreetMap classes.

Seven foundation models are evaluated with linear probing and fine-tuning, at two training set sizes (1.0× and 0.3×) and three seeds each, against a supervised ResNet-18 and a random features floor. That gives thirty conditions, all on one fixed spatial split, so training and test assets never share a spatial block.

What we found

Fine-tuning matters more than the choice of backbone. Four of the six fine-tuned foundation models beat the supervised ResNet-18 baseline of 39.2% macro F1, and none of the frozen linear probes did. The best was DINOv3 ViT-L/16 fine-tuned on the full training set at 57.9%, a 48% relative improvement over the baseline.

Results vary widely by class. Airports (85.3%), train stations (82.1%), and data centers (77.6%) score well, while many power sector classes stay difficult, ranging from 27.5% to 46.2%.

Bar chart of test macro F1 grouped by model, showing each model's linear probe and fine-tuned conditions at both data scales
Test macro F1 grouped by model. Each model shows its linear probe and fine-tuned conditions at both data scales, with hatching marking the 0.3× scale. Mean of three seeds, error bars one standard deviation.
Heatmap of per-class test F1 for each foundation model at its best condition, with macro and weighted F1 summary rows
Per-class test F1 for each model at its best condition, over ten classes, with macro and weighted F1 as summary rows. Mean of three seeds.

Paper, data, and code

Paper
arXiv:2609.09482. The benchmark, the evaluation protocol, and the full results.
Dataset
10.5281/zenodo.22118892. The curated tile dataset, archived on Zenodo.
Code
justing0909/infra-bench-cls, archived at 10.5281/zenodo.22256967. Curation pipeline, evaluation notebooks, and figure scripts.

Cite

Please cite the paper. If you use the tiles or the code, cite those alongside it.

Paper

@misc{guthrie2026infrabenchcls,
  title         = {Infra-Bench CLS: A Global, Open-Source Benchmark for
                   Critical Infrastructure Classification with Earth
                   Observation Foundation Models},
  author        = {Guthrie, Justin and Oughton, Edward and Wessels, Konrad
                   and Rice, Matthew and Corley, Isaac},
  year          = {2026},
  eprint        = {2609.09482},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CV},
  url           = {https://arxiv.org/abs/2609.09482}
}

Dataset

@dataset{guthrie2026infrabenchclsdata,
  title     = {Infra-Bench CLS: A Global Multimodal Satellite Imagery Dataset
               for Critical Infrastructure Classification},
  author    = {Guthrie, Justin and Oughton, Edward and Wessels, Konrad
               and Rice, Matthew and Corley, Isaac},
  year      = {2026},
  publisher = {Zenodo},
  doi       = {10.5281/zenodo.22118892},
  url       = {https://doi.org/10.5281/zenodo.22118892}
}

Code

@software{guthrie2026infrabenchclscode,
  title     = {Infra-Bench Classification (Infra-Bench CLS) GitHub Repository},
  author    = {Guthrie, Justin},
  year      = {2026},
  version   = {1},
  publisher = {Zenodo},
  doi       = {10.5281/zenodo.22256967},
  url       = {https://doi.org/10.5281/zenodo.22256967}
}

The team