Multiscale Semantic Segmentation of Remote Sensing Images Based on Edge Optimization

MSEONet

Wubiao Huang, Fei Deng*, Haibing Liu, Mingtao Ding, Qi Yao

This repository is an official implementation of [Paper] [PDF]

---

Table of Contents

• News
• Abstract
• Dependencies and installation
• Dataset
• Usage
• Models and results
• Citation
• Acknowledgement
• Contact

---

News

• [2024-4-10] The models have been released.
• [2025-4-01] The codes have been released.
• [2025-3-19] The paper has been accepted by IEEE Transactions on Geoscience and Remote Sensing (TGRS).

Abstract

Semantic segmentation of remote sensing images is crucial for disaster monitoring, urban planning, and land use. Due to scene complexity and multi-scale features of targets, semantic segmentation of remote sensing images has become a challenging task. Deep convolutional neural networks capture remote contextual dependencies is limited. Meanwhile, restoring the image size quickly leads to under-sampling at object edges, resulting in poor boundary prediction. Therefore, this paper proposes a multi-scale semantic segmentation network of remote sensing images based on edge optimization, namely MSEONet. The decoder of the network consists of a multi-scale context aggregation (MSCA) module, a coarse edge extraction (CEE) module, and an edge point feature optimization (EPFO) module. The MSCA module is used to capture multi-scale contextual information and global dependencies between pixels. The CEE module is used for boundary extraction of multi-class coarse segmentation results. The EPFO module is used to optimize edge point features during the up-sampling process. We conducted extensive experiments on the International Society for Photogrammetry and Remote Sensing (ISPRS) Potsdam 2D dataset, the ISPRS Vaihingen 2D dataset and FLAIR #1 dataset. The results show the effectiveness and superiority of our proposed MSEONet model compared to most of the state-of-the-art models. The CEE and EPFO modules can enhance the edge segmentation effect without increasing the computational and parametric quantities too much. The code is publicly available at https://github.com/HuangWBill/MSEONet.

Dependencies_and_installation

# 0. Basic environmental 
anaconda, cuda==11.1

# 1. create new anaconda env
conda create -n MSEONet python=3.8
conda activate MSEONet

# 2. git clone this repository
git clone https://github.com/HuangWBill/MSEONet.git
cd MSEONet

# 3. install torch and dependencies
pip install -r requirements.txt

# The mmcv, mmengine, mmsegmentation, torch, torchaudio and torchvision versions are strict.

Dataset

Dataset	Class	Link	Storage path
Potsdam	impervious surfaces, building, low vegetation,tree, car, background	[ISPRS]	data\Potsdam_RGB_512
Vaihingen			data\Vaihingen_RGB_512
FLAIR #1	building, pervious surface, impervious surface, bare soil, water, coniferous, deciduous, brushwood, vineyard, herbaceous vegetation, agricultural land, plowed land	[FLAIR #1]	data\FLAIR1_512

• The datasets used in the paper are all publicly available and can be downloaded and preprocessed according to the description in the paper.
• Strictly organize data according to the example data.

Usage

# train MSEONet in Potsdam
python tools/train.py --config configs/MSEONet_r101-d8_4xb4-80k_Potsdam-512x512.py --work-dir result/MSEONet_Potsdam

# train MSEONet in Vaihingen
python tools/train.py --config configs/MSEONet_r101-d8_4xb4-80k_Vaihingen-512x512.py --work-dir result/MSEONet_Vaihingen

# train MSEONet in FLAIR #1
python tools/train.py --config configs/MSEONet_r101-d8_4xb4-80k_FLAIR1-512x512.py --work-dir result/MSEONet_FLAIR1

# test MSEONet in Potsdam
python tools/test.py --config configs/MSEONet_r101-d8_4xb4-80k_Potsdam-512x512.py --checkpoint result/MSEONet_Potsdam/iter_80000_Potsdam.pth --work-dir result/MSEONet_Potsdam/result --out result/MSEONet_Potsdam/result/dataset_pre

# test MSEONet in Vaihingen
python tools/test.py --config configs/MSEONet_r101-d8_4xb4-80k_Vaihingen-512x512.py --checkpoint result/MSEONet_Vaihingen/iter_80000_Vaihingen.pth --work-dir result/MSEONet_Vaihingen/result --out result/MSEONet_Vaihingen/result/dataset_pre

# test MSEONet in FLAIR #1
python tools/test.py --config configs/MSEONet_r101-d8_4xb4-80k_FLAIR1-512x512.py --checkpoint result/MSEONet_FLAIR1/iter_80000_FLAIR1.pth --work-dir result/MSEONet_FLAIR1/result --out result/MSEONet_FLAIR1/result/dataset_pre

Models_and_results

• train log and model download

Model	Dataset	Device	Iterations	mIoU	Log	checkpoint
MSEONet	Potsdam_RGB_512	RTX4090	80000	74.51	log	download
MSEONet	Vaihingen_IRRG_512	RTX4090	80000	70.08	log	download
MSEONet	FLAIR1_512	RTX4090	80000	58.72	log	download

• Results

Table 1. Quantitative comparison results with State-of-the-art network.

Method	Potsdam			Vaihingen			FLAIR #1
	OA (%)	mF1 (%)	mIoU (%)	OA (%)	mF1 (%)	mIoU (%)	OA (%)	mF1 (%)	mIoU (%)
DANet	88.54	83.40	73.77	87.68	80.31	69.11	72.74	69.38	55.09
CCNet	88.57	83.48	73.87	87.56	80.04	68.83	73.75	70.84	56.96
ACFNet	88.38	83.66	73.89	87.56	80.06	68.80	69.21	67.50	52.53
GCNet	88.67	83.88	74.28	87.68	80.46	69.23	74.16	71.82	57.87
DNLNet	88.70	83.78	74.18	87.72	80.46	69.27	73.18	69.83	55.70
LANet	88.37	82.97	73.28	87.58	79.88	68.59	71.30	66.36	52.01
A2FPN	88.55	83.92	74.24	87.47	80.04	68.73	72.05	70.41	55.75
CGRSeg	88.41	83.57	73.80	87.67	80.32	69.02	72.48	70.26	55.63
MSEONet (ours)	88.68	84.17	74.51	87.61	81.38	70.08	74.06	72.69	58.72

Potsdam

Vaihingen

FLAIR #1

Visualization of the effect of the EPFO module on some edge points on different datasets. (a) The Vaihingen dataset, (b) the Potsdam dataset, 1-true labels, 2-coarse results before EPFO, 3-fine results after EPFO, color bar - object classes.

Citation

Please kindly cite the papers if this code is useful and helpful for your research:

@article{huang2025mseonet,
  title = {Multiscale Semantic Segmentation of Remote Sensing Images Based on Edge Optimization},
  author = {Huang, Wubiao and Deng, Fei and Liu, Haibing and Ding, Mingtao and Yao, Qi},
  journal = {IEEE Transactions on Geoscience and Remote Sensing},
  volume = {63},
  number = {},
  pages = {1-13},
  year = {2025},
  DOI = {10.1109/TGRS.2025.3553524}
}

Acknowledgement

This implementation is based on MMSegmentation. Thanks for the awesome work.

Contact

If you have any questions or suggestions, feel free to contact Wubiao Huang.