cover.

The swift evolution of remote sensing technologies has made high-resolution satellite imagery widely available, creating unique opportunities for the analysis and management of land resources. The Land Use and Land Cover (LULC) segmentation, a prominent application of such imagery, is critical in land management domains, including urban planning, agricultural monitoring, environmental conservation, and disaster response. Precisely segmenting satellite images into distinct land cover types—such as urban areas, forests, water bodies, and agricultural lands—is vital for well-informed decision-making in these fields [1]. This need has spurred increasing efforts toward developing reliable and efficient LULC segmentation models that can address the inherent complexities of the task. As the demand for accurate, high-resolution LULC maps grows, particularly with the increasing availability of satellite data, new methodologies and innovations in image analysis are becoming indispensable to meeting these challenges [2].

Transformers, which transformed natural language processing, have also been adapted for computer vision tasks, leading to the creation of vision transformers. These models capture long-range dependencies in images, unlike traditional Convolutional Neural Networks (CNNs), which primarily focus on local regions [3]. Vision transformers have demonstrated superior performance in capturing global context and object relationships in images, making them highly valuable for tasks like LULC segmentation, where the spatial distribution and interrelationships of land types across large areas are crucial. Similarly, Segmentation Transformers (SETRs) have shown remarkable success in image segmentation tasks. However, SETRs often face limitations in handling large images due to the high computational demands of the self-attention mechanism, which compares every pixel to all other pixels, creating scalability issues. As a result, optimizing these models for large-scale image datasets, such as those produced by satellite imaging, remains a key area of active research. CNNs, while powerful, often struggle to handle boundaries in remote sensing images accurately. Traditional methods treat all boundaries the same, ignoring their diverse nature. However, boundaries in remote sensing are crucial, as they represent transitions between different land cover types. These transitions can vary widely, from sharp to gradual, and can be influenced by factors like shadows in high-resolution images. Current methods often use boundary information as an intermediate feature, limiting its impact on the final classification. It is essential to associate boundaries with specific categories to achieve optimal results. Refining boundary information and categorizing transitions between land cover types can significantly improve segmentation accuracy and preserve fine-grained details in satellite images.

To improve LULC classification, researchers have explored various deep-learning techniques. Patch-based approaches, which divide images into small segments and classify each as a unit, can introduce misclassification errors if patches contain pixels from multiple classes. This approach overlooks spatial coherence within patches, leading to inaccuracies in boundary areas, where multiple land cover types may converge. In contrast, pixel-based approaches classify each pixel individually, reducing misclassification risks and providing a more detailed, fine-grained classification of land cover types. While unsupervised learning methods, such as those utilized by Kussul et al. [4], typically require substantial training data and are constrained by their reliance on inherent image features without label guidance, supervised learning methods, as demonstrated by Torres et al. [5], have shown effectiveness with smaller datasets, offering greater flexibility and adaptability when labeled data is limited.

Most methods treat all image boundaries similarly, ignoring their diverse nature. Treating Image boundaries similarly can lead to information loss. Boundaries in remote sensing images are significant, as they represent transitions between different land cover types. These transitions vary widely, from sharp to gradual, and can be influenced by factors like shadows. Current methods often use boundary information as an intermediate feature, limiting its impact on the final classification. To improve accuracy, it is essential to associate boundaries with specific categories. Traditional methods like patch-based and pixel-based approaches have limitations. Patch-based methods can misclassify patches with multiple classes, while pixel-based methods may overlook contextual information. Unsupervised methods require large datasets, while supervised methods can be more effective with smaller datasets.

The main contribution of this paper can be summarized as the following.

Proposal of an architecture that uses a modified lightweight Vit encoder with patch embeddings and Multi-Head Self-Attention (MHSA) mechanism to capture and process multiple inputs at a time, which are connected to the Feature Pyramid Network (FPN) decoder for performing semantic segmentation using features extracted by the transformer. The FPN decoder helps in the drastic reduction of the original parameters while still maintaining the required efficiency.

The architecture leverages the power of a transformer with an attention mechanism that gives several advantages over the traditional CNN method for capturing features, as it has global context awareness, dynamic feature selection property, parallelization efficiency, and multiple scale feature integration.

The use of semi-supervised Conditional Random Fields (CRFs) has proven to be a significant step as it helps to refine the predicted segmentation masks by improving the alignment of object boundaries and reducing noise. It leverages labeled data (Supervised) and unlabelled data (Unsupervised) to enhance segmentation accuracy. This paper contributes to the semantic segmentation of satellite imagery for proper LULC mapping by proposing a lightweight model architecture.

The rest of the paper is structured as follows:

The literature survey is discussed in Section 2. Section 3 describes the methodology, including dataset, data preprocessing, and model architecture. Section 4 discusses the results and outcomes of the method used. Section 5 includes evaluation metrics used. Finally, Section 7 concludes this paper.

Towards the advancements of segmentation of satellite images, Unet has been a remarkable model for segmentation tasks. However, the proposed model, mUnet, is based on a modified U-Net architecture designed for pixel-level semantic segmentation. This model follows an encoder-decoder, or ladder-like, structure that uses convolutional layers to encode features, followed by decoding layers that reconstruct a segmented output. Compared to traditional methods, mUnet is advantageous due to its lower number of trainable parameters, making it more efficient, and demonstrates superior segmentation performance on high-resolution, 3-band FCC satellite imagery [6].

Multipath Feature Fusion Network (MPFFNet) is a novel method for precise LULC classification, especially for high-resolution satellite images. It effectively merges deep learning techniques with traditional image processing, particularly Gabor filters. By combining both strengths, MPFFNet surpasses previous methods, especially in fine-grained classification tasks, delivering superior results [7]. The study focuses on pixel-level land cover classification using satellite imagery for monitoring and change detection applications. It employs U-Net with different ResNet backbones (ResNet18, ResNet34, ResNet101) for segmentation tasks. Among the configurations, the highest mean Intersection over Union (IoU) of 85.1 was achieved with U-Net and ResNet101, highlighting its effectiveness for high-resolution segmentation and detailed land cover mapping. The paper introduces a novel approach to classify land cover accurately in medium-resolution satellite images. It leverages the power of the Swin Transformer, a state-of-the-art deep-learning architecture renowned for its ability to capture long-range dependencies within data. The Swin Transformer makes it particularly well-suited for analyzing medium-resolution images, where traditional methods often struggle due to limited spatial information. The paper incorporates an improved Swin Unet model designed explicitly for image segmentation tasks to enhance performance. Additionally, the authors integrate preprocessing, image enhancement, and spectral selection techniques to optimize the model's input data. By combining these advanced techniques, the proposed method significantly improves classification accuracy, making it a valuable tool for various Earth observation applications.

This work employs a Fully Convolutional Network (FCN-8) with VGG-16 weights for semantic segmentation of high-resolution satellite images into four LULC categories: Forest, built-up, farmland, and water. A non-overlapping grid-based approach is proposed to enhance segmentation accuracy. The FCN-8 model researches low-resolution features onto high-resolution pixel space for dense classification. Tested on the Gaofen-2 dataset, the model achieved an average accuracy of 91.0% and an IoU of 84.2% [8]. It combines a Denoising Diffusion Probabilistic Model (DDPM) for refined semantic features and a vision transformer for global context. DDPM for LULC segmentation and feature-level fusion between DDPM and Transformer for improved segmentation accuracy [9]. The model uses a diffusion-based U-Net to effectively capture multiscale features, including detailed context and edge information, making it ideal for high-resolution segmentation. It also includes a lightweight classification module with a spatial-channel attention mechanism that helps the model focus on the most important spatial and channel features. Incorporating unsupervised pre-trained components addresses class imbalance and speeds up training. This approach not only improves accuracy but also reduces complexity and training time [10].

This research aimed to develop a powerful yet efficient semantic segmentation model designed explicitly for LULC mapping. The challenge was balancing model complexity and computational efficiency while still achieving high-quality, accurate results. To accomplish this, we proposed a hybrid model architecture that combines the strengths of a lightweight transformer with a CNN decoder. The transformer component uses MHSA and a Multi-Layer Perceptron (MLP) to capture global and local spatial details within each image effectively. These layers enable the model to understand broader contextual relationships and finer details simultaneously. The CNN decoder then hierarchically processes and stacks these features, structuring them to enhance the model's ability to distinguish between different classes. Finally, the processed features are passed to the segmentation head, which produces the output mask in the desired shape, allowing for accurate LULC mapping while keeping computational demands manageable. Fig. 1 shows the workflow component required for practical model training and predictions.

In this research, we implemented a hybrid architecture for semantic segmentation, combining a modified lightweight transformer for feature extraction with an FPN decoder and CRF for post-processing. This combination enhanced segmentation quality by refining edges and improving boundary definition, resulting in smoother, more accurate masks. Query: Describe the architecture represented in Fig. 2.

Fig. 2 illustrates a deep learning architecture for image segmentation, likely applied to satellite imagery for land management. It employs a Vision Transformer Encoder to extract hierarchical features from the input image. These features are fed into an FPN Decoder, aggregating multiscale information to generate a detailed segmentation mask. Skip connections between the encoder and decoder help preserve fine-grained details, and a CRF is used post-processing to refine the segmentation output.

The result of this proposed study is represented as follows:

Table 5 highlights the performance of various segmentation models based on multiple metrics, including the Accuracy Index, Dice Coefficient, F1-score, Precision, Recall, and Accuracy. The mUnet model demonstrates a Jaccard Index 70.6, with solid Precision and Recall values (87.13 and 85.66, respectively), but lacks Dice Coefficient data. MPFFNET shows a higher Jaccard Index of 81.02 and a notable F1-score of 92.1, indicating strong performance. The Unet++ with ResNet101 improves further with a Jaccard Index of 87.1, a Dice Coefficient of 88.23, and the highest accuracy (92.06%). DDPM-SegFormer exhibits impressive results with a Jaccard Index of 83.57 and a high Precision (91.72) alongside strong Recall (90.23). The Swin-Unet Transformer model also demonstrates competitive performance, with a Jaccard Index of 85.9 and a high F1- score of 93.89. Finally, the Proposed Model outperforms all others with a Jaccard Index of 90.01, a Dice Coefficient of 92.08, and the highest Accuracy of 95.1%, showcasing its highly reliable segmentation performance across all metrics.

This study developed a hybrid deep learning model by integrating a Vision Transformer encoder, an FPN decoder, and a Semi-Supervised CRF for boundary refinement. This architecture balanced segmentation accuracy and computational efficiency, with the ViT capturing long-range dependencies through global attention and the FPN aggregating multiscale features for precise boundary delineation. The CRF further enhanced segmentation quality by refining edge boundaries, ensuring smoother and more accurate predictions. The model was trained using the Adam optimizer with a combined loss function of weighted cross-entropy and Dice loss, complemented by L2 regularization to prevent overfitting. Training results showed consistently higher metrics than testing, suggesting effective learning but highlighting potential overfitting. Despite this, the model demonstrated strong pixel-level classification capabilities, achieving high scores in accuracy, Dice coefficient, and Jaccard index, indicating robust feature extraction and prediction. Future iterations will focus on reducing overfitting through advanced data augmentation, optimized regularization techniques, and exploring transfer learning to enhance adaptability across diverse geographic landscapes. Expanding the dataset to cover more varied terrain types and challenging environmental conditions will further improve the model's generalizability, solidifying it as a comprehensive solution for semantic segmentation across dynamic real-world scenarios.

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors. All associated costs were covered by the author.