An improved U-Net model with multiscale fusion for retinal vessel segmentation

2.1. Medical image segmentation

Deep learning has gradually become a commonly used technology for medical images segmentation ^[14]. Among them, U-Net has a simple but effective structural design and good applicability for domain-specific tasks such as medical images ^[15,16]. In addition, it can achieve better segmentation results with relatively less labeled data and training time. Currently, most segmentation methods based on deep learning use the encoder-decoder architecture. On this basis, the segmentation effect is improved by introducing attention mechanisms and optimizing the loss ^[17].

Some representative models based on U-Net are introduced as follows. Ahmad et al. ^[18] developed MH-UNet, which is a new architecture for medical image segmentation. It combines multiscale information and expands the receptive field size of feature maps. For example, Deng et al. used ELU-Net, which is an efficient lightweight U-Net where the deep skip connections have the same massive skip connections as the encoder for comprehensive feature extraction^[19]. Nguyen et al. proposed a cascaded context module (CCM) and a balanced-attention module (BAM)^[20]. Xiang et al. proposed a network that adds a multiscale feature extraction (MFE) module while reducing the number of convolutional kernel filters per layer^[21]. It enhances the sensitivity of the model.

Those models above show that U-Net has become one of the most popular deep learning frameworks. However, it should be further studied in retinal vessel segmentation.

2.2. Retinal vessels segmentation

Deep neural networks demonstrate significant potential for retinal vasculature segmentation ^[22]. But the vessels are characterized by a complex structure, weak texture, and low edge contrast. Therefore, precise retinal vessel segmentation continues to be a difficult undertaking ^[23].

Scholars have conducted many studies to improve segmentation. Some researchers adopted the strategy of integrating multiscale features. For instance, Ye et al. proposed a U-shaped network, which enhances scale features and preserves more vessel details during decoding^[24]. In order to learn adaptive weights to improve vessel features, Shi et al. computed the multiscale force field to capture its semantic relationship with the mask image pixels^[25]. Zhou et al. divided the fundus image into plaques of different sizes and designed a cascaded dilated spatial pyramid pooling (CDSPP) module in the model to enhance the multiscale features and vessel continuity^[26]. Despite the fact that multiscale fusion has many advantages and achieves respectable segmentation performance, they are nevertheless constrained by the following issues. On the one hand, multiscale fusion makes the network structure more complex, which reduces the interpretability of the model and makes it difficult to intuitively understand the exact process of segmentation of retinal images by the network. On the other hand, the introduction of multiscale fusion increases the number of parameters of the network, which requires more training data for adequate training, leading to the problem of over-fitting.

In retinal vessel segmentation, some scholars used attention mechanisms to improve models. For example, Guo et al. introduced a spatial attention module and replaced the original convolutional blocks of U-Net with dropout blocks to achieve network lightweighting^[27]. Su et al. proposed a cross-scale attention (CSA) block to model multiscale interactions within vascular regions, thereby extracting rich contextual semantics^[28]. Dong et al. proposed CRAUNet^[29]. In CRAUNet, dropblock regularization is introduced to reduce over-fitting. Mahmoud et al. extended the Residual U-Net architecture through a cross-attention U-Net framework, enhancing network adaptability and contextual comprehension capabilities to address critical limitations inherent in residual U-Net designs^[30]. Wang et al. proposed a multi-attention method based on directed graph search^[31]. By building a directed graph representation of the vessels, the method improves the topology and spatial connectivity. Although the introduction of the attention mechanism has become a mainstream method, the existing approaches are still unable to properly balance the global information capture of vessel structure and the segmentation accuracy of fine vessels.

To address these limitations, a morphological optimization strategy is proposed at the image processing stage. When processing retinal vessel images, the noise points in the images are reduced and the edge burrs of vessels are minimized, resulting in smoother and more continuous boundaries. In addition, the PSP structure and dual attention module are integrated in our proposed model. This allows the model to enhance the detailed features of vessels while capturing and fusing information at different scales, improving segmentation performance and reducing redundant information.

3.1. Image processing

The fundus images may be affected by lighting conditions, lens quality and other factors, resulting in poor image quality or insufficient contrast. Image processing is a crucial routine step in medical image segmentation, which can enhance image quality and contrast. In this paper, fundus images are processed using grayscale conversion, CLAHE, normalization, Gamma correction and morphological processing. Figure 2 presents the steps used for image pre-processing of input images.

Figure 2. Image processing steps of the proposed method.

At first, the input color images are converted to grayscale. Converting color images to grayscale for vessel segmentation does not entail complete information loss. Instead, it simplifies input data by reducing redundant color channels, allowing the model to focus on critical luminance and contrast properties essential for vascular structure identification. Grayscale imaging further enhances vessel boundaries and structural details. Additionally, this approach reduces computational burden by streamlining data processing, thereby improving training efficiency and model performance. Normalization adjusts the value of each pixel to standardize. CLAHE enhances the contrast of fundus images ^[32]. Gamma correction strengthens the morphology of vessels by adjusting the brightness of images. Morphological processing removes fine noise from an image by convolving its structural elements. It makes the vessel boundaries clearer. Retinal vessels face the problem of limited data availability. After completing dataset pre-processing, we implemented data augmentation on the retinal fundus image collection. Our augmentation pipeline applied deterministic geometric transformations, specifically 90°, 180°, and 360° rotations combined with flipping, generating six distinct variants per original image. Additionally, all augmented images underwent standardized partitioning into $$ 48\times48 $$ patches prior to model input, mitigating overfitting risks.

3.2. Network architecture

Figure 3 presents the structure of the proposed model, which is a U-shaped network. Considering the problem that the dataset images are small, the data features are also relatively small. In order to fully compute the image edge feature information, we perform padding around the image before the convolution calculation. In this way, the edge features will not be under-computed due to the positional problem even after the convolutional computation. However, as the network becomes more complex, the problem of vanishing gradients often arises, worsening the learning process. Therefore, we add a normalization operation Batch Normalization (BN) after the convolution operation, so that the distribution of features is brought back to the sensitive region of the function. The BN operation can turn the original distribution of eigenvalues of the data after each layer of convolution operation into a standard normal distribution, so that the distribution of features of the data can always be in the sensitive region of the activation function, and the loss function can also have a large change in the process of calculation ^[33]. In the decoder, we also add padding operation and BN operation. In addition, to capture global context information through pooling operations at different scales, we add a PSP structure after each up-sampling layer. The acquired information is spliced with the features of the current layer to improve image performance. In our model, a DAB module is added before the output layer. DAB is used to dynamically adjust the weights of the network so that it focuses on fine vessels. It can adjust its attention to changes in fine vessels based on the spatial location of the pixels. This can help to improve segmentation results.

Figure 3. Network structure of the proposed model.

3.3. PSP structure

The PSP is shown in Figure 4. Using average pooling kernels such as $$ 1\times1 $$, $$ 2\times2 $$, $$ 3\times3 $$ and $$ 6\times6 $$, we get pooling feature maps of different scales. Then, we up-sample the low-dimensional feature map by bilinear interpolation to make it the same scale as the originals. Finally, we splice it together to form the final global feature. The merge operation improves the global utilization information. The whole process is fusing the detailed features of retinal vessels with the global features containing contextual information.

Figure 4. The structure of the PSP module. PSP: Pyramid pooling structure.

3.4. DAB module

To enhance the extraction of fine vessel features, this paper proposes a DAB to enhance fine vessel segmentation in retinal images. The feature map is decoded using DAB and the image size is gradually adjusted to produce the results. The structure of DAB is shown in Figure 5. The channel attention mechanism enhances the feature response of the important channels, operating independently of input modalities by dynamically recalibrating feature channel responses in the input tensor. This enables the network to amplify critical features while suppressing less informative ones through learned inter-channel dependencies. Within the DAB module, single-channel features are processed identically to multi-channel inputs, applying channel-wise recalibration to enhance discriminative feature representation, allowing the model to distinguish the fine vessels from the background more effectively. The spatial attention mechanism captures local features of fine vessels by enhancing the feature responses of important spatial locations, and more accurately localizes and segments fine blood vessels.

Figure 5. The structure of the DAB module. DAB: Dual attention block.

In Figure 5, we use $$ 1\times1 $$, $$ 3\times3 $$ and $$ 1\times1 $$ convolution for feature extraction. On this basis, the channel attention module flexibly adjusts the weights and suppresses irrelevant information, which can be given as:

where $$ I $$ is the input feature image, $$ * $$ denotes element-wise multiplication operation, $$ \sigma $$ represents the Sigmoid function, $$ LP $$ indicates the weighted perceptron, $$ I_{\text {Avg }}^{c} $$ and $$ I_{\text {Max }}^{c} $$ are the average pooling and maximum pooling feature map outputs of $$ I $$ in spatial dimension, respectively, and $$ I_{c} $$ stands for the output.

At last, the enhanced results are fed into the spatial attention module to improve feature representation and suppress interference information, which can be given as:

where $$ I_{\mathrm{Avg}}^{s} $$ and $$ I_{\mathrm{Max}}^{s} $$ are the outputs of $$ I_{r} $$ in the channel dimension, respectively. $$ \left[I_{\mathrm{Avg}}^{s}; I_{\mathrm{Max}}^{s}\right] $$ is the output of the channel dimension. $$ I_{s} $$ will get the final output.

3.5. Loss function

Since the binary cross entropy loss function can better optimize the model in the case of class imbalance, we choose it as the loss function of our model.

where $$ y_{i} $$ represents the true probability and $$ p_{i} $$ denotes the probability that pixels $$ i $$ is predicted. $$ N $$ signifies the total number of pixels.

4.1. Datasets and experimental configuration

The model proposed in this paper is verified on three public datasets, namely DRIVE, CHASE-DB1, and STARE ^[34,35]. (1) DRIVE: DRIVE is a widely used dataset. It includes 40 images, each with a size of $$ 584\times565 $$ pixels and is detailed and manually labeled. Of these images, 20 are used for training and 20 for testing; (2) CHASE-DB1: CHASE-DB1 includes 28 images, which include expert-labeled vessel segmentation results. Each image has a size of $$ 999\times960 $$ pixels, with 16 used for training and 12 for testing; (3) STARE: STARE contains 20 images from 10 different patients. Each image is a size of $$ 700\times605 $$ pixels. Of these images, 14 are selected for training and 6 are used for testing. Some examples of these datasets are shown in Figure 6. Mask in Figure 6 refers to binary matrices that designate pixel-wise spatial localization of Regions of Interest, where foreground pixels are encoded as 1 and background regions as 0. Specifically for vasculature segmentation tasks, these masks usually indicate the true location of blood vessels. Through paired utilization with original fundus images, masks enable the model to learn discriminative feature representations for identifying and segmenting targeted specified structures or pathological regions, such as retinal vessels and lesion areas.

Figure 6. Original image, label image and mask image of three datasets.

Our model is built based on the Tensorflow. Experiments are implemented on a NVIDIA GeForce GTX 1050 Ti platform using Keras (Tensorflow as the backend). During hyperparameter optimization, we conducted systematic experiments across varying epoch counts (50, 100, 150) and batch sizes (16, 32, 64), aimed at methodically assessing training duration effects. Based on comprehensive comparisons of the critical relationships between hyperparameters (such as segmentation accuracy and convergence rate) and model efficacy, we selected a batch size of 16 with 200 epochs and a 0.0001 learning rate using the Adam optimizer to balance accuracy and efficiency. To ensure equitable evaluation across all compared methods, we maintained rigorous consistency in experimental conditions: identical dataset partitions, standardized evaluation protocols, and uniform hardware environments. Each method underwent hyperparameter optimization within identical computational constraints. This framework explicitly minimizes confounding factors, guaranteeing that performance differentials solely reflect algorithmic capabilities rather than experimental variability.

4.2. Evaluation metrics

In order to more accurately evaluate our model, accuracy ($$ ACC $$), sensitivity ($$ SE $$), specificity ($$ SP $$), F1 score ($$ F1 $$), and area under the ROC ($$ AUC $$) metrics were used as most of the references in this field.

$$ ACC $$: $$ ACC $$ indicates the proportion that the model correctly predicts, as given in

where $$ TP $$ (True Positive) means that the prediction algorithm segmentation result is the same as the total target vessel pixels manually annotated by experts; otherwise, it is $$ FP $$ (False Positive); for the background pixels, $$ TN $$ (True Negative) denotes that the total prediction algorithm segmentation result is the same as that manually annotated by experts; otherwise, it is $$ FN $$ (False Negative).

$$ SE $$: $$ SE $$ indicates that the model can correctly identify vessels, as given in

$$ SP $$: $$ SP $$ has the effect of correctly identifying negative class samples, as given in

$$ F1 $$: $$ F1 $$ score can combine the accuracy and completeness, as given in

$$ AUC $$: $$ AUC $$ is used to evaluate the performance of the model in classifying vessels and non-vessels.

4.3. Experimental results

In the detection process, we input the pictures and masks of the test set into the model, and the size of each picture is $$ 48\times48 $$. For each pixel point of the prediction result picture, there is also a pixel point corresponding to a pixel point in the real-value mask picture. By comparing these two pixel points, we utilize the confusion matrix to determine whether the pixel point of the prediction value is in the results. At the same time, the confusion matrix of various types of pixel points is counted and calculated, and finally the evaluation index results are given. The segmentation results on the three public datasets are shown in Tables 1-3.

First, we evaluated the model on the DRIVE dataset. There are many vessel ends in the low-contrast region of the image. It is critical to accurately segment the end of vessels. In Table 1, our model achieves 96.78%, 81.33%, 98.14%, 82.44% and 98.62% on $$ ACC $$, $$ F1 $$, $$ SP $$, $$ SE $$ and $$ AUC $$, respectively. The observed $$ F1 $$ score of the proposed model reflects a deliberate design choice, which prioritizes detection rate and structural continuity of fine vasculature, compromising precision to mitigate vascular fragmentation and enhance clinical utility. Among them, our method obtained the best in $$ ACC $$, $$ SP $$, $$ SE $$ and $$ AUC $$. This indicates that even in low-contrast regions, our proposed model can correctly segment the vessel ends. By introducing the PSP structure and DAB module, our model can effectively fuse multiscale features to capture relatively complete topology. It can also improve the segmentation of fine vessels.

Secondly, results conducted on the CHASE-DB1 dataset are presented in Table 2. Our model is more prominent in $$ ACC $$ and $$ SP $$, reaching 97.17% for $$ ACC $$ and 99.02% for $$ SP $$. $$ SP $$ of the proposed model is an improvement of 1.13%, 0.76% and 0.0.85% compared to the latest SDDC-Net, ETAU-Net and PGU-Net, respectively. The $$ ACC $$ of the proposed model improves by 0.48%, 0.18% and 0.59% compared to the latest SDDC-Net, ETAU-Net and PGU-Net, respectively. When compared to nnU-Net, our model performs excellently in global structure recognition and noise resistance (the $$ ACC $$ and $$ SP $$ of our proposed model increase by 0.28% and 0.38%, respectively). There is a certain gap in $$ F1 $$ and $$ SE $$ between our proposed model and some other models. This is primarily due to the attention mechanism failing to adequately focus on low-contrast vessels, and morphological processing potentially damaging weak-signal vessels, leading to reduced sensitivity.

Finally, Table 3 describes comparative results based on the STARE dataset, which includes many lesions, including hemorrhages and exudative inflammation, and half of the pathologic images ^[26]. However, at the same time, the STARE dataset has a relatively high image quality and clear vascular structures. From Table 3, compared with the models in recent papers (see ^[53] and ^[54]), our method outperforms them in $$ ACC $$, $$ SE $$, and $$ AUC $$ metrics, and $$ SE $$ of our model is 3.73% higher than that of the model in ^[53] (the second best model in $$ SE $$). The main reason is that our model is allowed to focus on different regions at various spatial scales to enhance the detailed information in specific regions, due to the proposed PSP structure and DAB module. This helps the model to better distinguish different structures in fundus images and improves segmentation accuracy.

4.4. Analysis on visualization and experimental time

In order to visualize the segmentation performance, this study was analyzed through visualization. Figure 7 shows the comparison image of our model, the U-Net and the nnU-Net. Although U-Net can segment the main vessels, it performs poorly in the fine vessels, resulting in poor vessel connectivity. nnU-Net achieves marginally superior overall performance compared to U-Net, yet exhibits consistently suboptimal preservation of morphological details in fine vessel segmentation. In contrast, our proposed new model performs better. Segmentation accuracy is primarily augmented through two interconnected mechanisms. Firstly, DAB's dual-attention framework adaptively prioritizes capillary signatures via channel-wise feature recalibration while spatially modulating focus to accommodate morphological heterogeneity. Secondly, PSP's hierarchical fusion concurrently processes macrovascular context and microstructural details, establishing comprehensive multiscale representations essential for complex vasculature. The interaction between the PSP structure and the DAB module preserves more vessel details. To sum up, the proposed model performs exceptionally.

Figure 7. Comparison of the visualization results between our model, the U-Net and the nnU-Net.

To show the computation efficiency, the experimental time of the three models (our proposed model, the U-Net and the nnU-Net) on the DRIVE dataset is given out, which is listed in Table 4. The results show that our proposed model demonstrates significant efficiency advantages. U-Net requires 0.1225 seconds per image. While nnU-Net's adaptive pre-processing and cross-validation strategies enhance segmentation accuracy, these improvements incur substantially increased computational costs, yielding 0.6834 s per image. Comparatively, though introducing additional model complexity relative to U-Net, our proposed novel architecture achieves only 0.3445 s per image while maintaining competitive segmentation performance.

4.5. Ablation experiments

Among three datasets used, DRIVE is the most generally applicable dataset, and we use it as an example for ablation experiments. The proposed method has three main components, including morphological pre-processing (PRE-Pro), PSP and DAB modules. U-Net was selected as the baseline and three important components were incorporated in sequence. Table 5 presents the results.

Firstly, morphological pre-processing was applied to the baseline (U-Net + PRE-Pro). It is clear that the performance of the U-Net + PRE-Pro model exceeds that of the baseline network. The information obtained from Table 5 allows us to determine that the evaluation metrics were all improved to some extent after adding the pre-processing optimization method morphological treatment to the baseline (U-Net + PRE-Pro). Compared with U-Net, $$ F1 $$ score is improved by 0.62% from 79.01% to 79.63%, and the $$ AUC $$ is improved by 1.24%. The main reason was that the morphological manipulation resulted in smoother and clearer edges of the vessel region, which helped to reduce edge fragments and noise in the segmentation results. In addition, the morphological processing enhanced the connectivity and morphological features of the vessel structure, which made it easier for the segmentation network to recognize and segment the vessel region.

Second, we added the PSP structure (U-Net + PRE-Pro + PSP). In Table 5, the performance is improved in terms of $$ SE $$. It can be seen that since the PSP structure is able to capture and integrate feature information, it helps the network to understand blood vessels and facilitate effective feature fusion. The capacity to extract features is enhanced as compared to U-Net and superior segmentation performance can be obtained.

We import the DAB module into the model (U-Net + PRE-Pro) called (U-Net + PRE-Pro + DAB) to validate its effectiveness. The network is assisted in learning the relationships between different channels by the channel attention module, which enables it to better extract useful feature information. By enhancing the weights of key channels, the network is able to better recognize vessel structures. Thereby, $$ F1 $$ score is improved by 2.24% compared to U-Net. In addition, the spatial attention module concentrates more attention on the vessel region, which enhances the recognition of vessel structures. This helps to reduce missed detections and false negatives, which in turn improves the $$ SP $$ by 0.92% compared to U-Net.

Finally, to efficiently capture contextual information and enhance segmentation of fine vessel structures, we designed a new model referred to as (U-Net + PRE-Pro + PSP + DAB). Compared with U-Net, our model can improve the segmentation of fine blood vessels due to the DAB module. In Table 5, compared with U-Net, our method shows that all evaluation metrics are improved. $$ ACC $$, $$ F1 $$, $$ SP $$, $$ SE $$ is improved by 0.39%, 2.32%, 0.76%, 1.91%, and the $$ AUC $$ is improved by 1.46$$ \% $$. Meanwhile, compared with U-Net + PRE-Pro + DAB, the $$ SP $$ of our proposed model dropped a little bit. The main reason is that PSP aggregates too much information at different scales, resulting in fuzzy or confusing feature representations, which cause redundancy and conflict of information, thus affecting the segmentation performance.

Overall, our model shows excellent performance in retinal vessel segmentation. Specifically, the results show that each component in the model has a certain improvement on the segmentation results. After integrating the components, we also obtain the best segmentation results.

Authors' contributions

Funding acquisition: Ni, J.

Project administration: Ni, J.; Gu, Y.

Writing - original draft: Shi, J.

Writing - review and editing: Zhan, Q.; Zhang, Z.; Gu, Y.

Availability of data and materials

Publicly available datasets were analyzed in this study. These data can be found here: https://www.kaggle.com/datasets/andrewmvd/drive-digital-retinal-images-for-vessel-extraction/data, https://www.kaggle.com/datasets/buffyhridoy/chase-db1, and https://www.kaggle.com/datasets/vidheeshnacode/stare-dataset.

Financial support and sponsorship

This work was partly supported by the National Natural Science Foundation of China (61873086).

Conflicts of interest

Ni, J. is an Associate Editor of the journal Intelligence & Robotics. Ni, J. was not involved in any steps of the editorial processing, notably including reviewer selection, manuscript handling, and decision-making. The other authors declare that there are no conflicts of interest.

Ethical approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Copyright

© The Author(s) 2025.