1. Introduction[1]

Many health care applications, such as medical diagnostics, patient health monitoring, computer-aided surgery, and drug evaluation, require images with both functional and anatomical information. But any single modality of medical imaging cannot provide images with complete and accurate information about both the anatomy and function of organs. In fact, various types of medical imaging sensors provide different kinds of information that are complementary and redundant. Structural medical images, such as MRI (magnetic resonance imaging), ultrasound, and CT (computed tomography) images, represent the anatomy of organs, whereas functional medical images, such as PET (positron emission tomography) and SPECT (single-photon emission computed tomography), represent the function of organs. Medical image fusion techniques are employed to integrate the information provided by individual imaging sensors and to obtain an image with high functional and anatomical information content [1].

Each of the image fusion categories has its advantages and drawbacks. The first category, including SD (spatial domain)-based methods, has a basic ability in spatial information visualization, such as edges and spatial details [2]. Furthermore, the methods of this group are relatively fast and easy to implement. However, they do not take the local dissimilarities between the source images into account, such that they suffer from color distortion [3]. In contrast, the second category, including TD (transform domain)-based methods, has a superior color information consistency but with higher spatial distortion [3]. In addition to these general categories, fusion of medical images can be performed at three levels, i.e., the score level (or decision level), object-level (or feature level), and sensor level (or pixel level). Sensor-level fusion directly synthesizes the pixels of the source images to obtain the fused image. At this level of fusion, a perfect registration of source images is required. Object-level fusion extracts informative features of the source images (for example, by neural networks, clustering algorithms, or template methods [4]) and subsequently combines them to obtain a single fused feature vector. Score level fusion methods usually process the source images individually to extract useful information and then combine the information using a classifier or voting method. The sensor level, at which our method is performed, is the most common type of fusion level owing to the fact that it is computationally efficient. The next section reviews the literature in the area of medical image fusion at the sensor level.

During the last two years, the use of cartoon-texture decomposition in the field of image fusion has shown very promising results [5-8]. The fusion methods based on cartoon-texture analysis transfer the source images into a new space by using the cartoon-texture decomposition, in which spatial details (such as edges) and color information are discriminated properly, and thus the fusion procedure can be efficiently performed. The cartoon component reveals information about the isophotes, geometric structures, and smooth pieces of the source images, while the texture component reveals information about textures, fine details, oscillating patterns, and noise [5]. Due to the success of the cartoon-texture decomposition in different image fusion tasks, such as multi-focus image fusion and remote sensing image fusion, we use this decomposition for the medical image fusion task.

The pulse-coupled neural network, proposed by Eckhorn et al. in 1990, is a biologically inspired spiking neural network [9]. This neural network, characterized by the pulse synchronization of neurons and global coupling, was inspired by pulse bursts in monkey and cat visual cortex. The pulse-coupled neural network is a two-dimensional, laterally connected, single-layered neural network using pulse-coupled neurons, which are connected with pixels of the image [10]. This neural network is widely used in various image processing applications. Particularly, several image fusion approaches based on the pulse-coupled neural network have been proposed [11-15]. This neural network is usually employed in the transform domain, e.g., PCA (principal component analysis) [10] and shearlet [11]. According to the experimental results reported in the literature, one can conclude that pulse-coupled neural network-based fusion approaches can achieve state-of-the-art performance. However, these algorithms have some significant limitations. For example, image contrast is reduced, and sometimes fine spatial information is lost [16-18]. The main reason is that during most of the pulse-coupled neural network-based fusion methods, the neuron is motivated by only one coefficient in the transform domain. To cope with this problem, this paper exploits the EOG (energy of image gradient) of the coefficients to motivate the neurons of the pulse-coupled neural network.

The main contribution of this paper is that a novel multimodal image fusion framework is established. The proposed framework makes use of the cartoon-texture decomposition to map the source images into a new space in which spatial information and color information are discriminated from each other. The EOGs of the coefficients of texture components are computed and fed to the pulse-coupled neural network for coefficient selection. In addition, the regional energy of the cartoon components is used for fusing the cartoon components of the source images. Simulation results demonstrate that a clear fused image containing a large amount of spectral and spatial information can be produced by the proposed method. Particularly, the proposed method using cartoon-texture decomposition can achieve better performance than methods using multiscale transforms, such as contourle.

The outline of the rest of the paper is as follows. Section 2 reviews the literature in the area of medical image fusion at the sensor level. In Section 3, the principle of cartoon-texture image decomposition is briefly presented. Section 4 describes the pulse-coupled neural network model. Section 5 presents the proposed fusion framework. In Section 6, several experiments are performed to evaluate the performance of the proposed medical image fusion method. Then, the results of the experiments are analyzed to show the advantages of the proposed method. Finally, concluding remarks are drawn in Section 7.

2. Review of Fusion Methods at the Sensor Level

The following subsections review the two classes of image fusion approaches at the sensor level, namely, TD-based methods and SD-based methods.

2.1. TD-Based Fusion Methods

In the TD-based fusion methods, the source images are mapped into a new space in which the information of the source images is represented by using a joint frequency-space domain. Then, the fused image in the transform domain is obtained using a specific fusion rule, such as the maximum selection rule, which takes the activity level of coefficients into account. Finally, the fused image in the spatial domain is obtained by applying the inverse transform [19]. To this end, Ref. [20] introduced a new image fusion framework for medical images based on non-subsampled contourlet transform (NSCT). In this method, the source images are transformed by the NSCT (non-subsampled contourlet transform) to obtain low- and high-frequency components. Then, this method makes use of two different fusion rules based on phase congruency and directive contrast to obtain fused low- and high-frequency coefficients. Finally, the synthesized image is constructed by applying the inverse NSCT to all composite coefficients. An approach based on non-Gaussian statistical modeling of discrete wavelet coefficients was proposed in [21]. This approach exploits families of alpha-stable and generalized Gaussian distributions to model image wavelet coefficients. Then, this method estimates distribution parameters through the expectation maximization (EM) algorithm. Finally, this technique incorporates the model into a weighted average image fusion algorithm.

Recently, edge-preserving filters have been employed to provide multiscale representations of images. A novel edge-preserving image fusion approach was proposed by Bai et al. using the multiscale toggle contrast operator [22]. This method extracts the multiscale dilation and erosion features of the source images through the multiscale toggle contrast operator to obtain edge information of the source images. The fused image is constructed by combining the dilation and erosion features into a base image. Experimental results showed that clear and well-preserved edge features are produced by this algorithm. The multiscale geometrical bilateral filter (MGBF) transform was introduced in Ref. [23], which combines the ability of edge-preserving of the bilateral filter and that of representing directional information of the directional filter bank (DFB). This fusion method decomposes the source images into the directional subbands and approximation subbands using the MDBF. Then the fused subbands are obtained by the max fusion rule. Applying the inverse MDBF gives the fused image. A fast and effective image fusion method was proposed based on guided filtering in Ref. [24]. The guided filtering-based weighted average technique used in this fusion framework can take advantage of spatial consistency for the fusion of approximation and detail subbands. Several fusion methods were proposed based on cartoon-texture decomposition, which can be considered as edge-preserving filters [5-8]. These methods show promising results in the field of image fusion.

There are a number of fusion methods that combine different transforms to achieve superior performance. For example, an image fusion approach based on a combination of wavelet and curvelet transform was proposed in [25]. Owing to the fact that wavelets cannot effectively represent long edges and curvelets cannot effectively capture small features, combinations of these two transforms can improve the quality of fusion. Ref. [26] introduced a hybrid multiresolution method combining the wavelet transform and the NSCT to fuse images. This method makes use of some complementary characteristics between the two mentioned transforms simultaneously. In the fusion method proposed by Kavitha et al. [27], the wavelet transform was combined with the discrete ripplet transform (DRT) to obtain an efficient hybrid transform.

2.2. SD-Based Fusion Methods

A straightforward fusion technique is to obtain each pixel of the fused image by calculating the weighted average of the corresponding pixels in the source images. Usually, the activity level of pixels determines the weights. For example, the fusion methods proposed in [28] and [29] employ the support vector machines (SVMs) and neural networks to weight the source pixels with the highest level of activity. The method proposed in Ref. [30] divides the source images into fixed-size blocks and calculates the spatial frequency (SF) in each block to measure the activity level. In this way, the proposed method takes advantage of spatial context information for fusion. To further improve the fusion performance, Ref. [31] proposed a new method that adaptively determines the optimal block size by means of a quad-tree structure. Another strategy adopted to take the spatial correlation of neighboring pixels into account is the post-processing of the pixel-wise fusion map. For instance, a new method was introduced by Zhang et al. [32], which consists of three steps. In the first step, the pixels with a high activity level are detected using the Laplacian operator. Second, the KNN (K-nearest neighbor) matting method is employed to obtain a globally optimal fusion map. The third step obtains the fused image according to the optimal fusion maps.

Apart from the methods calculating a weighted average of source pixels, there are a number of total variation (TV) based methods which perform fusion in the spatial domain. Ref. [33] introduced a TV-based fusion technique in which the fusion task is considered as an inverse problem and a locally affine model is utilized as the forward model. Then, a seminorm TV-based approach, along with the PCA, is employed to iteratively calculate the fused image. Piella proposed a variational method for image fusion [34]. This method can fuse any number of images while preserving salient features. The geometry of source images is described using the structure tensor. In addition, this method combines the geometry merging of the source images with intensity correction and perceptual enhancement to provide a ‘sharp’ and ‘natural’ fused image. A general variational method for image fusion was proposed in [35], which formulates the synthesized image as a convex combination of the source images and makes use of concepts from perceptually inspired contrast enhancement criteria, such as non‐linearity and locality.

3. Cartoon-Texture Decomposition

The cartoon-texture decomposition tries to separate an image into a sketchy approximation with sharp edges and boundaries, which is called the cartoon part, and a subband with small-scale oscillatory patterns, which is called the textural component. The general scheme by which the image  is decomposed into the cartoon component denoted by  and the texture component denoted by is a minimization problem [36]:

where the values of the functions  are greater than or equal to zero for all inputs and are spaces of the functions in such a way that  and  if and only if . Furthermore,  is a trade-off parameter.

 The variational space is the appropriate one for the sparse representation of the cartoon part. Therefore, the function  is selected to be the TV of the cartoon part, which eliminates smooth features while preserving boundaries. In addition, the textural part can be represented by the L²-TV model. This means that the cartoon-texture decomposition makes use of the total variation and L²-norm to obtain the components [37]. A mathematically convex and tractable model for the cartoon-texture decomposition is as follows [38]:

where  and is the divergence of . Here, the tuning parameters  and  are greater than zero. Moreover, denotes the p-norm of  and  is the i^th element of . Fig. 1 shows a PET image and a SPECT image decomposed by this model (p is set to 1 in this figure).

The low-resolution pseudo-color image (PET or SPECT) is decomposed by Eq. (2) in our method. Here, we consider the following energy function when decomposing the high-resolution gray-scale MRI image:

where  is the cartoon component of the MRI image and  is the cartoon component of the i^th band of the pseudo-color image obtained by applying Eq. (2). The coefficients  (for i=1, 2, 3) are calculated through a linear combination of spectral bands of the pseudo-color image. To this end, we exploit the adaptive method proposed by Rahmani et al. [39] to obtain the coefficients. This new term adapts the scale of the cartoon part of the MRI image to that of the pseudo-color image. So, the proposed model for the MRI image decomposition is as follows:

where  is a trade-off parameter. In what follows, we describe the numerical algorithm used to solve Eqs. (2) and (4).

Considering that Eq. (2) is a special case of Eq. (4), we only solve Eq. (4) using the numerical algorithm. One can rewrite Eq. (4) as a separable convex optimization problem subject to linear constraints by making use of the auxiliary variables as follows:

The optimization problem in Eq. (5) is defined as the augmented Lagrangian function [40]:

where  is the Lagrange multiplier and coefficients  (i=1, 2, 3, 4) denote penalty parameters. A variable splitting approach, such as ADMM-GBS (alternating direction method of multipliers with Gaussian back substitution) [41], is used to solve this equation. Ref. [40] has analyzed the convergence of this approach.

Fig. 1. Cartoon-Texture Decomposition: (a) SPECT Image, (b) Cartoon Component, (c) Texture Component, (d) PET Image, (e) Cartoon Component, (f) Texture Component.

4. Pulse-Coupled Neural Network

The pulse-coupled neural network is a model based on the monkey’s and cat’s primary visual cortex. In the pulse-coupled neural network model proposed by Eckhorn et al. [9], a pulse-coupled neuron denoted by N_ij, is made up of three functional units (see Fig. 2): 1- the receptive fields, 2- the modulation product, and 3- the pulse generator.

Each neuron receives input signals not only from other neurons, but also from external sources by means of two channels in the receptive fields. The first channel receives the feeding input F_ij, and the second channel receives the linking input L_ij. The function of a neuron can be represented by the following equations [11]:

In fact, the pulse-coupled neural network combines the linking input L_ij and the feeding input F_ij in a second-order manner to obtain the value of total internal activity , and then compares  with the value of dynamic neuromime threshold  to obtain the output . If is sufficiently large, then the neuron fires a pulse.

The parameters ,  and  are the decay constants of the pulse-coupled neural network model, and the parameters , , and  are the magnitude scaling constants. Furthermore,  and  are the synaptic weight strengths used to weight the outputs of other neurons in the feeding receptive field and linking receptive field, respectively. In addition, the parameter  is the linking strength.

In image processing applications, the input to the F channel of the (i,j)^th neuron, is the value of its corresponding pixel, which is denoted by :

The output of the pulse-coupled neural network is an image denoted by , whose size is the same as that of the input image. The value of the (i,j)^th pixel of the image  is obtained by taking the summation of its corresponding neuron outputs over n [15]:

Fig. 2. The Pulse-Coupled Neural Network Model.

5. Proposed Method

For fusion of the MRI and PET (or SPECT) medical images, the proposed image fusion framework is as follows:

Step 1: Use the given MRI and PET (or SPECT) images as inputs, which are denoted by  and , respectively (  is the i^th band of the pseudo-color image).

Step 2: Decompose the i^th band of the pseudo-color image  using Eq. (2) to obtain the cartoon component  and the texture component .

Step 3: Calculate the non-negative coefficients  (for i=1, 2, 3) using the algorithm proposed in Ref. [39]. This algorithm minimizes the following objective function to obtain coefficients :

Step 4: Decompose the MRI image  using Eq. (4) to obtain the cartoon component  and the texture component .

Step 5: To obtain the cartoon component of the i^th fused band, the regional energy is used as the activity level measurement:

where  denotes the cartoon part (either  or ),  denotes the local energy (either  or ), and  is a 3×3 weighting window.

After calculating the local energy of the cartoon components of both the MRI and PET (or SPECT) images, the maximum selection rule is applied to select the coefficient with the highest activity measure:

where  is the cartoon component of the i^th fused band.

Step 6: For the texture components, compute the EOG as follows:

in which, denotes the texture component (either  or ).

Step 7: Use the EOG of texture components as the input of the pulse-coupled neural network and motivate the neurons according to Eqs. (7) to (11).

Step 8: Calculate the firing time of the pulse-coupled neural network (sum of the = 1 or ) at the n^th iteration, as follows:

Step 9: If the iteration number n exceeds the maximum iteration number n_max , fuse the texture components by means of the firing time calculated in the previous step.

where  is the texture component of the i^th fused band.

Step 10: Apply the inverse cartoon-texture transform to the fused cartoon and texture coefficients to obtain the final fused image:

6. Experimental Results and Comparisons

In this section of the paper, experiments on several medical source images are presented for testing the performance of the proposed fusion framework. Firstly, the datasets used in our experiments are introduced. The parameters are then tuned, owing to the fact that the parameters have a significant effect on the fusion performance of the proposed approach. Subsequently, five quality indices, applied to evaluate the performance of the synthesized images, are given. Finally, the performance of the proposed framework is presented and compared with several pulse-coupled neural network-based fusion frameworks, including Method 1, Method 2, and Method 3:

Method 1: This fusion method exploits NSCT and pulse-coupled neural network to make use of their strengths [16]. Spatial frequency in the NSCT domain is calculated and used as the input to motivate the network, and then coefficients in the NSCT domain with large firing times are chosen as the coefficients of the fused image.

Method 2: This fusion method is based on NSST (non-subsampled shearlet transform) and pulse-coupled neural network [11]. Firstly, the source images are decomposed using the NSST to obtain several subimages. Then, the regional energy is employed to fuse the low-frequency subbands. Finally, high-frequency subbands are fused by means of the pulse-coupled neural network mode.

Method 3: The discrete multi-parameter fractional random transform (DMPFRNT) and pulse-coupled neural network are exploited in this fusion technique [42]. This method converts the source images into the DMPFRNT domain. High amplitude spectrum (HAS) and low amplitude spectrum (LAS) components in the DMPFRNT spectrum domain contain different information about source images. In this fusion approach, the local standard deviation of the amplitude spectrum is selected as the link strength of the pulse-coupled neural network.

Method 4: In this method, two input images are fed into the branches of a twin convolutional neural network such that both branches share the same weights and architecture. Each one includes three convolutional layers, and the feature maps essentially act as the role of activity level measures [43].

 Two pairs of representative data sets are selected to be used in visual tests, which are medical images from the human brain:

• Dataset A

This data set consists of a pair of MRI and PET images (sagittal view), belonging to a 38-year-old man in excellent health. Both the MRI and PET images in dataset A are of size 256×256.

• Dataset B

This data set consists of a pair of MRI and SPECT (SPECT-TI) images, belonging to a 51-year-old woman with neoplastic disease (brain tumor). The type of tumor in this patient is a glioma. Both the MRI and SPECT images in dataset B are of size 256×256.

These datasets were downloaded from the website of the Atlas project, whose web address is: http://www.med.harvard.edu/AANLIB. The source images used in our experiments have been previously registered.

Without loss of generality, we empirically take , , , , , , and  in the cartoon-texture decomposition algorithm. In addition, the maximum number of iterations is set equal to 55 when decomposing the source images by the variable splitting approach. Furthermore, the parameters of the pulse-coupled neural network are set as , , , , , , and .

For visual evaluation, the fused images corresponding to “dataset A” and “dataset B” obtained by different fusion methods are represented in Figs. 3(a)-(f) and 4(a)-(f), respectively. Figs. 3(a) and 3(b) show the PET image and MRI image of “dataset A”, respectively. Figs. 3(c) to (f) represent the combined images of the first dataset given by the proposed method, Method 1, Method 2, and Method 3, respectively. Figs. 4(a) and 4(b) show the MRI image and SPECT image of “dataset B”, respectively. Figs. 4(c) to (f) represent the combined images of the second dataset given by the proposed method, Method 1, Method 2, and Method 3, respectively. To evaluate the images more accurately, two zoomed areas corresponding to the “dataset A” are shown in Fig. 5, and two zoomed areas corresponding to the “dataset B” are shown in Fig. 6. The first zoomed area contains functional information, while the second zoomed area contains anatomical information.

Compared with the reference images, the fused images obtained by the proposed method successfully preserve the informative features and combine them to generate a clearer image in all areas of the scene. For example, the pink color in the region of calcarine fissure is the same as that in the reference image (see the white arrows in Figs. 3(a) and 3(c)), and the structural information represented in Fig. 5(b) is more similar to the reference image compared to other methods. Some black artifacts exist around the fused images generated by Method 1 (see the red arrow in Fig. 4(d)). These black artifacts do not exist in the source images and thus impair image quality severely. In addition, one can see that Method 1 suffers from color distortion such that the desired color is not reproduced in the fused results obtained by Method 1. The colors in the fused images of Method 2 become darker than those of the reference images. This phenomenon is more obvious in Fig. 5(d). Furthermore, as can be seen from Figs. 5(i) and 6(i), Method 2 somewhat blurs the structural information of the MRI image. The color distortion is more severe in the fused results obtained by Method 3. For example, red colors produced in Fig. 5(e) do not exist in the reference image shown in Fig. 5(a). So, the fused results of Method 3 can mislead the radiologist in the diagnosis in clinical applications. Moreover, fusion results of Method 3 cannot meet the expectations in terms of anatomical information content (see Figs. 5(j) and 6(j)). Figs. 3(g) and 4(g) indicate that the Method cannot retain spatial information well, such that local spatial information is not clearly visible (see the white arrow in Fig. 3(g)).

To make the experiments comprehensive, two new data sets called “dataset C” and “dataset D” are used in quantitative evaluations (see Fig. 7). Quantitative assessment is essential in the field of image fusion [44]. The five indices utilized for quantitative evaluation are:

• Percentage fit error (PFE)

  When the reference image is available, the performance of fusion methods can be assessed using the PFE index:

where  is the fused image and  is the reference image. To obtain the final value, the PFE value between the fused image and pseudo-color image is calculated, and that between the fused image and the MRI image is computed. Then, the average of the obtained values is reported. This index will be zero when the reference and the fused image are the same, and the value of this index will increase when the synthesized result is deviated from the source image [45].

Fig. 3. Fusion Results of the First Dataset: (a) PET Image, (b) MRI Image, (c) Proposed Method, (d) Method 1, (e) Method 2, (f) Method 3, (g) Method 4.

Fig. 4. Fusion Results of the Second Dataset: (a) MRI Image, (b) SPECT Image, (c) Proposed Method, (d) Method 1, (e) Method 2, (f) Method 3, (g) Method 4.

Fig. 5. Zoomed Areas of the First Dataset: (a) PET Image, (b) Proposed Method, (c) Method 1, (d) Method 2, (e) Method 3, (f) Method 4, (g) MRI Image, (h) Proposed Method, (i) Method 1, (j) Method 2, (k) Method 3, (L) Method 4.

• Fusion symmetry (FS)

The index fusion symmetry FS [46] was proposed to indicate the symmetry of the fusion process with respect to the source images. This index is mathematically formulated as:

 where  denotes the mutual information between the fused image and the source image A and  denotes the mutual information between the fused image and the source image B.

• Edge-dependent fusion quality index (EFQI)

The FEQI index considers the significance of edge information of the fused image, and is formulated as follows:

where , , and  denote the edge information of , , and , respectively, and WFQI measures the amount of information of the source images, which has been transferred into the synthesized image (to see more details, see Ref. [47]). The contribution of the edge information depends on the parameter . In our experiments, the parameter  is set to 1.

• Gradient magnitude similarity deviation (GMSD)

This newly proposed index is used to evaluate the perceptual quality of images [48]. Owing to the fact that image gradients are very sensitive to image distortions, the GMSD index makes use of the global variation of a gradient-based quality index to predict the overall quality of the image. The GMSD index is computed as follows:

where is the fused image,  is the reference image, is the total number of pixels in the image, and:

Fig. 6. Zoomed Areas of the Second Dataset: (a) SPECT Image, (b) Proposed Method, (c) Method 1, (d) Method 2, (e) Method 3, (f) Method 4, (g) MRI Image, (h) Proposed Method, (i) Method 1, (j) Method 2, (k) Method 3, (L) Method 4.

Table (1): UIQI values obtained by different fusion schemes.

Table (2): PFE values obtained by different fusion schemes.

Table (3): FS values obtained by different fusion schemes.

Table (4): EFQI values obtained by different fusion schemes.

Table (5): GMSD values obtained by different fusion schemes.

Table (6): UIQI values obtained by different misalignments.

Table (7): Running time of different fusion schemes (in Seconds).

Table (8): Sensitivity analysis of different parameters (impact on UIQI index using standard deviation).

Fig. 7. Datasets C and D: (a) MRI Image of dataset C, (b) PET Image of dataset C, (c) MRI Image of dataset D, (d) PET Image of dataset D.

 in which  denotes the convolution operator. The GMSD value between the fused image and the pseudo-color image, and that between the fused image and the MRI image, are first obtained. Then, the average of these values is used as the final index.

• Universal image quality index (UIQI)

Proposed by Wang and Bovik, this index measures the similarity between the fused image and the source image by considering three factors, i.e., loss of correlation, contrast distortion, and luminance distortion [49]:

where  and  are the mean value of the source image and that of the fused image, respectively,  and  are the variance of the source image and that of the fused image, respectively, and  is the covariance between the source image and the fused image. Again, this index is obtained for the two source images, and the average of the obtained values is reported.

   Values of different objective indices obtained using various fusion methods are tabulated in Tables (1) to (5) for four different pairs of medical images. According to these tables, all the objective indices prove that the fused image obtained by the proposed method is strongly correlated with the reference images and more informative features, e.g., edges and textures are preserved during the fusion process, which means that the proposed fusion approach is the best one among the implemented algorithms. In addition, Table (6) shows the effect of different types of misalignments on the results using the UIQI index. From this table, it can be seen that misalignments can decrease the performance of the proposed method. Moreover, Table (7) reports the execution time of different fusion algorithms. According to this table, the shortest execution time belongs to Method 1, which indicates the superiority of this method in computational complexity over other methods. Finally, Table (8) reports the sensitivity of the proposed model to its parameters. As can be seen from the table, the model is more sensitive to parameters of the pulse-coupled neural network such as , , , ,  and . According to this table, the maximum standard deviation belongs to , which indicates that the model has the highest sensitivity to this parameter.

7. Conclusion

An efficient MRI and PET/SPECT image fusion scheme in the cartoon-texture domain was proposed. The proposed method decomposes the source images using the cartoon-texture transform to obtain their components. For this purpose, the ADMM-GBS technique, one of the variable splitting algorithms, is employed. Subsequently, the proposed method fuses the cartoon components based on the regional energy of coefficients. Then, the texture components are combined by means of the pulse-coupled neural network whose inputs are the EOGs of texture coefficients. The fusion performance of the proposed framework was tested and compared against some pulse-coupled neural network-based fusion methods using visual inspection, as well as objective measures. Visual analysis of the results demonstrated that the proposed approach can preserve the structural information of the MRI image and metabolic information of the PET/SPECT image with good contrast. The quantitative results obtained showed that the proposed technique preserves important features of the source images in terms of classical criteria such as UIQI and modern perceptual criteria such as GMSD.

[1] Submission date:29, 12, 2023

Acceptance date:07, 07, 2025

Corresponding author: Hamid Reza Shahdoosti, Department of Electrical Engineering, Hamedan University of Technology, Hamedan, Iran