3.1

Basic principle of generating countermeasure network

There are two main designs in Generative Adversarial Networks, which are generators to generate close-to-real images and discriminators to separate real and generated images. During Gan training, the generator and the discriminator are trained sequentially¹⁷. The goal of training the generator is to produce images that can successfully fool the discriminator. There is no ultimate winner in the game between generators and discriminators. GAN training is expected to find a Nash equilibrium between the generator and the discriminator. The discriminator is no longer able to successfully determine whether the image came from the creator and can only guess at random results. At this point, the generator assumes that it has learned a distribution that maps the noise from the training data to the real data, and can create images that confuse real and fake.

As shown in Figure 1, G represents the generator network, which is a neural network defined by parameter θ_g, and D represents the discriminator network, which is defined by parameter θ_d. In the original GAN, multilayer perceptron is used to build the network, and since DCGAN, it is mostly convolutional neural network ¹⁸. The input of the generator network is random sampling noise g obeying arbitrary distribution P_z, and the output is the mapping G(z; θ_g) from pseudo data P_z to real data distribution calculated by θ_g. The input x of the discriminator D(x; θ_d) is the real image and the generated image, and the output model considers the probability that x is the real image.

Figure 1.

Generation countermeasure network structure

The training of GAN will be a minimax game of two models G and D, and the objective function V(G, D) is shown in formula (1):

When training the discriminator, if the input is a real image, D(x) expects the output to be close to 1; If the input is the image generated by the generator, D(G(z)) expects the output to be 0, and in general, the objective function is expected to be maximized. When training the generator, the output value of the image generated by the generator after passing through the discriminator is expected to be as close to 1 as possible, that is, G(D(z)) is close to 1. At this time, the expected objective function is minimized.

In order to solve the minimum and maximum value problem of objective function 1, an iterative method must be used to train Gan, and the optimization of generator and discriminator needs to be performed in turn ¹⁹. It can be seen from equation 1 that the gradient of the generator’s objective function depends on the effect of the discriminator, so in an iteration, it is necessary to update the parameters of the discriminator first.

For any generator A, the training standard of the discriminator is to maximize V(A, e), expand equation (1) to obtain the following equation (2):

Let equation (2) take the derivative of e(x) and let the derivative be 0, and the optimal discriminator is the following equation (3):

From equation (3), it can be seen that the essence of the discriminator is to distinguish whether sample x comes from real distribution p_data or generated distribution p_g. The goal of GAN training is to make p_g = p_data. From equation (3), we can know that . At this time, the discriminator cannot distinguish which distribution the sample comes from, and can only give a probability of 1/2 for both. Bring the optimal discriminator into equation (1), and the loss function of the generator is the following equation (4):

The following formula (5) can be found:

Therefore, equation (4) can be written as the following equation (6):

Here we introduce two commonly used statistical methods to measure the similarity between probability distributions Y and U: KL divergence and JS divergence. Their definitions are as follows (7) (8):

According to equations (8) and (9), equation (6) can be further deformed into the following equation (9):

Since the divergence between the can be obtained when the two distributions are exactly the same. Therefore, the minimum value of the loss function of the generator is − 2 log 2, if and only if p_g = p_data. In addition, it can be seen that when the optimal discriminator has been obtained, the goal of the optimization generator is to reduce the JS between p_g and p_data²⁰.

3.2

Condition generation countermeasure network

CGAN consists of two different types of networks: generator networks and discriminant networks. Its structure is shown in Figure 2. Current feedback network models can learn only one type of data at a time, and multi-class batch models must learn in layers. Therefore, the model has the disadvantage of low performance, while the CGAN model adds the same condition to the generator and the discriminator, so GAN can generate different information ²¹. The generator used in this paper has 6 transpose convolution layers, 5 ReLU layers, 5 sets of normalization layers, and a Tanh layer at the end. The discriminant network consists of 6 convolutional layers, 5 leaky ReLUs and 4 sets of normalization. Compared with traditional GAN, CGAN adjusts its total loss as Equation (10).

Figure 2.

Condition generation countermeasure network structure

3.3

Improved U-Net network structure design

The improved U-Net is a network structure obtained by minor structural adjustment on the basis of u-net. In general, it still adopts a symmetrical encoding and decoding structure and uses jump connection ²². The model input is the preprocessed 512x512 lung CT image, and the number of channels is 1. The left side of the Figure is the encoder structure. First, the operation flow of “3x3 conv + ReLU”, that is, “3x3 convolution layer + correction linear unit” is performed twice continuously on the input data ²³. Then execute the downsampling process for four times, of which the mode of the first three downsampling is the same. First, execute a 2x2 Max pooling operation with a sliding step of 2, and then execute the “3x3 conv + ReLU” operation flow twice in succession. It should be noted that the filling mode of 3x3 convolution layer adopts the same mode, which means that 0 will be used to fill the edge in the convolution process to ensure that the size of the characteristic image will not change, so the size of the characteristic image before and after convolution is unchanged. After each down sampling, the number of feature channels will increase by 16, and the size of the feature map will be reduced to half of the original. The initial number of channels set in the improved U-Net is 16.

In this paper, a BN layer and a layer with the same number of channels are simultaneously added to a two-layer residual block network, and all BN channels are replaced by residual waste blocks. Hence the original design. The remaining parts simultaneously provide layer and BN layer properties. First, the convergence speed of the model increases, and the model does not rely on a good parameter initialization process; Second, we can choose a larger study area to solve the gradient diffusion problem in the backpropagation process; In addition, the frequency of using the release process will decrease, and the capacity of this model will increase to some extent. Figure 3 shows the residual block structure based on the BN layer, and Figure 4 shows the residual block structure based on the BIN layer. The core layer consists of a 32-channel BN layer and a 32-channel inner layer. Adding a BN layer and an inner layer can not only speed up the loss but also preserve the semantic information of the training features ²⁴.

Figure 3.

Residual block structure BN layer

Figure 4.

Residual block structure BIN layer

3.4

Model of this paper

The algorithm used in this paper is divided into three steps: (1) Preprocessing. First, the collection of epidemic CT data is usually pre-processed by rotation, rotation, offset, scaling, etc. After that, the data set is leveled using improved connectivity testing. number of samples ²⁵. (2) Train the model. To reduce the spatial dimension, transform and merge operations are alternately used at the encoder to modify all the connection layers in the encoding layer and the decoding layer to better extract the hierarchical features. Both 3x3 rotation, 3x3 bin residue block, and 2x2 merge operations are used. Performs a rotation function to achieve image pre-removal. The remaining block adds the input to the result of two layers that effectively reduce the gradient loss. The purpose of using business integration is to reduce and prevent conflicts. The decoder part is symmetric with the encoder part and has two 3x3 rotations and one 3x3 in the residual block to recover the spatial dimension and lost content of the target object caused by the fusion operation. Finally, add a 1x1 rotation layer to the final category. Deconvolution can halve the number of channels and double the image quality resolution. The output of the improved U-Net network is a 32x32x1024 tensor. Finally, to perform the classification, the data is transformed into a tensor match with a smoothing function. Multilayer Perceptron (MLP) consists of two layers of 128 and 64 neurons, and the processing unit is ReLU. Finally, image classification is performed using the sigmoid layer thickness activation function and a single neuron. The network model was able to correctly classify CT images in an average of 1.79 seconds. (3) Performance Evaluation: The strengths and weaknesses of the application process will be evaluated using evaluation criteria.

4.1

Materials

The ability to generate negative bias networks is used in data processing to increase the number of samples in the data set and reduce the probability of better results. Table 1 shows the distribution of the data¹⁶.

Table 1.

Dataset distribution of CT images

4.2

Experimental design

4.2.3

Evaluation index

Five different evaluation methods, including accuracy, precision, sensitivity, F1 score, and specificity, were used to evaluate the process of diagnosing epidemic based on CT images proposed in this paper. Among them, accuracy refers to the percentage of correctly identified samples for each sample. As shown in formula (11); The truth is expressed by the formula (12), Represents the ratio of the number of identified samples to the total number of sequences to be analyzed; As shown in Equation (13), the sensitivity, i.e., the recovery rate, is used to measure the sample recognition; There can be both sensitivity and accuracy. derived from the confusion matrix. The F1 score is a combined assessment of precision and sensitivity as shown in Equation (14); Exceptions refer to the proportion of each negation word as shown in Equation (15); AUC stands for “area under the ROC curve”. Compared to ACC, AUC is less sensitive to group differences.

Among them, SD is the number of correctly classified positive examples, QW is the number of correctly classified positive examples to negative examples, LM is the number of incorrectly classified positive examples, and CN is called the false positive rate. Negative samples refer to the number of misclassifications, known as the false negative rate.

4.3

Result analysis

Five-way cross-validation is used to test the performance of the model, and the training data is divided into five parts. Of these five parts, four parts were selected as training methods and the rest as validation methods. This process was repeated five times to train and evaluate the performance of the proposed model in this paper. This paper compares the (BUF-Net) algorithm with several algorithms reported in other literature to learn more about the performance of the proposed model. Table 2 shows the performance indices (sensitivity, precision, F1 value and sensitivity) of each model. It can be seen that the accuracy and precision of the model presented in this paper are 97.1% and 93.1%, respectively, and the precision value is not very good compared to other models. Additionally, most other models do not use Grad-CAM technology to view distribution results.

Table 2.

Comparison between BUF-Net algorithm and other algorithms

A comparison of the confusion matrix between the original dataset a and dataset B after data refinement is shown in Figure 5 and Figure 6.Figure 7 and Figure 8 show the convergence comparison between the multilayer perceptron model using the improved convolutional. Compared to before the addition of residual blocks, the integration speed improved from round 4 to round 18, and the integration speed remained unchanged after iteration until round 26. Figures 9 and 10 show the comparison between the original data and the improved data design and other classical methods. The results show that AlexNet version 1 has the highest rate of 87.9%. This metric refers to the ability to accurately identify epidemic CT. GoogleNet has good functionality and adds a little extra after data processing. Compared to the pre-development data, the accuracy of VGGNet16 is very good. In this paper, the proposed model is improved on four parameters: precision, accuracy, sensitivity, and F1 index. From all the test results, it can be concluded that the BUF-Net model outperforms other algorithms. After data refinement, the performance is improved and the accuracy rate reaches 93%, which is 9% better than CGAN data refinement. Therefore, it can be concluded that the case for the network has the advantage of improving information. Figure 11 shows the ROC curve of the algorithm proposed in this paper. The AUC value of BUF-Net reaches 0.932. Figure 12 shows the comparison curves of accuracy and loss during training. Use an early stop strategy for loop 40 to prevent instability. As shown in the figure, the accuracy and loss rate of 30 methods are achieved and the loss is reduced by 1.3. No change until 0.13.

Figure 5.

COVID19 confusion matrix

Figure 6.

COVID19+ confusion matrix

Figure 7.

Comparison of U-Net convergence

Figure 8.

Comparison of U-Net+bin convergence

Figure 9.

Performance comparison of COVID19

Figure 10.

COVID19+ performance comparison

Figure 11.

ROC curve of BUF-Net

Figure 12.

Model training curve of BUF-Net

In this way, the gradient of the class is calculated based on the features extracted from the deepest convolutional layer in the training model, and global averaged into the aggregation layer of the important weights involved in the decision. According to Grad-CAM, it is clear that the proposed model has a good role in the diagnosis of epidemic CT.