Medical Data Feature Learning Based on Probability and Depth Learning Mining: Model Development and Validation

Introduction

Over the past two decades, there has been dramatic growth in the amount of data being generated in many areas worldwide, including health care data, sensor data, various types of user-generated data, internet data, and financial company data. Big data is emerging as the amount of data in every field grows; however, “big data” is an abstract concept that does not simply mean a large collection of data. Big data has some features that are different from data sets, and its characteristics differ from those of massive data and large data sets. Research studies examining concepts such as the Internet of Things and wearable technology have helped reduce the cost of real-time monitoring of human health, which has driven development in this industry [1]. Big data technology provides unlimited potential for efficient storage, processing, querying, and analysis of medical data. Technologies such as deep learning and machine learning simulate human thinking, assist physicians in diagnosis and treatment, provide personalized health services, and promote intelligent processes of health care applications. The development and application of the Internet of Things, wireless networks, the internet, cloud computing technology, etc., provide guarantees for the analysis, processing, and transmission of big data. In short, in the field of health care, the rapid development of big data analysis, wearable technology, artificial intelligence, Kyrgyz computing, supercomputing technology, etc., all provide possibilities for the realization and development of smart medical applications [2,3].

Medical health data is multimodal, complex data that continues to grow rapidly and contains a wealth of information. The challenges associated with medical health data include how to quickly and accurately collect medical health data and how to efficiently use high-speed networks to reliably and efficiently transmit medical health data [4]. Other challenges include the use artificial intelligence–related machine learning and deep learning techniques to extract useful information from health medical big data and the development of intelligent applications for medical staff and ordinary people. In this paper, our aims included analyzing health care data, addressing intelligent application–related issues, predicting the number of hospital outpatient visits for mass health impacts, and analyzing the characteristics of health care big data. Designing a corresponding data feature learning model will help patients receive more effective treatment and enable rational use of medical resources.

Methods

Cascaded Deep Learning Model

Model Framework

Deep learning is a process of feature extraction and combination. Through multilayer nonlinear operation combination, the model can abstract high-order semantic information as data. In practice, a cascaded multilayer operation is performed on the preprocessed raw data. Each layer consists of three sublayers: feature extraction, nonlinear transformation, and feature selection. The input of the initial layer is the pretreated original data, the input of the second layer is the output of the upper layer, and the output of the third layer is the final abstract representation of the data. In each layer, the representation characteristics of the data are extracted by feature transformation, which is generally a process of dimension increasing [5]. Compared with the input, the transformed features have certain representative characteristics; however, the number of features is much greater. Through feature selection, we reduce the dimension once, and at the same time, we choose the discriminant feature or the representation feature. Some feature selection methods are used to improve the regional adaptability of the model, such as max-pooling and mean-pooling operations in convolutional neural networks. Nonlinear transformation is usually performed in the middle of feature transformation and feature selection, and it is an important part of the framework. Nonlinear transformation can imitate the activation and inactivation of neurons. Another important application of nonlinear transformation is when linear transformation is used in the feature extraction. Multilayer linear transformation is still a linear transformation. Multilayer operation plays the same role as learning a linear transformation directly and does not play a role of layer-by-layer abstraction.

In this paper, to eliminate possible differences in measurement scales between different features, the original data were normalized by the maximum and minimum method. In the feature extraction stage, to obtain the structural information of the data, a method of feature transformation with little relationship to the domain was selected, that is, the vanishing component analysis (VCA) method was used to extract the polynomial characteristics of the data. Because the VCA method itself is a nonlinear transformation, the nonlinear operation cannot be carried out in the framework [6]. Because the dimension of the VCA input data cannot be too high, we used principal component analysis (PCA) to reduce the dimensions of the VCA input data (PCA reduces the dimension of the loss of information and can reduce the feature dimensions; therefore, it can also be regarded as a special feature selection method). In the final classification, we used a boosting algorithm, which is a method that can classify and select features; in the classification of the use of the features, we could use all the features learned at each level or only use the last level of features. At this point, our depth feature learning framework was formed, as shown in Figure 1.

‎View this figure

Characteristic Learning

According to the existing cascaded depth learning framework, we implemented a specific depth model. Therefore, we needed to study and explain the training process of the model. The cascade depth model proposed in this paper is a multilayer structure. Between different layers, the output of the upper layer is the input of the next layer. Within the same layer, a PCA dimensionality reduction stage, VCA feature transformation stage, and L_2,p – RSR feature selection stage, where RSR is regularized self-representation, are included. Each layer of the model can learn the output features of the current layer [7]. The abstract information of the different layers is different. The features of the lowest layer are closest to the original feature space. The high-level features can provide complementary information for the low-level features. We made full use of the characteristics learned from all levels and proposed an effective feature combination method. Finally, we used the boosting classifier based on the binary classification problem and extended its success to the multiclassification problem.

PCA Dimension Reduction Stage

VCA feature transformation requires stringent data space dimension control; therefore, it was necessary to reduce the dimension before the VCA feature transformation. There are two ways to reduce the dimension of PCA; one is to specify the reserved dimension directly, and the other is to set the ratio of an eigenvalue to the total sum of the eigenvalues. Proportion setting can theoretically control the retention percentage of data information; however, it is challenging to control the retained feature dimension using this method, especially when the feature dimension is difficult to control. In this model, the VCA transform will produce a large number of features, and its input space needs to strictly control the feature dimension. Therefore, we directly set the reservation dimension of the PCA transform [8]. Because different dimensions retain different information, the performance of the model and the experimental results are affected, and this effect is not positively correlated with the retention dimension. Therefore, in our experiments, it was necessary to debug the PCA dimension.

VCA Feature Transformation Stage

The VCA method can map raw data into zero polynomial space, thus playing the role of feature extraction. Using this method, it is not necessary to know the domain of data usage or other prior knowledge because as long as the input space is a real matrix, we can learn its zero space polynomial transformation representation. This transformation method can not only extract the linear features in the data samples but can also extract the nonlinear features, that is, if the first-order polynomial contains the zero space of the data, the polynomial contains the linear information of the data. Other polynomials with different numbers can extract 2 or even higher order nonlinear characteristics of data. In this model, the use of VCA for feature transformation involves two specific problems: (1) polynomial number setting and initial feature dimension setting; (2) algorithm solution using singular value decomposition with the minimum setting.

L_2,p – RSR Feature Selection Phase

VCA feature transformation will produce a large number of features, and we need to select features for many reasons. Moreover, feature selection is one of the reasons why the depth learning model is effective, as it can effectively select task-related features. The L_2,p – RSR method proposed in this paper can not only effectively select features that play important roles in the linear representation of features but can also exclude the roles of singular samples due to the use of L_2,1 norm constraint loss terms. This method is based on the self-representation property of the feature space. Any matrix space data possesses this property; therefore, it is domain-independent, which meets the requirements of the generalization ability of our model. In this model, we only needed to set the P value of L_2,p norm and regularization parameter λ value in the method. The input of the L_2,p – RSR feature selection operation is the output space after VCA transformation, and the output is the output feature of the current layer of the depth model.

Boosting Classification and Feature Selection

After features are learned, it is necessary to classify them. There are many general classification methods, of which the nearest neighbor classifier is the simplest. Support vector machine (SVM) classifiers are also widely used in research and applications, and kernel-based SVM classifiers can also solve nonlinear problems. For this model, we used a classification method with a feature selection function: the Gentle Adaptive Boosting (AdaBoost) classifier based on a pile function. The Gentle AdaBoost algorithm based on a pile function can not only classify but can also select features from the feature space, that is, it can select one feature from each feature space and classify it, and it can select features by controlling the number of weak classifiers [9]. This is in good agreement with the framework proposed in this paper. The Gentle AdaBoost algorithm can not only play a classification role but can also perform feature selection. It can also be used only as a classification method or for feature selection. Next, we analyzed and implemented the boosting classifier based on a pile function.

Brief Summary of the Discrete AdaBoost Algorithm Based on a Pile Function

For the discrete AdaBoost algorithm based on a pile function, the inputs are training sample X and tag Y, and the output is the classifier model .

Step 1: Weight matrix initialization: w_i = 1/m,i = 1, …, m

Step 2: Repeat: t= 1, 2, …, T

Step 3: For d = 1, …, n, do: (err^d, δ^d, a^d, b^d) =

Step 4: feaId = ar g min (err_d), (feaId, δ, a, b) = (feaId,δ ^feaId, a^feaId, b^feaId)

Step 5: f_t(x) = ah(x ^feaId > δ) + b

Step 6: Update: , i = 1, 2, …, m, standardization of w makes

Step 7: Repeating end

The Gentle AdaBoost algorithm based on the pile function follows the boosting algorithm framework and uses a simple classifier model:

in which the weak classifier f_m is defined as

The h function is the indicator function. represents the d dimension characteristic of the i sample. δ is the threshold (the so-called pile). a and b are parameters of the linear regression function. When learning the weak classifier, each feature of the sample learns a pile function based on the least squares, and the error value of the least squares is obtained and recorded; then, the corresponding feature is selected when the error value is the minimum. Therefore, (d, δ, a, b) can be obtained through the weighted least squares method:

After obtaining the weak classifier F_m, the weights of W are updated:

The F function is updated to F = F + f. The final classification result is sign (F(x)). The absolute value of the F(x) value provides the credibility of the classification.

Feature Combination

Through the research and analysis of each stage of the cascade depth model, we successfully constructed a multilevel depth learning model and learned multilevel features. The different layer features provide different information. The underlying feature is the closest to the real information of the data and contains the most data information; however, it contains less semantic information and is not sufficiently abstract. When a model has two or three layers, it abstracts the features at different levels, including certain semantic information. When the number of layers is higher, the model contains a higher level of abstract information. We believe that each layer of features is useful; the underlying features ensure that the data information distortion is not too great, and the high-level features can provide the underlying features with complementary structure information that the underlying features do not possess [10]. If we can effectively combine each layer of features, we will obtain a good machine learning model. Using the combination of Gentle AdaBoost feature selection and classification functions, in this section, we propose an effective method of combining each layer of features and classifying them using the combination features [11-14]. The combination of the two levels of features is shown in Figure 2.

The classifier we used in the cascaded depth model is the Gentle AdaBoost classifier with a feature selection function. Therefore, we input the features of the different layers into the classifier algorithm to classify them and achieve the purpose of combining the features of each layer. The Gentle AdaBoost classifier is in the form of . Assuming that the feature space of the first layer is X₁ and the feature space of the second layer is X₂, a strong classifier is learned from the features of the first layer. Because the classification algorithm gives the current weight W of each sample after each weak classification, it can initialize the sample weight before F₂ training by using the weight distribution W of the sample at the end of F₁ after F₁ has been generated. Then, we learn the classifier , and the final classification result is F₁ + F₂. F₁ uses only the first layer feature and F₂ uses only the second layer feature; however, the sample weight W is used in the F₁ process.

‎View this figure

Data Analysis

We used the Letter, Pendigit, and USPS data sets to conduct comparative experiments [15-18]. Specifically, the example, feature, and class labels are 20,000, 16, and 26 for the Letter data set, 10,992, 16, and 10 for the Pendigit data set, and 9298, 256, and 10 for the USPS data set, respectively. From Table 1, we can see that the data pair classification accuracy was low; we selected the lowest number, and the corresponding sequence was (31 41 45 125 128 157 164 173 304). The bandwidth of the first layer PCA was (4810 1316), that of the second layer was (358 11 13 15 20) and that of the third layer as (358 11 13 15). The classification accuracy of the selected data under one level of PCA is shown in Table 1. As can be seen from the table, the optimal value was PCA1=13. With a fixed PCA1 of 13, the second layer under different PCA2 values of classification accuracy is shown in Table 2; the optimal value was PCA2=11. Moreover. the third layer results are shown in Table 2, and the corresponding PCA3 was 13. In actual bandwidth settings, PCA reserve values can be set for several more groups because Letter data have a smaller sample size per class. Finally, each layer used the PCA reservation dimension (13 11 13) setting to obtain the total classification accuracy of Letter data under each layer. Tables 1-2 also show that the classification accuracy increases with the increase of the number of layers when the model classifies Letter data with attribute values. Figure 3 and Figure 4 show the classification accuracies of the Pendigit and USPS data sets at different layers, respectively.

‎View this figure

‎View this figure

Prediction of Hospital Outpatient Volume Based on the RNN-RBM Network Model

Prediction Model Construction

Recurrent neural networks (RNNs) and restricted Boltzmann machine (RBM) networks have strong ability to predict time series. To make use of the advantages of these two network models to predict hospital outpatient volumes, the two networks were combined to form a depth structure network RNN-RBM model. This deep neural network can describe the time dependence of high-dimensional sequences.

Among the various deformations of RBM, there is a deformation called conditional RBM. Conditional RBM is different from standard RBM in that it adds two types of connections: one is an autoregressive connection, and the other is a hidden layer connection between the previous time step and the current time step. A conditional RBM can also be trained by a contrastive divergence (CD) algorithm. This structure can be used to handle time series data effectively; therefore, it can be used to solve time series problems. The RNN-RBM model in this paper can also refer to conditional RBM. and represent the offset vectors of the visible layer and the hidden layer in the RBM model at time t, and they are updated by the hidden unit u^{t – 1} in the RNN model at time t – 1. Weight matrices W_uv and W_uh are used to connect the RNN network model and RBM network model. The bias vector can be expressed as follows:

where b_v and b_h are the initial offset values of the visible and hidden layers in the RBM network model. The RNN network model expands gradually with the time step and is used to generate the state of the hidden units in the RBM network model, which are based on the input layer v^(t) and the hidden layer u^(t) in the RNN network model. In this way, the hidden layer can only blame the activation function of the hidden unit.

u^(t) = sigmoid (b_u+W_uuu^(t-1)+W_vuv^(t))

From this, we can see that the overall process of the algorithm is:

1. The hidden unit in the RNN model is activated.
2. u^(t-1) is used in Step 1 to update the bias values in the RBM network model.
3. The parameters are updated in the RBM network model.
4. The RBM output is used as the input of the prediction layer, and the parameters are initialized randomly.
5. The backpropagation (BP) method is used to fine-tune the model from top to bottom. Error values are propagated back to the RBM and RNN network models. The weight matrices w_uv and w_uh are updated, and the RNN network models are trained to predict.

Results

The experimental simulation data were obtained from real-world hospital outpatient volume data. In the data pretreatment stage, in this paper, we extracted the outpatient volume information of each department of the hospital and performed certain statistical processing methods. In the actual processing data, the outpatient data for hospital holidays were significantly reduced; to avoid the impact of these data, we deleted the related statistical processing methods.

To better satisfy the prediction function, these data were counted according to the three time intervals of day, week, and month. To better conform to the prediction model based on the depth neural network proposed in this paper, these data were processed and expressed as a data matrix. The matrix is as follows:

in which represents the outpatient volume data of department n in interval T.

In this paper, RNN and RBM neural networks were combined to form a deep-seated neural network model, and the model was used to predict the outpatient volume of the hospital. In the actual simulation, we selected the outpatient volume data of 15 important outpatient clinics as the input of the depth model, that is, the input layer of the model was set to 15. The number of hidden layer neurons in the RNN was set to 20, the number of hidden layer neurons in the RBM was set to 30, and the output of the predicted layer was 15.

Figure 5 and Figure 6 show the prediction results of the RNN-RBM model. In the simulation experiment, the outpatient volume data were trained in different intervals (daily outpatient volume, weekly outpatient volume, and monthly outpatient volume). From the simulation results, it can be seen that the forecast of the daily outpatient volume is closer to the real value than that of weekly outpatient volume and monthly outpatient volume. The depth model created in this paper is more suitable for the daily outpatient volume forecast, and we analyzed the causes of this phenomenon. We believe that there are two reasons for this result. On the one hand, the amount of training data in the daily outpatient volume forecast model is larger; therefore, the training parameters of the model more closely fit the actual data relationship. On the other hand, the weekly and monthly outpatient volume is the cumulative daily outpatient volume; therefore, the errors caused by the prediction will gradually accumulate, and the greater the interval, the lower the prediction accuracy.

In practical applications, medical managers often require more short-term outpatient volume forecasts. Because forecasting a shorter interval of outpatient volume can provide support for medical management, this method still has certain advantages in practical applications.

We compared the outpatient volume forecasting method based on the RNN-RBM depth model with existing popular forecasting algorithms, and the results are shown in Table 3. Here, the prediction algorithm based on the auto regressive integrated moving average (ARIMA) model, the BP neural network prediction algorithm, the radial basis function (RBF) neural network prediction algorithm, and the SVM algorithm were selected.

Compared with the current popular outpatient volume prediction algorithm, it can be readily concluded that the prediction algorithm based on the RNN-RBM depth model is superior to other current prediction algorithms for the daily outpatient volume, weekly outpatient volume, and monthly outpatient volume, and the prediction accuracy is relatively high.

‎View this figure

‎View this figure

Discussion

Big data in the field of health care is an integral part of the strategic layout of national big data, and the analysis and mining of valuable information is also related to the development of national health care. At present, the problems that must be solved in the analysis and application of health care data include timely and accurate collection and acquisition of health care data as well as efficient use of high-speed networks for reliable transmission of health care–related digital, image, voice, and other information. Machine learning and in-depth learning technology related to artificial intelligence can be used to mine useful information from health care–related big data and develop intelligent applications for medical staff and ordinary people. In this paper, we studied a feature learning model of medical health data based on probabilistic and in-depth learning mining, mining information from medical big data and addressing intelligent application–related problems, and we studied the differences between medical risk assessment–related data and general big data, multimodal data feature representation, and learning–related content. Several data feature learning models are proposed to extract the relationship between the outpatient volume data and to obtain precise predictive value of outpatient volume, which is very helpful to the rational allocation of medical resources and the promotion of intelligent medical treatment.