Application of an Isolated Word Speech Recognition System in the Field of Mental Health Consultation: Development and Usability Study

Introduction

Constraints on speech recognition such as small vocabularies, specific speakers, and isolated words need to be relaxed. At the same time, there are many new problems that must be solved. First, expanding the vocabulary makes it difficult to select and build templates. Second, in continuous speech, there is no obvious boundary between each phoneme, syllable, and word, and there is a phenomenon of coordinated pronunciation that is strongly influenced by the context of each pronunciation unit. Third, different people say the same words with different acoustic characteristics. Even when the same person speaks the same content multiple times, their physiological and psychological states may differ and cause notable differences in their speech. Fourth, there is often background noise or other interference accompanying speech. Therefore, the original template matching method is no longer applicable.

There have been further breakthroughs in using speech recognition technology for various applications for smartphones. This study focused on mental health issues and investigated the interaction between smartphone software and users' mental health based on speech recognition technology. This study involves basic application research on the use of intelligent software design and speech recognition technology in the context of mental health.

Methods

Programming of a Speech Recognition System Based on VC2005 Isolated Words

In this study, C language programming was used to implement data feature extraction based on the Markov model. It was then used to programmatically realize speech recognition for specific speech instances, as well as write speech recognition functions into functions that can be called by other modules. Additionally, it was used to implement a speech recognition system foundation, and to cultivate and improve the ability of the system to consult the literature and comprehensively use new knowledge [1].

Speech recognition is essentially a pattern recognition process, one by which an unknown speech pattern is compared with known reference patterns of speech, and the best-matched reference pattern is the recognition result. Figure 1 is a block diagram of an automatic speech recognition system based on the pattern matching principle [2].

‎View this figure

Composition of an Isolated Word Speech Recognition System

The reference pattern is based on the template word unit shown in Figure 2. The main technical items of the isolated word speech recognition system are shown in Table 1.

‎View this figure

Speech Recognition Design Process

Sample Voice Collection

The standard Chinese numerals 0-9 were spoken and recorded indoors as a sample. The recording software used Microsoft Visual C++ Windows Media Player (Microsoft), with a sampling rate of 16 kHz and sampling bits of 16 bits. The voice data is stored in the .wav file format, and its audio format is Windows PCM (pulse-code modulation) [3].

Speech Signal Preprocessing

There were several elements involved in speech signal preprocessing. First, to digitize voice signals, data was extracted from the speech signal by sampling and quantizing. During data extraction, it is extremely important to master the storage form of the voice file, and to effectively extract and ascertain the meaning of each part of the data to improve the analysis of the data, and lay the groundwork for the next step.

Second, the high-frequency portion of the signal spectrum was enhanced and flattened, in order to facilitate channel parameter analysis or spectral analysis. Pre-emphasis of the speech signal is done by using the mean power spectrum and muzzle glottal excitation radiation effects; the high end at about 6 dB/octave is above 800 Hz, ie, 6 dB/octave (2 octaves) or 20 dB/decade (10 octaves). When seeking a voice signal spectrum, the higher the frequency, the smaller the corresponding component. For this reason, pre-emphasis is performed as part of preprocessing. The purpose of pre-emphasis is to flatten the signal spectrum, and hold the entire band from low to high frequency. The signal to noise ratio requirements can use the same spectrum or spectral analysis to analyze channel parameters. Pre-emphasis generally uses a first-order digital filter of the formula μ: H(Z) = 1 – μz^-1, where μ has a value close to 1, or formula y(n) = x(n) – αx(n-1), where x(n) is the original signal sequence, y(n) is the pre-emphasis sequence, and α is the pre-emphasis coefficient [4].

Third, preprocessing included endpoint detection and framed windowing. Breakpoint detection is mainly used to extract the effective part of the data. The threshold value is 0.3 (maximum value-minimum value). The speech signal is a typical nonstationary signal. In processing, a window function is generally used to intercept one segment for analysis. Part of the extracted signal is short-term stable. Another effect of windowing is to eliminate the Gibbs effect caused by the truncation of infinite sequences. Common window functions [5] are as follows:

Both the Hamming window and the Haning window belong to the generalized raised cosine function. By analyzing their frequency response amplitude characteristics, it can be found that the rectangular window has good spectral smoothing performance, but the side lobe is too high, which may cause spectrum leakage and loss of high-frequency components. The Haning window decays too quickly and the low-pass characteristics are not smooth; the Hamming window is widely used because of its smooth low-pass characteristics and because it has the lowest side lobe height [6].

Mel Frequency Cepstral Coefficient Feature Representation

The training process of Mel Frequency Cepstral Coefficient (MFCC) parameters and Pearson Linear Correlation Coefficient (PLCC) parameters was extracted, that is, state transition matrix A, mixed Gaussian distribution weight matrix C, mean vector µ and covariance matrix U. A maximum likelihood estimation was performed.

MFCC Extraction

The human ear has different perception capabilities for speech at different frequencies; this is a nonlinear relationship. Combining the physiological structure of the human ear and using the logarithmic relationship to simulate the human ear's perception of speech at different frequencies, Davies and Merenstein proposed the concept of Mel frequency in 1980 [7]. The meaning of 1 Mel is 1/1000 of the tone perception degree of 1000 Hz. The conversion relationship between Hz frequency f_Hz and Mel frequency f_Mel is as follows:

The MFCC is proposed based on the above Mel frequency concept, and its computer flow is shown in Figure 3.

‎View this figure

First, the original voice signal is pre-emphasized, and a frame of voice signal is obtained after frame-by-frame windowing. Second, the fast Fourier transform (FFT) is performed on a frame of speech signal to obtain the discrete power spectrum X (k) of the signal. Third, triangle filter center frequency f(m) and frequency response H (k) are calculated as follows:

In Equation 5, f_l and f_h are low-pass frequency filter bank coverage and high-pass frequency, respectively; F is the sampling frequency with the unit Hz; M is the number of filter bank filters; N represents the points that are FFT; B^–1 is the inverse function of Equation 6.

B^–1(b) = 700(e^b/127 – 1) (6)

Fourth, each filter produces an output spectral energy, after taking the number of coefficients so as to obtain the following set [8]:

A discrete cosine transform is used to convert S(m) to the time domain. The calculation process of the MFCC c(i) is as follows:

The curve and filter bank distribution corresponding to the MFCC's Hz-Mel scale are shown in Figure 4.

‎View this figure

HMM Pattern Matching

HMM pattern matching is a double random process evolved from Markov chains. An HMM with IV states is usually represented by λ = (A,B,π). The meaning of these parameters is explained as follows: N is the number of states of the model. An input observation sequence O = o₁,o₂,…o_T can only be in {S} at a certain moment, which is one of the N states of {S₁,S₂,…,S_N). A = {a_ij} is the state transition probability matrix defined by the following equation: a_ij = P(q_t+1 = S_j | q_i = S_i), 1 ≤ i, j ≤ N. It is an implicit Markov chain. The probability of each transition from state S_i to state S_j is only related to state S_i, and is not related to its previous state. The matrix elements must satisfy the following equation:

π = {π₁,π₂,…,π_N} is the initial probability distribution of each state, which represents the probability value that the observation sequence O = o₁,o₂,…o_T may be in each state of the model at ==========t = 1, that is, p_i = P(q₁ = S_i), i = 1,2,…,N, and it satisfies the following equation:

B is the output probability of any observation o_i in the input speech feature sequence O = o₁,o₂,…o_T in each state. It has two types: discrete and continuous. For the discrete HMM, B is a probability matrix B = {b_j(k)}, j = 1,2,…,M; where b_j(k) = P(o_k | q_t = S_j), and M is the total number of symbols in the coded symbol set and satisfies the following condition:

For the continuous HMM, B = {b_j(o)}, 1 ≤ j ≤ N and c_ji; among these, Jo is any feature vector K in the speech feature parameters, M is the number of Gaussian elements contained in each state, L is the weight of the jth state and the lth mixed Gaussian function, N is the normal Gaussian probability density function, m_ji represents the mean vector of the l mixed Gaussian element in the j state, and U_ji represents the covariance matrix of the l mixed Gaussian element in the j state, and it satisfies the following condition:

Results

Depending on different parameters of the HMM, it has different classification methods. One type of classification is to divide the HMM into two structures, ergodic and left to right, according to the transition probability matrix A = {a_ij}. The HMM experienced by each state is that any state in the model can reach all other states through a finite step; from left to right, the HMM increases with time, and the state serial number is nondecreasing. This model is divided into spanning and no spanning. The HMMs of various states are mostly used for speaker recognition, language recognition, etc. The content of speech has a strong correlation with timing. This timing can be expressed by the state relationship, so speech recognition must use the left to right HMM structure. This study is based on isolated word speech recognition, and it is not allowed to skip a certain part of the middle of a speech fragment, so the HMM structure of left to right without crossing must be used. Its state transition probability matrix A = {a_ij} must satisfy a_ij = 0, j ≠ i and j ≠ i + 1 [9].

Another classification method is to divide HMMs into continuous, discrete, and semicontinuous based on different output probabilities B. The output probability B of each state of the discrete HMM is a discrete probability matrix, and the vector of the feature parameter of the speech signal must be vector quantized before use. The output probability B of the continuous HMM is a continuous output probability density function. It has three forms: single, mixed, and differentiated Gaussian probability density function. The semicontinuous HMM is a method that combines discrete HMM and continuous HMM. This paper uses a continuous HMM.

The following problems are to be solved by the isolated word speech recognition system based on the HMM: First, how to determine an optimal state transition sequence q = (q₁,q₂,…,q_T), and calculate the output probability P(O|λ) of the observation sequence O = o₁,o₂,…o_T to the HMM, and judge the recognition result of the voice command based on this probability. Second, how to adjust the parameters that λ = (A,B,π) to maximize the output probability P(O|λ). This is a problem of parameter training of the HMM. In the process of solving the above two problems, the output probability needs to be calculated, which is another key problem that needs to be solved by this algorithm [10].

Discussion

Small Vocabulary Speech Recognition System Applied in the Field of Mental Health

Speech Recognition System and Acquisition Method

For different speech types that need to be recognized [10], the system collected data in different ways. For mobile phone software, the intelligent degree of speech recognition is completely dependent on the preset scheme. The same speaker's speech may get completely different results due to different collection methods preset by the recognition system. Therefore, for users with special voice types, the mobile phone software adopts multiple (1-3) collection methods to reduce errors.

Speech Processing System

The speech processing system mainly analyzes and processes speech to achieve the purposes of transmission, automatic recognition, and machine understanding. The analysis and processing are implemented based on the filtering, sampling, and Fourier transform algorithms; the mobile phone software runs the experimental results. The speech processing system also processes voice signals such as echo, user's voice disturbance, and voice noise to manage some typical voice transmission problems.

Establishment of Related Databases

A psychological database that contains psychological cases and current user psychological data was established. It establishes a relationship between all data in the database and uses the data dictionary to expand the function of the table to make the database design simpler. The database also needs to regularly update relevant information to better enable the software platform to provide users with mental health information. The user steps are as follows: (1) After opening the mobile phone software, the system prompts the user to fill in relevant information such as gender and age (personal information). (2) The voice chat system will conduct a human-machine voice chat, with humorous and interesting content occasionally mixed with some questions. (3) After the chat is over, the user is notified that there is a waiting time. The software system analyzes the voice chat data and further analyzes the experimental results. (4) The user is notified of the analysis result, and the software performs the first operation on the user if they have identified psychological problems. (5) The software then establishes a specific personal psychological treatment plan for users with mental disorders.

Application of Analysis Software

The experimental data showed that our mobile phone mental health software meets the requirements for accuracy, practicability, and simplicity. The software was able to realize specific operations on related data by programming, to obtain the most reliable parameters and achieve an accurate probability of the user's voice information, thereby inferring any psychological changes. The program was able to make a scientific, professional, and safe analysis of users' mental health with different personality characteristics. Using this software is convenient for users.

Conclusion

In response to the special requirements of speech recognition, the design of this software system is based on digital signal processing and uses a fast Fourier transform. Overall, the design requirements were met. However, due to time and knowledge limitations, there are still existing problems with the design, such as the incomplete treatment of environmental noise effects. There is room for improvement in this software system. This article introduces this research and factual issues such as the application of mobile phone mental health software. The software platform is quantified and modularized using user needs. It analyzes and processes specific experimental data, emphasizing that mental health software in a mobile phone is convenient.