In the speech enhancement of the joint sparse dictionary, due to the similarity of the joint dictionary, the speech and noise confusion is generated in the sparse reconstruction stage, which will generate the speech distortion problem. In view of this, an objective function under the Fisher criterion is proposed in the training stage. This function contains the distinguishing constraint of speech and noise, and adjusts the weights with the balance factor related to the signal change, so as to make the confusion error as small as possible. At the same time, in order to make the objective function converge, an algorithm is designed for alternately optimizing the dictionary and sparse coefficients. The algorithm is iterated to find the needed dictionary and sparse coefficient, and completes the output of the speech dictionary and noise dictionary. A joint dictionary with dissimilarity and good discrimination performance is obtained. In the enhancement phase, the noisy speech signal is represented sparsely in the joint dictionary, and the speech amplitude spectrum and noise amplitude spectrum are estimated. Finally, combining the advantages of the Wiener filter and ideal binary mask, a new soft mask filter is proposed. The residual noise is further eliminated. Through the experiments of noisy speech with different signal-to-noise ratios (SNR), the new algorithm has high SNR and auditory perception evaluation, which verifies the effectiveness of the new algorithm in improving speech performance.
