Facial paralysis is a kind of facial disease with the functional disorder of the facial expression muscle,and adversely affects the patients’ mental and physical health.The aided diagnosis and evaluation system of facial paralysis based on computer vision can reduce the influence of doctors’ subjective experience on the diagnosis results,and improve the accuracy and efficiency of diagnosis.However,the existing evaluation methods suffer mainly from three drawbacks:① the facial asymmetry features are extracted from the static facial images,which can not represent the asymmetrical features of facial movements;② the shallow machine learning models have their limitations on extracting useful facial features;③ it is difficult for depth models to learn the effective facial features from small-scaled videos of facial paralysis.To solve these problems,we present an automatic facial paralysis evaluation method based on self-supervised facial asymmetry learning (self-SFAL).The key idea behind our method is that the pre-trained 3D-CNNs on the pretext task are transferred to extract the patients’ facial spatiotemporal features for the downstream task of facial paralysis evaluation.With the assistance of the pretext task,the 3D-CNNs can leverage numerous videos without any labels,and can be easily pre-trained to adapt to our facial paralysis evaluation task.Furthermore,inspired by the doctor’s evaluating facial paralysis by focusing on the asymmetry of the facial muscle movements on both the patients’ whole faces and the involved facial regions,our method combines the global and local facial spatiotemporal features for the final facial paralysis evaluation.Experimental results have verified a better performance of the proposed method,with accuracy,Recall and F1 improved.
