Abstract:

Federated learning is a distributed machine learning paradigm,in which the original training sets of the nodes do not have to leave the local area and they collaborate to train machine learning models by sharing model updates.Most of the current privacy-preserving and Byzantine attack detection researches in the field of federated learning are carried out independently,and the existing Byzantine attack detection methods cannot be directly applied to the privacy-preserving environment,which does not meet the practical application requirements of federated learning.To address these problems,this paper proposes a federated learning algorithm for Byzantine robustness in a privacy-preserving environment with data non-independent and identically distributed.First,privacy protection is provided for model updates (local model gradient information) by differential privacy techniques; then the credibility is evaluated for the current state of nodes based on historical model updates uploaded by nodes; and finally,global model aggregation is performed based on the evaluation results.Simulation results show that in a federated learning environment with data non-independent and identically distributed,and with the privacy protection and Byzantine node ratio of 20%~80%,the proposed algorithm performs Byzantine node detection with both the miss detection rate and the false detection rate at 0%.Meanwhile,the time overhead of Byzantine node detection tends to linearly increase with the increase in the number of the nodes.Compared with the existing Byzantine node detection algorithms,the proposed algorithm can obtain a global model with a higher accuracy in the case of data being non-independent and identically distributed and model privacy protection.

Key words: federated learning, Byzantine attack, anomaly detection, privacy-preserving techniques, differential privacy