基于Hadoop平台下的Canopy-Kmeans高效算法

Abstract

Abstract:

This paper studies MapReduce programming model under the Hadoop platform,analyzes the advantages and the disadvantages of traditional Kmeans and Canopy algorithms,and then proposes an improved Kmeans algorithm based on Canopy.The "minimum maximum principle" is used to improve the randomicity problem of Canopy-Kmeans algorithm to avoid the blindness of Cannopy.The MapReduce parallel programming method is carried out in massive news aggregation.The experiments show that this method has higher accuracy and stability than the traditional Kmeans and Canopy algorithms.

Key words: Hadoop;MapReduce;Canopy-Kmeans algorithm;clustering

CLC Number:

TP301.6

Cite this article

ZHAO Qing. Efficient Algorithm of Canopy-Kmeans Based on Hadoop Platform[J]., 2014, 27(2): 29-.

share this article

0
/ / Recommend

Add to citation manager EndNote|Reference Manager|ProCite|BibTeX|RefWorks

URL: https://journal.xidian.edu.cn/dzkj/EN/

https://journal.xidian.edu.cn/dzkj/EN/Y2014/V27/I2/29

References

［1］ANAND R,JEFFREY D U.大数据:互联网大规模数据挖掘与分布式处理［M］.王斌,译.北京:人民邮电出版社,2012.

［2］陆嘉恒.Hadoop实战［M］.2版.北京:机械工业出版社,2012.

［3］WHITE T.Hadoop权威指南［M］.周敏奇,钱卫宁,金澈清,等,译.北京:清华大学出版社,2011.

［4］ZHANG T,RAMAKRISHNAN R,LIVNY M.BIRCH:an efficient data clustering method for very large databases［C］.Proc.ACM SIGMOD Intl.Conf.on Management of data,1996:103-114.

［5］GUHA S,RASTOGI R,SHIM K.CURE:An efficient clustering algorithm for large databases［C］.Proc.ACM SIGMOD Intl.Conf.on Management of Data,1998:73-84.

［6］汪中,刘贵全,陈恩红.一种优化初始中心点的K-means算法［J］.模式识别与人工智能,2009,22(2):299-304.

［7］陈光平,王文鹏,黄俊.一种改进初始聚类中心选择的K-means算法［J］.小型微型计算机系统,2012,33(6):1320-1323.

［8］张玉芳,毛嘉莉,熊忠阳.一种改进的K-means算法［J］.计算机应用,2003,23(8):31-33,60.

［9］赖玉霞,刘建平.K-means算法的初始聚类中心的优化［J］.计算机工程应用,2008,44(10):147-149.

［10］袁方,周志勇,宋鑫.初始聚类中心优化的K-means算法［J］.计算机工程,2009,32(8)65-66.

Related Articles 15

[1]	HU Xiaorui，LUO Yanjie，MAI Yunfei. Application of Genetic Algorithm In Constant Tension Control System [J]. , 2017, 30(4): 72-.
[2]	ZHU Huaiyi. The Simulation Implementtation of CZT Spectrum Zooming Algorithm Based on NiosII [J]. , 2017, 30(4): 76-.
[3]	LIANG Yingxin，TIAN Haoshan. Improved Hybrid Algorithm Based on Bacterial Foraging and Particle Swarm Optimization [J]. , 2017, 30(4): 79-.
[4]	SONG Ping, SHAO Qing. Correction of the Correlation Similarity in Recommendation Systems Using the Normal Distribution Function [J]. , 2016, 29(10): 58-.
[5]	LI Tiantian. Servo Parameters Tuning Method for the Direct Feed Drive Based on the Improved PSO Algorithm [J]. , 2016, 29(10): 69-.
[6]	HAO Yumei. Parameters for the Lifetime Estimating Model by EM Algorithm [J]. , 2016, 29(8): 68-.
[7]	WANG Zhihao. Research on PID Parameters Based on NSGAII in a Combination of Different Indicators [J]. , 2016, 29(8): 70-.
[8]	LI Gen, WANG Yagang, ZHOU Xiaowei. A Spectral Clustering Algorithm Based on Average Density [J]. , 2016, 29(8): 74-.
[9]	JI Lili, LI Ye . Virtual Machine Cluster Optimization Based on Migration in Cloud Environment [J]. , 2016, 29(8): 117-.
[10]	XU Xiaodong, DAI Shuguang. Analysis of model for Brushless DC Motor Based on Improved WFPSO Algorithm [J]. , 2016, 29(9): 37-.
[11]	ZHANG Chen, YOU Xiaoming. Improved Quantum ant Colony Algorithm of Path Planning for Mobile Robot Based on Grid Model [J]. , 2016, 29(7): 1-.
[12]	XU Xiaolei, NIE Wenhui, CAO Fei. Prediction of the Production of Transparent Material by HAFBPS [J]. , 2016, 29(7): 22-.
[13]	ZHANG Haibing. Dynamic Autoregression Prediction Model Based on Particle Filter [J]. , 2016, 29(6): 15-.
[14]	YUAN Manman MU Pingan DAI Shuguang. Installation Error Calibration Based on Line Structure Improved ICP Optical Scanning System [J]. , 2016, 29(6): 178-.
[15]	FANG Xinxin,GONG Rubin,LI Dawei. Multi-objective Particle Swarm Optimization Algorithm Based on Cosine Distance [J]. , 2016, 29(3): 48-.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Comments

Recommended 0

No Suggested Reading articles found!

Abstract
References
Related Articles
Metrics
Comments
Recommended

TOP

Efficient Algorithm of Canopy-Kmeans Based on Hadoop Platform

PDF (PC)

Like

Knowledge