Research and Optimization of Distributed Clustering Algorithms for High-Dimensional Data

Runfeng Zou

doi:10.47297/taposatWSP2633-456935.20250603

PDF
Export
Share
Collection
Album

Research and Optimization of Distributed Clustering Algorithms for High-Dimensional Data

Volume 6, Issue 3

Views: 0 Downloads: 0

Research and Optimization of Distributed Clustering Algorithms for High-Dimensional Data
Research and Optimization of Distributed Clustering Algorithms for High-Dimensional Data
Queen's University of Kingston, Canada
Vol. 6, Issue 3, Pages: 188-195(2025)
Published：25 March 2025
DOI：10.47297/taposatWSP2633-456935.20250603
Accepted：

Scan QR Code

Zou Runfeng.Research and Optimization of Distributed Clustering Algorithms for High-Dimensional Data[J].Theory and Practice of Science and Technology,2025,06(03):188-195.
DOI：

Zou Runfeng.Research and Optimization of Distributed Clustering Algorithms for High-Dimensional Data[J].Theory and Practice of Science and Technology,2025,06(03):188-195. DOI： 10.47297/taposatWSP2633-456935.20250603.

摘要

Abstract

High-dimensional data analysis is a crucial task in various domains

including bioinformatics

image recognition

and text mining. However

traditional clustering algorithms

such as K-means and DBSCAN

struggle with the "curse of dimensionality

" leading to inefficiencies in both accuracy and computational performance. To address these challenges

this paper proposes an optimized approach that integrates dimensionality reduction techniques with clustering algorithms in a distributed computing environment. Specifically

we evaluate and compare the effectiveness of principal component analysis (PCA)

t-distributed stochastic neighbor embedding (t-SNE)

and autoencoders for dimensionality reduction

followed by K-means and DBSCAN clustering. Experiments are conducted on high-dimensional datasets

including text

image

and genomic data

to assess clustering quality

computational efficiency

and scalability. The results demonstrate that combining appropriate dimensionality reduction methods with clustering significantly improves performance while mitigating the impact of high dimensionality. Furthermore

our distributed computing framework enhances scalability

making it feasible for large-scale data processing. This study provides a comprehensive evaluation of different combinations of dimensionality reduction and clustering methods

offering practical insights into optimizing high-dimensional data clustering for real-world applications.

关键词

Keywords

references

Bellman, R. (1961). Adaptive Control Processes: A Guided Tour. Princeton University Press.

Aggarwal, C. C., Hinneburg, A., & Keim, D. A. (2001). On the surprising behavior of distance metrics in high dimensional space. International Conference on Database Theory, 420-434.

Jolliffe, I. T. (2002). Principal Component Analysis. Springer.

van der Maaten, L., & Hinton, G. (2008). Visualizing data using t-SNE. Journal of Machine Learning Research, 9(Nov), 2579-2605.

Ester, M., Kriegel, H. P., Sander, J., & Xu, X. (1996). A density-based algorithm for discovering clusters in large spatial databases with noise. Proceedings of KDD, 226-231.

Zhang, X., & Hong, M. (2023). Fed-SC: One-Shot Federated Subspace Clustering over High-Dimensional Data. IEEE Transactions on Neural Networks and Learning Systems, 34(7), 3456-3467.

Xiao, Y., Xue, J., & Meng, D. (2023). Hashing-Based Distributed Clustering for Massive High-Dimensional Data. arXiv preprint arXiv: 2306.17417 .

Wang, Y., & Chen, H. (2020). Enhanced synchronization-inspired clustering for high-dimensional data. Complex & Intelligent Systems, 6(2), 345-356.

Li, J., & Zhang, S. (2024). A Novel Hierarchical High-Dimensional Unsupervised Active Learning Clustering Algorithm. International Journal of Fuzzy Systems, 26(1), 123-135.

20 Newsgroups Dataset. Scikit-learn. Available at: https://scikit-learn.org/0.19/datasets/twenty_newsgroups.html.

MNIST Dataset. PyTorch Vision. Available at: https://pytorch.org/vision/main/generated/torchvision.datasets.MNIST.html.

Gene Expression Dataset. Kaggle. Available at: https://www.kaggle.com/datasets/crawford/gene-expression.

Alert me when the article has been cited

Submit

Privacy Protection of Clustering High-Dimensional Data Based on African Vultures

Research and Implementation of Intelligent Data Analysis System for Online Education

Source Analysis of Major Pollutants in Fine Particulate Matters in Shenyang City

A CNN-based Physical Layer Design for Communications System

Related Author

Chenglin Zhang School of Information & Electronics Technology, Jiamusi University

Lili He School of Information & Electronics Technology, Jiamusi University

Mingwan LUO Yangjiang Vocational and Technical College;.Paul University Philippines, Tuguegarao, 3500, Cagayan

Marifel Capili Kummer .Paul University Philippines, Tuguegarao, 3500, Cagayan

Junzhao MA

Xun YAO

Xueyu BAI

Haibo LI

Related Institution

School of Information & Electronics Technology, Jiamusi University

Yangjiang Vocational and Technical College

.Paul University Philippines, Tuguegarao, 3500, Cagayan

School of Resources and Civil Engineering, Northeastern University

University of Electronic Science and Technology of China (UESTC)University of Glasgow

⁰