Systematic review of clustering high-Dimensional and large datasets

Abstract

Technological advancement has enabled us to store and process huge amount of data in relatively short spans of time. The nature of data is rapidly increasing its dimensionality to become multi and high-dimensional. There is an immediate need to expand our focus to include analysis of high-dimensional and large datasets. Data analysis is becoming a mammoth task, due to in cemental increase in data volume and complexity in terms of heterogony of data. It is due to this dynamic computing environment that the existing techniques either need to be modified or discarded to handle new data in multiple high dimensions. Data clustering is a tool that is used in many disciplines, including data mining, so that meaningful knowledge can be extracted from seemingly unstructured data. The aim of this paper is to understand the problem of clustering and various approaches addressing this problem. This paper discusses the process of clustering from both micro (data treating) and macro (overall clustering process) views. Different distance and similarity measures, which form the cornerstone of effective data clustering are also identified. Further, an in-depth analysis of different clustering approaches focused on data mining, dealing with large-scale data sets is given. These approaches are so comprehensively compared to bring out a clear differentiation among them. This paper also surveys the problem of high-dimensional data and the existing approaches that helps to make it more relevant. It also explores the latest trends in cluster analysis, and there al life applications of this concept. This survey is exhaustive as it tries to cover all the aspects of clustering in the field of data mining.