Comprehensive Comparison of Various Approaches for Implementation of Expert IR System through Pre-processing and Clustering

Abstract

We live in a digital world wherein tremendous electronic data evolves out every day, generating from huge amount of sources present. This data is tedious and nearly impossible to manage as it being literally large. Data storage and retrieval becomes a truly difficult task. Thus through data mining approach this type of data can be treated with various efficient techniques for cleaning, compression and sorting of data. Preprocessing can be used to remove basic English stop-words from data making it compact and easy for further processing; later dimensionality reduction techniques make data more efficient and specific. This data later can be clustered for better information retrieval. This paper elaborates the various dimensionality reduction and clustering techniques applied on sample dataset C50test of 2500 documents giving promising results, their comparison and better approach for relevant information retrieval, using tool based as well as programming based approach for providing comprehensive choices among the users.