Hadoop Map Reduce Performance Evaluation and Improvement Using Compression Algorithms on Single Cluster

Abstract

In todays scenario a word Big Data used by researchers is associated with large amount of data which requires more resources likes processors, memories and storage capacity. Data can be structured and non-structured like text, images, and audio, video, social media data. Data generated by various sensor devices, mobile devices, social media. Data is stored into repository on the basis of their attributes like size, colours name. Data requires more storage space. In this paper we have evaluated performance of Hadoop MapReduce examples like TeraGen, TeraSor, TeraValidate. We have evaluated Hadoop Map Reduce performance by configuring compression related parameter and different compression algorithm like DEFLATE, Bzip2, Gzip , LZ4 on single Cluster through Word Count example. After evaluating compression algorithm through Word Count Example we found job elapsed time, I/O time and storage space requirement is reduced marginally along with increase in the CPU computation time.