Energy Efficient Scheduling of MapReduce over Big Data

Abstract

The majority of large-scale data intensive applications carried out by information centers are based on MapReduce or its open-source implementation, Hadoop. Such applications are carried out on rich clusters requiring ample amounts of energy, helping the energy costs an appreciable fraction of the data centers overall costs. Therefore, reducing the energy consumption when carrying out each MapReduce task is a critical worry for data centers. In this paper, we advise a framework for mending the energy ef?ciency of MapReduce applications, while satisfying the (SLA) Service Level Agreement. We ?rst prototype the problem of energy-aware scheduling of a single MapReduce task as an Integer Program. After that we court two algorithms, known as MapReduce scheduling algorithms and load scheduling algorithm, that ?nd the assignments of map and reduce tasks to the machines plenty in order to reduce the energy consumed when carrying out the application. The energy aware con?guration and scheduling will improve the energy e?ciency of MapReduce clusters thus help in reduction of the service costs of the data-centers.