Improving Energy Efficiency of MapReduce Framework using Dynamic Scheduling of Work

Abstract

Most common huge volume data processing programs do counting, sorting, merging etc. Such programs require to perform first a computation on each record that is it requires to map an operation to each record. Then combine the output of these operations in appropriate way to get the answer that is apply a reduce operation to groups of records. MapReduce runtime environment takes care of parallelizing their execution and coordinating their inputs/outputs. Here we are concern about energy efficiency in MapReduce framework so we are proposing dynamic scheduling of workload which offers dynamic load balancing method. Load balancing is the methodology of distributing the load among different node of a distributed framework to enhance both resource usage and reaction time while likewise keeping away from a circumstance where a percentage of the node are intensely stacked while different node are sit out of gear or doing next to no work. An answer for unbalance circumstance is to utilize parallelization approaches yet at the same time node will stay overwhelming. In this paper, we propose an integrated. We are proposing a methodology where the MapReduce concept introduced into the MongoDB with NoSQL as a back end to implement the MapReduce.