Integrated Monitoring Software with real time process level data analysis using Map Reduce Framework

Abstract

In Complex production environments, multiple servers are located at different locations and multiple processes are running on each server. It is utmost essential to monitor all these servers. Many monitoring software’s provide flexibility to add different form of servers and various global performance counters for real time monitoring of server. Application service managers use global performance counters to monitor server and application health. An email survey was conducted with application service managers of different software and IT organizations. The results of survey has shown that 17% of application service managers make use of real time process level statistics. These tools are developed internally within organization and not integrated with global level monitoring software. It was quite noteworthy that 70% voted for importance of real time process level performance counters integrated with global counters. As monitoring of all processes are not going to be productive, it was proposed and recommended to monitor top three or top five processes specific to performance counter. Application service managers monitor the system with global counters but it is more effective when it has been done in integrated monitoring software with real time process level counters and customized log monitoring across servers. An email survey was conducted with application service managers of different software and IT organizations. The results of survey has shown that 17% of application service managers make use of real time process level statistics. These tools are developed internally within organization and not integrated with global level monitoring software. It was quite noteworthy that 70% voted for importance of real time process level performance counters integrated with global counters. As monitoring of all processes are not going to be productive, it was proposed and recommended to monitor top three or top five processes specific to performance counter. Application service managers monitor the system with global counters but it is more effective when it has been done in integrated monitoring software with real time process level counters and customized log monitoring across servers.