I2mapreduce: Fine-Grain Incremental Processing In Big Data Mining

Abstract

I²MAPREDUCE: Fine-Grain Incremental Processing in big data mining a novel incremental processing extension to Map Reduce, the most widely used framework for mining big data. As compare with the high-tech work on In coop, I²MapReduce has its own advantages (i) It prefers key-value pair level incremental processing to perform instead of task level re-computation, (ii) It supports one-step computation along with more sophisticated iterative computation, which is extensively used in data mining applications, and (iii) It reduce I/O overhead for accessing preserved fine-grain computation states by incorporating the set of novel techniques. I evaluate I²MAPREDUCE: Fine-Grain Incremental processing in big data mining using a one-step algorithm and four iterative algorithms with assorted computation characteristics. Experimental results on Amazon EC2 show significant performance improvements of I²MapReduce compared to both plain and iterative MapReduce performing re-computation.