The Optimization of the Big Data Cleaning Based on Task Merging

Data quality issues will result in lethal effects of big data applications,so it is needed to clean the big data with the problem of data quality.MapReduce programming framework can take advantage of parallel technology to achieve high scalability for large data cleaning.However,due to the lack of effective design,redundant computation exists in the cleaning process based on MapReduce,resulting in decreased performance.Therefore,the purpose of...