RWSL Assignment Part two of the Continuous Assessment -Summarizing

Original text summarized is in the post titled "original summary text"

This is a summary of the Introduction section from the conference article “Don't match twice: redundancy-free similarity computation with MapReduce” by Lars Kolb, Andreas Thor, and Erhard Rahm (2013). 

 

They stated that redundancy is an outstanding issue associated with comparing object pairs and finding similarities. Applications that use large amounts of data rely heavily on the Pair-wise-Similarity-Computation (PSC) operation which is a resource intensive operation and PSC uses the MapReduce (MR) model to find similarities between objects that can be strings, documents or data entities like in a database. They noted that using PSC without in-depth understanding of it results in a multiple set of pairs of objects which is very costly and ineffective. To avoid this, they noted, is possible by grouping the objects into clusters and then performing the similarity search operation on the clusters following the MR model. The MR model generates signatures like keys or tokens for each object, before grouping the objects into clusters of small sizes each. The signatures are then used to identify the clusters during the computation and the objects in the same cluster are compared to each other. They argued that although this can reduce redundancy it has disadvantages because, first, creating the signatures can be very challenging and secondly the expected high standard result can be compromised by the type of data input. Moreover although creating small clusters is good, it might not include other objects that are of the same category. A cluster with location or zip code as the signature, they remarked, might not group similar objects into the same cluster because of their varied location or zip code. Most implementations use a combination of signatures to represent objects in a cluster, undermining the fact that there are still redundancy issues with the quality of the data. Such computations reduces the effectiveness of these applications which is why they propose an "optimization approach" to eradicate redundancy 

 

Reference:

Lars Kolb, Andreas Thor, and Erhard Rahm, (2013) ‘Don’t match twice: redundancy-free similarity computation with MapReduce’ In Proceedings of the Second Workshop on Data Analytics in the Cloud (DanaC '13), ACM, New York, NY, USA, 1-5. Available: http://doi.acm.org/10.1145/2486767.2486768

[Accessed 25 November 2013]