A partition based method for finding highly correlated pairs (Q615683)

Summary: The problem of finding highly correlated pairs is to output all item pairs whose (Pearson) correlation coefficients are greater than a user-specified correlation threshold. Effective discovery of such item pairs is of primary importance in many real data mining applications. Algorithm and Taper algorithm are special cases of our new algorithm with respect to the number of segments. Experimental results on real datasets demonstrate the feasibility and superiority of our algorithm. Recently, the Taper algorithm is developed to discover the set of highly correlated item pairs. In this paper, we present a generalised Taper algorithm to find strongly correlated pairs between items by partitioning the collection of transactions into different segments, so as to achieve better pruning effect and less running time. Consequently, it can be proved that both are naive.