All-Pairs Abstraction

All-Pairs( array A[i], array B[j], function F(x,y) ) returns matrix M where M[i,j] = F( A[i], B[j] )

The All-Pairs abstraction computes the Cartesian product of two sets, generating a matrix where each cell M[i,j] contains the output of the function F on objects A[i] and B[j]. You provide two sets of data files and a function F that computes on them. You may optionally provide additional parameters to control the actual computation(e.g. compute only part of the matrix). The abstraction then executes the computation in parallel, automatically handling load balancing, data movements, fault tolerance, and so on.

For More Information

Publications

(Showing papers with tag allpairs. See all papers instead.)

Li Yu, Christopher Moretti, Andrew Thrasher, Scott Emrich, Kenneth Judd, and Douglas Thain, Harnessing Parallelism in Multicore Clusters with the All-Pairs, Wavefront, and Makeflow Abstractions, Journal of Cluster Computing, 13(3), pages 243-256, September, 2010. DOI: 10.1007/s10586-010-0134-7

Christopher Moretti, Hoang Bui, Karen Hollingsworth, Brandon Rich, Patrick Flynn, and Douglas Thain, All-Pairs: An Abstraction for Data Intensive Computing on Campus Grids, IEEE Transactions on Parallel and Distributed Systems, 21(1), pages 33-46, January, 2010. DOI: 10.1109/TPDS.2009.49

Kyle Wheeler, Exploiting Locality with QThreads for Portable Parallel Performance, Ph.D. Thesis, University of Notre Dame, November, 2009.

Li Yu, Christopher Moretti, Scott Emrich, Kenneth Judd, and Douglas Thain, Harnessing Parallelism in Multicore Clusters with the All-Pairs and Wavefront Abstractions, IEEE High Performance Distributed Computing, pages 1-10, June, 2009. DOI: 10.1145/1551609.1551613

Christopher Moretti, Jared Bulosan, Douglas Thain, and Patrick Flynn, All-Pairs: An Abstraction for Data Intensive Cloud Computing, IEEE International Parallel and Distributed Processing Symposium (IPDPS), pages 1-11, April, 2008. DOI: 10.1109/IPDPS.2008.4536311

Christopher Moretti, Jared Bulosan, Douglas Thain, and Patrick J. Flynn, Poster: All-Pairs: An Abstraction for Data Intensive Computing, IEEE/ACM Grid Computing, September, 2007.