The All-Pairs abstraction computes the Cartesian product of two sets, generating a matrix where each cell M[i,j] contains the output of the function F on objects A[i] and B[j]. You provide two sets of data files and a function F that computes on them. You may optionally provide additional parameters to control the actual computation(e.g. compute only part of the matrix). The abstraction then executes the computation in parallel, automatically handling load balancing, data movements, fault tolerance, and so on.
Related Publications
Harnessing Parallelism in Multicore Clusters with the All-Pairs, Wavefront, and Makeflow Abstractions
Li Yu, Christopher Moretti, Andrew Thrasher, Scott Emrich, Kenneth Judd, and Douglas Thain
@article{abstr-jcc,author={Yu, Li and Moretti, Christopher and Thrasher, Andrew and Emrich, Scott and Judd, Kenneth and Thain, Douglas},title={{Harnessing Parallelism in Multicore Clusters with the All-Pairs, Wavefront, and Makeflow Abstractions}},journal={{Journal of Cluster Computing}},volume={13},number={3},pages={243-256},year={2010},note={{doi: 10.1007/s10586-010-0134-7}},cclpaperid={83},keywords={makeflow, workqueue, allpairs, wavefront, hecura},}
All-Pairs: An Abstraction for Data Intensive Computing on Campus Grids
Christopher Moretti, Hoang Bui, Karen Hollingsworth, Brandon Rich, Patrick Flynn, and Douglas Thain
IEEE Transactions on Parallel and Distributed Systems, 2010
@article{allpairs-tpds,author={Moretti, Christopher and Bui, Hoang and Hollingsworth, Karen and Rich, Brandon and Flynn, Patrick and Thain, Douglas},title={{All-Pairs: An Abstraction for Data Intensive Computing on Campus Grids}},journal={{IEEE Transactions on Parallel and Distributed Systems}},volume={21},number={1},pages={33-46},year={2010},note={{doi: 10.1109/TPDS.2009.49}},cclpaperid={12},keywords={allpairs, hecura},}
Exploiting Locality with QThreads for Portable Parallel Performance
@thesis{wheeler-thesis,author={Wheeler, Kyle},title={{Exploiting Locality with QThreads for Portable Parallel Performance}},editor={Thesis, Ph.D.},booktitle={{University of Notre Dame}},year={2009},cclpaperid={81},keywords={allpairs, wavefront, hecura},}
Harnessing Parallelism in Multicore Clusters with the All-Pairs and Wavefront Abstractions
Li Yu, Christopher Moretti, Scott Emrich, Kenneth Judd, and Douglas Thain
In IEEE High Performance Distributed Computing, 2009
@inproceedings{abstr-hpdc09,author={Yu, Li and Moretti, Christopher and Emrich, Scott and Judd, Kenneth and Thain, Douglas},title={{Harnessing Parallelism in Multicore Clusters with the All-Pairs and Wavefront Abstractions}},booktitle={{IEEE High Performance Distributed Computing}},pages={1-10},year={2009},note={{doi: 10.1145/1551609.1551613}},cclpaperid={5},keywords={workqueue, allpairs, wavefront, hecura},}
All-Pairs: An Abstraction for Data Intensive Cloud Computing
Christopher Moretti, Jared Bulosan, Douglas Thain, and Patrick Flynn
In IEEE International Parallel and Distributed Processing Symposium (IPDPS), 2008
@inproceedings{allpairs-ipdps08,author={Moretti, Christopher and Bulosan, Jared and Thain, Douglas and Flynn, Patrick},title={{All-Pairs: An Abstraction for Data Intensive Cloud Computing}},booktitle={{IEEE International Parallel and Distributed Processing Symposium (IPDPS)}},pages={1-11},year={2008},note={{doi: 10.1109/IPDPS.2008.4536311 }},cclpaperid={28},keywords={allpairs, hecura},}
Poster: All-Pairs: An Abstraction for Data Intensive Computing
Christopher Moretti, Jared Bulosan, Douglas Thain, and Patrick J. Flynn
@inproceedings{allpairs-grid07,author={Moretti, Christopher and Bulosan, Jared and Thain, Douglas and Flynn, Patrick J.},title={{Poster: All-Pairs: An Abstraction for Data Intensive Computing}},booktitle={{IEEE/ACM Grid Computing}},year={2007},cclpaperid={63},keywords={allpairs, hecura},}
