|CCL HomeSoftware Community Operations||
Makeflow is a workflow system for parallel and distributed computing that uses a language very similar to Make. Using Makeflow, you can write simple scripts that easily execute on hundreds or thousands of machines.
Work Queue is a system and library for creating and managing scalable master-worker style programs that scale up to thousands machines on clusters, clouds, and grids. Work Queue programs are easy to write in C, Python or Perl.
Parrot is a transparent user-level virtual filesystem that allows any ordinary program to be attached to many different remote storage systems, including HDFS, iRODS, Chirp, and FTP.
Chirp is a personal user-level distributed filesystem that allows unprivileged users to share space securely, efficiently, and conveniently. When combined with Parrot, Chirp allows users to create custom wide-area distributed filesystems.
Monitors the cpu, memory, io, and disk usage of applications running in distributed systems, and can optionally enforce limits on each resource. A visualizer allows you to explore the consumption of workloads consisting of thousands of tasks.
Confuga is an active storage cluster file system designed for executing DAG-structured scientific workflows. It is used as a collaborative distributed file system and as a platform for execution of scientific workflows with full data locality for all job dependencies.
Accelerated Weighted Ensemble
Accelerated Weighted Ensemble (AWE) is a system for large scale adaptive sampling molecular dynamics. The framework decomposes the resampling computations and the molecular dynamics simulations into tasks that are dispatched to Work Queue for distribution and execution across thousands of CPUs and GPUs.
The Scalable Assembler at Notre Dame (SAND) is a set of modules that augment the Celera genome assembler to scale up to thousands of nodes using the Work Queue system.
All-Pairs is a specialized framework for running large number of pairwise comparisons by harnessing hundreds to thousands of machines. All-Pairs problems are common in fields such as data mining, bioinformatics, and biometrics.
Wavefront is a specialized framework for running very large dynamic programming problems by harnessing hundreds to thousands of machines. Wavefront problems are common in game theory, economics, and bioinformatics.