Umbrella is a tool for specifying and materializing comprehensive execution environments, from the hardware all the way up to software and data. A user simply invokes Umbrella with the desired task, and Umbrella parses the specification, determines the minimum mechanism necessary to run the task, downloads missing dependencies, and executes the application through the available minimal mechanism, which may be direct execution, a system container (Parrot, Docker, chroot), a local virtual machine (i.e., VMware), or submission to a cloud environment (i.e., Amazon EC2) or grid environment (i.e., HTCondor).
An Umbrella specification includes six sections: hardware , kernel , os , software , data , and environ. By specifying the dependencies of an application clearly and materializing the execution environment during runtime automatically, the application becomes portable and reproducible.
Umbrella involves multiple sandboxing and virtualization techniques, however, the key idea of Umbrella is to construct a sandbox for an application during runtime by mounting all the os, software, and data dependencies into a virtual root filesystem without copying them. The usage of mounting mechanism allows multiple sandboxes share the same dependencies concurrently.
Related Publications
Facilitating the Reproducibility of Scientific Workflows with Execution Environment Specifications
Haiyan Meng and Douglas Thain
In The 17th International Conference on Computational Science (ICCS), 2017
@inproceedings{PAPER937,author={Meng, Haiyan and Thain, Douglas},title={{Facilitating the Reproducibility of Scientific Workflows with Execution Environment Specifications}},booktitle={{The 17th International Conference on Computational Science (ICCS)}},year={2017},note={{doi: 10.1016/j.procs.2017.05.116}},cclpaperid={937},keywords={makeflow, umbrella},}
Conducting Reproducible Research with Umbrella: Tracking, Creating, and Preserving Execution Environments
Haiyan Meng, Douglas Thain, Alexander Vyushkov, Matthias Wolf, and Anna Woodard
@inproceedings{umbrella-escience-2016,author={Meng, Haiyan and Thain, Douglas and Vyushkov, Alexander and Wolf, Matthias and Woodard, Anna},title={{Conducting Reproducible Research with Umbrella: Tracking, Creating, and Preserving Execution Environments}},booktitle={{IEEE Conference on e-Science}},year={2016},note={{doi: 10.1109/eScience.2016.7870889}},cclpaperid={931},keywords={parrot, umbrella, daspos},}
Techniques for Preserving Scientific Software Executions: Preserve the Mess or Encourage Cleanliness?
Douglas Thain, Peter Ivie, and Haiyan Meng
In 12th International Conference on Digital Preservation (iPres), 2015
@inproceedings{techniques-ipres-2015,author={Thain, Douglas and Ivie, Peter and Meng, Haiyan},title={{Techniques for Preserving Scientific Software Executions: Preserve the Mess or Encourage Cleanliness?}},booktitle={{12th International Conference on Digital Preservation (iPres)}},year={2015},note={{doi: 10.7274/R0CZ353M}},cclpaperid={921},keywords={parrot, prune, umbrella, daspos},}
Umbrella: A Portable Environment Creator for Reproducible Computing on Clusters, Clouds, and Grids
Haiyan Meng and Douglas Thain
In Workshop on Virtualization Technologies in Distributed Computing (VTDC) at HPDC, 2015
@inproceedings{umbrella-vtdc15,author={Meng, Haiyan and Thain, Douglas},title={{Umbrella: A Portable Environment Creator for Reproducible Computing on Clusters, Clouds, and Grids}},booktitle={{Workshop on Virtualization Technologies in Distributed Computing (VTDC) at HPDC}},year={2015},note={{doi: 10.1145/2755979.2755982}},cclpaperid={912},keywords={parrot, umbrella},}
