About the CCL
We design software
that enables our collaborators
to easily harness large scale distributed systems
such as clusters, clouds, and grids. We perform fundamental computer science research
that enables new discoveries through computing in fields such as physics, chemistry, bioinformatics, biometrics, and data mining.
Makeflow is used to manage the data processing workflow of the Airborne Lidar Processing System (ALPS) for the Experimental Advanced Airborne Research Lidar (EAARL) at the USGS. Over the course of several hours of flight time, about a gigabyte of raw LIDAR waveform data is collected. This data must be geo-referenced in order to convert it into point clouds broken into 2km tiles suitable for GIS use. When we collect this data in the field, it is critical that the field crew can process the data rapidly and take a look at it for obvious problems so they can be corrected before the next flight.
Using Makeflow, the data can be processed quickly on a portable 32-core cluster in the field in about 20 minutes. The data can be processed fast enough to do some cursory analysis and also re-process it a few times if needed to troubleshoot issues. Using Makeflow, it is easy to run the exactly same workflow in the field on the portable cluster or back in the office on a multi-core system.
- David Nagle, US Geological Survey