|CCL HomeSoftware Community Operations||
Using the CCL Storage PoolTo begin, add the the cctools to your path:
% setenv PATH /afs/crc.nd.edu/group/ccl/software/cctools/bin:$PATH
Or, download a recent version of the software.
Next, view the currently available storage devices by using the storage catalog.
The simplest way to access a Chirp server is via the Chirp command line tool. This allows you to connect to a server, copy files, and manage directories, much like an FTP client:
In scripts, you may find it easier to use the standalone commands chirp_get and chirp_put, which move single files to and from a Chirp server. These commands also allow for streaming data, which can be helpful in a shell pipeline. Also, the -f option to both commands allows you to follow a file, much like the Unix tail command:
An easier way to access Chirp servers is by using a tool called Parrot. Parrot is a personal virtual filesystem: it "speaks" remote I/O operations on behalf of ordinary programs. For example, you can use Parrot with your regular shell to access Chirp servers like so:
Parrot is certainly the most convenient way to access storage, but it has some limitations: it only works on Linux, and imposes a performance penalty. (This is because Parrot makes an extra data copy in the process of handling a program's system calls.)
The fourth way to access the storage pool is write your own programs that access the Chirp C interface. You must compile and link against the following file in the ordinary way:
..../ccl/software/devel/include/chirp_client.h ..../ccl/software/devel/include/chirp_reli.h ..../ccl/software/devel/lib/libchirp.aThe chirp_client.h interface allows you to explicitly connect to a server and open, close, read, and write files, much as in a traditional Unix interface. This interface is unreliable in the sense that a broken connection will cause all further operations to fail. To recover, you must explicitly re-connect to the server.
The chirp_reli.h interface is a reliable version of the chirp_client interface. The programmer need not explicitly connect or disconnect to servers, but simply names the host and file to access. The library transparently handles connection as well as recovery from temporary failures.