HOWTO: Setup a Chirp Cluster Filesystem

Multiple Chirp servers can be combined together to form a single filesystem with a very large capacity. This is known in various publications as a "unified cluster" or a "DSFS", or a "multi filesystem". This article describes how to set up such a filesystem.

A Chirp cluster filesystem consists of a directory server and one or more data servers. The filesystem will also have a unique "volume name", which can be any name that you choose. Let us suppose that you wish to create a volume titled bigvol that will be stored on the following machines:

  • dir.somewhere.edu - the directory server
  • data01.somewhere.edu - a data server
  • data02.somewhere.edu - a data server
  • data03.somewhere.edu - a data server
  • First make sure that you have installed Parrot and Chirp properly, started a Chirp server on each of the directory and data servers, and are able to connect to each Chirp server remotely. Now, using Parrot, set up the directory server by creating a few files as follows:

        % parrot_run tcsh
        % mkdir /chirp/dir.somewhere.edu/bigvol
        % cd /chirp/dir.somewhere.edu/bigvol
        % mkdir root
        % echo data01.somewhere.edu >> hosts
        % echo data02.somewhere.edu >> hosts
        % echo data03.somewhere.edu >> hosts
        % echo bigvolfiles > key
    

    Now that the directory server is established, you may can access the filesystem through the path /multi/dir.somewhere.edu@bigvol and use it to read and write files, which will be spread across the data servers. For example:

        % parrot_run tcsh
        % cd /multi/dir.somwhere.edu@bigvol/
        % df .
        % cp -r /tmp/mydata .
    
    [an error occurred while processing this directive]