HOWTO: Write Code that Uses Chirp

For most purposes, the easiest way to access Chirp servers is by way of several existing tools. You can see a chart of available tools to decide which is best to use. However, if you need to write (or modify) a program to access Chirp filesystems directly, then follow these instructions. This page does not describe every single Chirp call, but it will get you started with enough of a program to explore further. For the full details, please see the Chirp API Documentation.

Getting Started

Obtain the source code from the download page. Build and install the code and libraries in your home directory as follows:

   gunzip cctools-x_y_z-src.tar.gz
   tar xvf cctools-x_y_z-src.tar
   cd cctools-x_y_z
   ./configure --prefix ~/cctools
   make all install

Now, make a clean directory for you to work in:

   mkdir ~/mychirp
   cd ~/mychirp

And download the following samples to get started:

example.c

Makefile

Type make to build the example, and then ./example to run it. It should build against your installed version of the Chirp libraries and produce and executable named example. Run ./example and you should see something like this:

reading directory of host ccl00.cse.nd.edu:
    4096 .
    4096 ..
    ...
couldn't open /datafile for writing: Permission denied

There, you have successfully connected to a Chirp server, obtained a directory listing, and obtained a (correct) error message from the server. Let's go back and look at the program in more detail.

Looking at the Code

First, to use the Chirp API, you must include several files that access Chirp proper, as well as several supporting libraries for debugging, authentication, and the like.

#include "chirp_reli.h"
#include "debug.h"
#include "auth_all.h"
#include "md5.h"

This program is hard coded to access to following host and filename:

        const char *hostname = "ccl00.cse.nd.edu";
        const char *filename = "/datafile";

If you have access to another server, go ahead and change those lines appropriately.

This is followed by some code that displays a help message, and processes some common command line arguments. The most important argument to know is -d, which turns on the debugging subsystem. Try running ./example -d all, which shows nearly everything that happens over the network. Operations that produce slightly less output are -d chirp, which only shows remote Chirp operations, and -d tcp, which shows when TCP connections are made and broken.

In order to talk to a Chirp server, you must first authenticate yourself. The single call to auth_register_all configures the authentication system to an automatic mode so that you don't have to worry about it. Each time you connect to a Chirp server, your identity will be determined automatically. (For more details about authentication, read the manual or consult this HOWTO guide.)

The example makes use of the chirp_reli interface, which is fully described in the Chirp API Documentation Here, reli indicates a "reliable" interface: you don't have to worry about connecting to servers or retrying network errors, because internally, the library will retry errors for you, up to a specific timeout. In this example, we will choose sixty seconds from the current time:

        stoptime = time(0) + 60;

Now we are ready to use Chirp. We start by obtaining a directory listing of a server:

	dir = chirp_reli_opendir(hostname,"/",stoptime);

Of course, in a distributed system, many different things can go wrong, so we must always be careful to check for errors. Chirp functions that return an integer use negative values to indicate failure. Functions that return pointers use null pointers to indicate failure. In either case, the variable errno is set to the reason for the error. So, immediately after calling chirp_reli_opendir, we check for the error as follows:

        if(!dir) {
                fprintf(stderr,"couldn't read directory: %s\n",strerror(errno));
                return 1;
        }

But, if it succeeds, then we call chirp_reli_readdir in a loop to list and print all of the entries:

	while(1) {
		d = chirp_reli_readdir(dir);
		if(!d) break;
		printf("%8lld %s",d.info.cst_size,d->name);
	}

	chirp_reli_closedir(dir);

The program goes on to write data to a file. First, we open the file for writing: (If you are getting an error here, you need to change your pathname to point to a directory and file where you have write permissions.)

        file = chirp_reli_open(hostname,filename,O_WRONLY|O_TRUNC|O_CREAT,0700,stoptime);
        if(!file) {
                fprintf(stderr,"couldn't open %s for writing: %s\n",filename,strerror(errno));
                return 1;
        }

And then write a few bytes:

        result = chirp_reli_pwrite(file,line,strlen(line),0,stoptime);
        if(result<0) {
                fprintf(stderr,"couldn't write to %s: %s\n",filename,strerror(errno));
                return 1;
        }

And then close the file:

        chirp_reli_close(file,stoptime);

For fun, we also compute a checksum of the recently written file:

        result = chirp_reli_md5(hostname,filename,digest,stoptime);
        if(result<0) {
                fprintf(stderr,"couldn't checksum %s: %s\n",filename,strerror(errno));
                return 1;
        }

How to Learn More

The chirp_reli interface contains nearly every I/O operation found in the Unix interface. Each has a similar interface to that described above. You can read all about them in the Chirp API Documentation. If you have any difficulty, please see these instructions for getting help. [an error occurred while processing this directive]