| Mark Pasquier Computer Science 2009 |
REU Project: TeamTrak
The TeamTrak program works to provide a platform for testing software on wireless mobile networks. This platform consists of multiple sensor helmets equipped with various sensors to track the location and heading of the user. At a user specified interval, a photo is taken and this sensor data is stored in a related text file. This data can later be used to analyze the activity of the person wearing that sensor helmet, and the photos associated with it can be used for various image analyses processes.
My contribution to the project is the chirp lifesaver system which manages these photos. This management includes organizing, storing, and checking the health of the system in order to ensure a useable and robust image storage system. This system allows the TeamTrak image data to both be easily accessed for analysis and ensure that it will be retained in the case of disk failure. While this process was designed for the current system, the same methodology could be applied to similar situations of managing files that are stored across distributed systems.
The main organization of the system consists of two tables, a Photos table and a Files table. The Photos table stores all of the metadata corresponding to a unique image. Each one of these photos, and the corresponding metadata, are associated with each copy of the actual image file. The location of each one of these files on the chirp server is then stored in the Files table, and connected back to its metadata in the Photos table.
The process of implementing this consists of 5 major steps:
- The user generated photos and corresponding metadata are uploaded to a dropbox on the CCL server by an individual client.
- The Chirp Lifesaver program takes in the photos and corresponding metadata
- This metadata is then put into an entry in the MySQL database. This process generates a unique Photonumber which corresponds to each unique image in the database.
- Multiple file entries are then put into the Files table. Each one of these file entries generates a unique Filenumber which is then used to distinguish between each copy of a photo. When these file entries are created, they are linked to their metadata in the Photos table through the Photonumber generated in step 3.
- The unique Filenumber generated in step 4 is then used to ensure a unique filename for the photo as it is put onto a host in the CCL database. The host is chosen so that each new copy is put onto a different host, ensuring that if one fails, a copy will still exist.
Throughout this process, the data being moved on each step is checked for accuracy. In the initial loading stage, chirp lifesaver checks to make sure all of the necessary data is there before it is added to the Photos table. After each file is placed on the CCL server, the checksum of that file is compared to the checksum of the original photo, stored with the metadata in the Photos table, before that file is marked as healthy within the Files table.
While this scheme allows for the initial uploading and copying of Photos onto the server and into the database, it does not yet actively scan the system to ensure that the files that were put there are still there. The programs used for each step in the process, while capable of handling the current system affectively, are too rough to be easily modified to accommodate different systems. They also rely on a few nuances of the system, such as the format of the metadata text file, that make it less robust than would be ideal. The program works affectively for the current situation, extending it, and making it more flexible would make it a more complete and effective system.