Cooperative Computing Lab
CCL | Software | Install | Manuals | Forum | Papers
CCL Home

Research

Software Community Operations

Practice Problems for Makeflow

  1. Write a Makeflow to check if "Shakespeare speak" still exists
    1. Objectives
    2. Problem Statement
  2. Problem 2: Run your Makeflow on multiple platforms
    1. Objectives
    2. Problem Statement
  3. Problem 3: Apply Makeflow to your research
    1. Problem Statement

Use the information in the Makeflow page to learn more about Makeflow, if necessary.

Problem 1

Write a Makeflow to check if "Shakespeare speak" still exists

Objectives

You will be able to:

  • Decompose and execute workflows as smaller concurrent pieces.
  • Write Makeflow scripts for the execution of your workflows.

Problem Statement

You will compare the words seen in the works of Shakespeare with the words currently in use in the English language and identify those words that are still in use.

You will compare the words of Shakespeare listed in each of these text files:

Each file contains all the words found in a work of Shakespeare identified in the filename. For your convenience, each word is listed in a separate line to make it easier to scan and read them.

You will compare the words in each of these files with the dictionary provided in Unix systems (/usr/share/dict/words or /usr/dict/words). If a word is found to exist in the dictionary, you should print the word followed by the string "Art spoken ever and anon!". If not, you should print the word followed by the string "Ne'er spoken ever!".

To complete this problem, you will create a Makeflow script that decomposes the problem into concurrent tasks. Your script should contain 5 rules, each describing a search of the dictionary for the words in a file, or in other words, one rule per file. You will then run this Makeflow using the Torque batch submission system as described here.

Hint:

To read each file and compare the words listed in it against the dictionary, you can use this simple python program: shakespeare-compare.py. You will then have this program executed remotely by specifying the filename containing the words to check.

For example, to check if the words in hamlet.txt exist in the dictionary, run:

python shakespeare-compare.py hamlet.txt

Create Makeflow rules that check each of the files by running this program.

Problem 2

Configure Makeflow to run on multiple platforms

Objectives

This problem will familiarize participants with the following:

  • Running Makeflows on multiple platforms

Problem Statement

For this problem, you will run the Makeflow application you developed for Problem 1 on multiple platforms. Specifically, you will run your Makeflow on both the ND CRC and on your personal desktop or laptop.

The -T option to Makeflow specifies which system to run the makeflow on. To complete this assignment run the Makeflow you created in Part 1 on your local workstation using the -T local argument to Makeflow. Following that, clean up your ouput and rerun the workflow using the CRC's SGE cluster and -T sge.

Problem 3

Apply Makeflow to your research.

Problem Statement

In completing Problem 1 and Problem 2, you have mastered the fundamentals of expressing and running large worfklows using Makeflow!

Now, can you think of one or two instances in your research where Makeflow might be useful?