work_queue_json.h File Reference

A manager-worker library. More...

#include "work_queue.h"

Go to the source code of this file.

Functions
struct work_queue *	work_queue_json_create (const char *str)
	Create a new work_queue object. More...
int	work_queue_json_submit (struct work_queue *q, const char *str)
	Submit a task to a queue. More...
char *	work_queue_json_wait (struct work_queue *q, int timeout)
	Wait for a task to complete. More...
char *	work_queue_json_remove (struct work_queue *q, int id)
	Remove a task from the queue. More...
char *	work_queue_json_get_status (struct work_queue *q)
	Get the status for a given work queue. More...

Detailed Description

A manager-worker library.

The work queue provides an implementation of the manager-worker computing model using TCP sockets, Unix applications, and files as intermediate buffers. A manager process uses work_queue_json_create to create a queue, then work_queue_json_submit to submit tasks. Once tasks are running, call work_queue_json_wait to wait for completion.

Function Documentation

struct work_queue* work_queue_json_create

(

const char *

str

)

Create a new work_queue object.

Parameters

str	A json document with properties to configure a new queue. Allowed properties are port, name, and priority.

Returns

A new work queue, or null if it could not be created.

int work_queue_json_submit	(	struct work_queue *	q,
		const char *	str
	)

Submit a task to a queue.

Once a task is submitted to a queue, it is not longer under the user's control and should not be inspected until returned via work_queue_wait. Once returned, it is safe to re-submit the same take object via work_queue_submit.

Parameters

q	A work queue object.
str	A JSON description of a task.

task document: (only "command_line" is required.) { "command_line" : string, "input_files" : array of objects with one object per input file (see file document below), "output_files" : array of objects with one object per output file (see file document below), "environment" : object with environment variables names and values (see environment document below), "tag" : string, # arbitrary string to identify the task by the user. }

file document: { "local_name" : string, # name of the file at the machine running the manager "remote_name" : string, # name of the file local_name is copied to/from the machine running the task. "flags" : { "cache" : boolean, # whether the file should be cached at the worker. Default is false. "watch" : boolean, # For output files only. Whether appends to the file should be sent as they occur. Default is false. } }

environment document: { string : string, # name and value of an environment variable to be set for the task. string : string, ... }

Returns

An integer taskid assigned to the submitted task.

char* work_queue_json_wait	(	struct work_queue *	q,
		int	timeout
	)

Wait for a task to complete.

Parameters

q	A work queue object.
timeout	The number of seconds to wait for a completed task before returning. Use an integer time to set the timeout or the constant WORK_QUEUE_WAITFORTASK to block until a task has completed.

Returns

A JSON description of the completed task or the timeout was reached without a completed task, or there is completed child process (call process_wait to retrieve the status of the completed child process). Return string should be freed using free().

{ "command_line" : string , "tag" : string , "output" : string , "taskid" : integer , "return_status" : integer , "result" : integer }

char* work_queue_json_remove	(	struct work_queue *	q,
		int	id
	)

Remove a task from the queue.

Parameters

q	A work queue object.
id	The id of the task to be removed from the queue.

Returns

A JSON description of the removed task.

char* work_queue_json_get_status

(

struct work_queue *

q

)

Get the status for a given work queue.

Parameters

q	A work queue object.

Returns

A JSON description of the stats of the given work queue object.