cctools
Data Fields
work_queue_task Struct Reference

A task description. More...

#include <work_queue.h>

Data Fields

char * tag
 An optional user-defined logical name for the task. More...
 
char * command_line
 The program(s) to execute, as a shell command line. More...
 
work_queue_schedule_t worker_selection_algorithm
 How to choose worker to run the task. More...
 
char * output
 The standard output of the task. More...
 
struct list * input_files
 The files to transfer to the worker and place in the executing directory. More...
 
struct list * output_files
 The output files (other than the standard output stream) created by the program to be retrieved from the task. More...
 
struct list * env_list
 Environment variables applied to the task. More...
 
int taskid
 A unique task id number. More...
 
int return_status
 The exit code of the command line. More...
 
work_queue_result_t result
 The result of the task (see work_queue_result_t. More...
 
char * host
 The address and port of the host on which it ran. More...
 
char * hostname
 The name of the host on which it ran. More...
 
char * category
 User-provided label for the task. More...
 
category_allocation_t resource_request
 See category_allocation_t. More...
 
double priority
 The priority of this task relative to others in the queue: higher number run earlier. More...
 
int max_retries
 Number of times the task is tried to be executed on some workers until success. More...
 
int try_count
 The number of times the task has been dispatched to a worker. More...
 
int exhausted_attempts
 Number of times the task failed given exhausted resources. More...
 
int fast_abort_count
 Number of times this task has been terminated for running too long. More...
 
timestamp_t time_when_submitted
 The time at which this task was added to the queue. More...
 
timestamp_t time_when_done
 The time at which the task is mark as retrieved, after transfering output files and other final processing. More...
 
int disk_allocation_exhausted
 Non-zero if a task filled its loop device allocation, zero otherwise. More...
 
int64_t min_running_time
 Minimum time (in seconds) the task needs to run. More...
 
timestamp_t time_when_commit_start
 The time when the task starts to be transfered to a worker. More...
 
timestamp_t time_when_commit_end
 The time when the task is completely transfered to a worker. More...
 
timestamp_t time_when_retrieval
 The time when output files start to be transfered back to the manager. More...
 
timestamp_t time_workers_execute_last
 Duration of the last complete execution for this task. More...
 
timestamp_t time_workers_execute_all
 Accumulated time for executing the command on any worker, regardless of whether the task completed (i.e., this includes time running on workers that disconnected). More...
 
timestamp_t time_workers_execute_exhaustion
 Accumulated time spent in attempts that exhausted resources. More...
 
timestamp_t time_workers_execute_failure
 Accumulated time for runs that terminated in worker failure/disconnection. More...
 
int64_t bytes_received
 Number of bytes received since task has last started receiving input data. More...
 
int64_t bytes_sent
 Number of bytes sent since task has last started sending input data. More...
 
int64_t bytes_transferred
 Number of bytes transferred since task has last started transferring input data. More...
 
struct rmsummaryresources_allocated
 Resources allocated to the task its latest attempt. More...
 
struct rmsummaryresources_measured
 When monitoring is enabled, it points to the measured resources used by the task in its latest attempt. More...
 
struct rmsummaryresources_requested
 Number of cores, disk, memory, time, etc. More...
 
char * monitor_output_directory
 Custom output directory for the monitoring output files. More...
 
char * monitor_snapshot_file
 Filename the monitor checks to produce snapshots. More...
 
struct list * features
 User-defined features this task requires. More...
 
timestamp_t time_task_submit
 
timestamp_t time_task_finish
 
timestamp_t time_committed
 
timestamp_t time_send_input_start
 
timestamp_t time_send_input_finish
 
timestamp_t time_receive_result_start
 
timestamp_t time_receive_result_finish
 
timestamp_t time_receive_output_start
 
timestamp_t time_receive_output_finish
 
timestamp_t time_execute_cmd_start
 
timestamp_t time_execute_cmd_finish
 
timestamp_t total_transfer_time
 
timestamp_t cmd_execution_time
 
timestamp_t total_cmd_execution_time
 
timestamp_t total_cmd_exhausted_execute_time
 
timestamp_t total_time_until_worker_failure
 
int64_t total_bytes_received
 
int64_t total_bytes_sent
 
int64_t total_bytes_transferred
 
timestamp_t time_app_delay
 

Detailed Description

A task description.

This structure should only be created with work_queue_task_create and delete with work_queue_task_delete. You may examine (but not modify) this structure once a task has completed.

Field Documentation

char* work_queue_task::tag

An optional user-defined logical name for the task.

char* work_queue_task::command_line

The program(s) to execute, as a shell command line.

work_queue_schedule_t work_queue_task::worker_selection_algorithm

How to choose worker to run the task.

char* work_queue_task::output

The standard output of the task.

struct list* work_queue_task::input_files

The files to transfer to the worker and place in the executing directory.

struct list* work_queue_task::output_files

The output files (other than the standard output stream) created by the program to be retrieved from the task.

struct list* work_queue_task::env_list

Environment variables applied to the task.

int work_queue_task::taskid

A unique task id number.

int work_queue_task::return_status

The exit code of the command line.

work_queue_result_t work_queue_task::result

The result of the task (see work_queue_result_t.

Referenced by work_queue.PythonTask::output().

char* work_queue_task::host

The address and port of the host on which it ran.

char* work_queue_task::hostname

The name of the host on which it ran.

char* work_queue_task::category

User-provided label for the task.

It is expected that all task with the same category will have similar resource usage. See work_queue_task_specify_category. If no explicit category is given, the label "default" is used.

category_allocation_t work_queue_task::resource_request
double work_queue_task::priority

The priority of this task relative to others in the queue: higher number run earlier.

int work_queue_task::max_retries

Number of times the task is tried to be executed on some workers until success.

If less than one, the task is retried indefinitely. See try_count below.

int work_queue_task::try_count

The number of times the task has been dispatched to a worker.

If larger than max_retries, the task failes with WORK_QUEUE_RESULT_MAX_RETRIES.

int work_queue_task::exhausted_attempts

Number of times the task failed given exhausted resources.

int work_queue_task::fast_abort_count

Number of times this task has been terminated for running too long.

timestamp_t work_queue_task::time_when_submitted

The time at which this task was added to the queue.

timestamp_t work_queue_task::time_when_done

The time at which the task is mark as retrieved, after transfering output files and other final processing.

int work_queue_task::disk_allocation_exhausted

Non-zero if a task filled its loop device allocation, zero otherwise.

int64_t work_queue_task::min_running_time

Minimum time (in seconds) the task needs to run.

(see work_queue_worker –wall-time) All fields of the form time_* in microseconds.

timestamp_t work_queue_task::time_when_commit_start

The time when the task starts to be transfered to a worker.

timestamp_t work_queue_task::time_when_commit_end

The time when the task is completely transfered to a worker.

timestamp_t work_queue_task::time_when_retrieval

The time when output files start to be transfered back to the manager.

time_done - time_when_retrieval is the time taken to transfer output files.

timestamp_t work_queue_task::time_workers_execute_last

Duration of the last complete execution for this task.

timestamp_t work_queue_task::time_workers_execute_all

Accumulated time for executing the command on any worker, regardless of whether the task completed (i.e., this includes time running on workers that disconnected).

timestamp_t work_queue_task::time_workers_execute_exhaustion

Accumulated time spent in attempts that exhausted resources.

timestamp_t work_queue_task::time_workers_execute_failure

Accumulated time for runs that terminated in worker failure/disconnection.

int64_t work_queue_task::bytes_received

Number of bytes received since task has last started receiving input data.

int64_t work_queue_task::bytes_sent

Number of bytes sent since task has last started sending input data.

int64_t work_queue_task::bytes_transferred

Number of bytes transferred since task has last started transferring input data.

struct rmsummary* work_queue_task::resources_allocated

Resources allocated to the task its latest attempt.

struct rmsummary* work_queue_task::resources_measured

When monitoring is enabled, it points to the measured resources used by the task in its latest attempt.

struct rmsummary* work_queue_task::resources_requested

Number of cores, disk, memory, time, etc.

the task requires.

char* work_queue_task::monitor_output_directory

Custom output directory for the monitoring output files.

If NULL, save to directory from work_queue_enable_monitoring

char* work_queue_task::monitor_snapshot_file

Filename the monitor checks to produce snapshots.

struct list* work_queue_task::features

User-defined features this task requires.

(See work_queue_worker's –feature option.)

timestamp_t work_queue_task::time_task_submit
Deprecated:
Use time_when_submitted.
timestamp_t work_queue_task::time_task_finish
Deprecated:
Use time_when_done.
timestamp_t work_queue_task::time_committed
Deprecated:
Use time_when_commit_start.
timestamp_t work_queue_task::time_send_input_start
Deprecated:
Use time_when_commit_start.
timestamp_t work_queue_task::time_send_input_finish
Deprecated:
Use time_when_commit_end.
timestamp_t work_queue_task::time_receive_result_start
Deprecated:
Use time_when_retrieval instead.
timestamp_t work_queue_task::time_receive_result_finish
Deprecated:
Use time_when_done instead.
timestamp_t work_queue_task::time_receive_output_start
Deprecated:
Use time_when_retrieval.
timestamp_t work_queue_task::time_receive_output_finish
Deprecated:
Use time_when_done instead.
timestamp_t work_queue_task::time_execute_cmd_start
Deprecated:
Use time_when_commit_end instead.
timestamp_t work_queue_task::time_execute_cmd_finish
Deprecated:
Use time_when_retrieval instead.
timestamp_t work_queue_task::total_transfer_time
Deprecated:
Use (time_when_commit_end - time_when_commit_start) + (time_when_done - time_when_retrieval).
timestamp_t work_queue_task::cmd_execution_time
Deprecated:
Use time_workers_execute_last instead.
timestamp_t work_queue_task::total_cmd_execution_time
Deprecated:
Use time_workers_execute_all instead.
timestamp_t work_queue_task::total_cmd_exhausted_execute_time
Deprecated:
Use time_workers_execute_exhaustion instead.
timestamp_t work_queue_task::total_time_until_worker_failure
Deprecated:
Use time_workers_execute_failure instead.
int64_t work_queue_task::total_bytes_received
Deprecated:
Use bytes_received instead.
int64_t work_queue_task::total_bytes_sent
Deprecated:
Use bytes_sent instead.
int64_t work_queue_task::total_bytes_transferred
Deprecated:
Use bytes_transferred instead.
timestamp_t work_queue_task::time_app_delay
Deprecated:
The time spent in upper-level application (outside of work_queue_wait).

The documentation for this struct was generated from the following file: