This class is used to create a task specification.
Public Member Functions | |
def | __init__ |
Create a new task specification. | |
def | clone |
Return a copy of this task. | |
def | specify_command |
Set the command to be executed by the task. | |
def | specify_algorithm |
Set the worker selection algorithm for task. | |
def | specify_tag |
Attach a user defined logical name to the task. | |
def | specify_category |
Label the task with the given category. | |
def | specify_preferred_host |
Indicate that the task would be optimally run on a given host. | |
def | specify_file |
Add a file to the task. | |
def | specify_file_piece |
Add a file piece to the task. | |
def | specify_input_file |
Add a input file to the task. | |
def | specify_output_file |
Add a output file to the task. | |
def | specify_directory |
Add a directory to the task. | |
def | specify_buffer |
Add an input bufer to the task. | |
def | specify_max_retries |
Indicate the number of times the task should be retried. | |
def | specify_cores |
Indicate the number of cores required by this task. | |
def | specify_memory |
Indicate the memory (in MB) required by this task. | |
def | specify_disk |
Indicate the disk space (in MB) required by this task. | |
def | specify_priority |
Indicate the the priority of this task (larger means better priority, default is 0). | |
def | tag |
Get the user-defined logical name for the task. | |
def | category |
Get the category name for the task. | |
def | command |
Get the shell command executed by the task. | |
def | priority |
Get the priority of the task. | |
def | algorithm |
Get the algorithm for choosing worker to run the task. | |
def | output |
Get the standard output of the task. | |
def | id |
Get the task id number. | |
def | return_status |
Get the exit code of the command executed by the task. | |
def | result |
Get the result of the task, such as successful, missing file, etc. | |
def | total_submissions |
Get the number of times the task has been resubmitted internally. | |
def | host |
Get the address and port of the host on which the task ran. | |
def | hostname |
Get the name of the host on which the task ran. | |
def | submit_time |
Get the time at which this task was submitted. | |
def | finish_time |
Get the time at which this task was finished. | |
def | app_delay |
Get the time spent in upper-level application (outside of work_queue_wait). | |
def | send_input_start |
Get the time at which the task started to transfer input files. | |
def | send_input_finish |
Get the time at which the task finished transferring input files. | |
def | execute_cmd_start |
The time at which the task began. | |
def | execute_cmd_finish |
Get the time at which the task finished (discovered by the master). | |
def | receive_output_start |
Get the time at which the task started to transfer output files. | |
def | receive_output_finish |
Get the time at which the task finished transferring output files. | |
def | total_bytes_received |
Get the number of bytes received since task started receiving input data. | |
def | total_bytes_sent |
Get the number of bytes sent since task started sending input data. | |
def | total_bytes_transferred |
Get the number of bytes transferred since task started transferring input data. | |
def | total_transfer_time |
Get the time comsumed in microseconds for transferring total_bytes_transferred. | |
def | cmd_execution_time |
Time spent in microseconds for executing the command until completion on a single worker. | |
def | total_cmd_execution_time |
Accumulated time spent in microseconds for executing the command on any worker, regardless of whether the task finished (i.e., this includes time running on workers that disconnected). | |
def | resources_measured |
Get the resources measured for the task execution if resource monitoring is enabled. | |
def | resources_requested |
Get the resources the task requested to run. |
def work_queue::Task::__init__ | ( | self, | ||
command | ||||
) |
Create a new task specification.
self | Reference to the current task object. | |
command | The shell command line to be exected by the task. |
def work_queue::Task::clone | ( | self | ) |
Return a copy of this task.
def work_queue::Task::specify_command | ( | self, | ||
command | ||||
) |
Set the command to be executed by the task.
self | Reference to the current task object. | |
command | The command to be executed. |
def work_queue::Task::specify_algorithm | ( | self, | ||
algorithm | ||||
) |
Set the worker selection algorithm for task.
self | Reference to the current task object. | |
algorithm | One of the following algorithms to use in assigning a task to a worker. See work_queue_schedule_t for possible values. |
def work_queue::Task::specify_tag | ( | self, | ||
tag | ||||
) |
Attach a user defined logical name to the task.
self | Reference to the current task object. | |
tag | The tag to attach to task. |
def work_queue::Task::specify_category | ( | self, | ||
name | ||||
) |
Label the task with the given category.
It is expected that tasks with the same category have similar resources requirements (e.g. for fast abort).
self | Reference to the current task object. | |
name | The name of the category |
def work_queue::Task::specify_preferred_host | ( | self, | ||
hostname | ||||
) |
Indicate that the task would be optimally run on a given host.
self | Reference to the current task object. | |
hostname | The hostname to which this task would optimally be sent. |
def work_queue::Task::specify_file | ( | self, | ||
local_name, | ||||
remote_name = None , |
||||
type = None , |
||||
flags = None , |
||||
cache = None | ||||
) |
Add a file to the task.
self | Reference to the current task object. | |
local_name | The name of the file on local disk or shared filesystem. | |
remote_name | The name of the file at the execution site. | |
type | Must be one of the following values: WORK_QUEUE_INPUT or WORK_QUEUE_OUTPUT | |
flags | May be zero to indicate no special handling, or any of the work_queue_file_flags_t or'd together The most common are: | |
cache | Legacy parameter for setting file caching attribute. If both flags and cache are undefined, the file is cached. |
# The following are equivalent >>> task.specify_file("/etc/hosts", type=WORK_QUEUE_INPUT, flags=WORK_QUEUE_NOCACHE) >>> task.specify_file("/etc/hosts", "hosts", type=WORK_QUEUE_INPUT, cache=false)
def work_queue::Task::specify_file_piece | ( | self, | ||
local_name, | ||||
remote_name = None , |
||||
start_byte = 0 , |
||||
end_byte = 0 , |
||||
type = None , |
||||
flags = None , |
||||
cache = None | ||||
) |
Add a file piece to the task.
self | Reference to the current task object. | |
local_name | The name of the file on local disk or shared filesystem. | |
remote_name | The name of the file at the execution site. | |
start_byte | The starting byte offset of the file piece to be transferred. | |
end_byte | The ending byte offset of the file piece to be transferred. | |
type | Must be one of the following values: WORK_QUEUE_INPUT or WORK_QUEUE_OUTPUT | |
flags | May be zero to indicate no special handling, or any of the work_queue_file_flags_t or'd together The most common are: | |
cache | Legacy parameter for setting file caching attribute. If both flags and cache are undefined, the file is cached. |
def work_queue::Task::specify_input_file | ( | self, | ||
local_name, | ||||
remote_name = None , |
||||
flags = None , |
||||
cache = None | ||||
) |
Add a input file to the task.
This is just a wrapper for specify_file with type set to WORK_QUEUE_INPUT.
def work_queue::Task::specify_output_file | ( | self, | ||
local_name, | ||||
remote_name = None , |
||||
flags = None , |
||||
cache = None | ||||
) |
Add a output file to the task.
This is just a wrapper for specify_file with type set to WORK_QUEUE_OUTPUT.
def work_queue::Task::specify_directory | ( | self, | ||
local_name, | ||||
remote_name = None , |
||||
type = None , |
||||
flags = None , |
||||
recursive = 0 , |
||||
cache = None | ||||
) |
Add a directory to the task.
self | Reference to the current task object. | |
local_name | The name of the directory on local disk or shared filesystem. Optional if the directory is empty. | |
remote_name | The name of the directory at the remote execution site. | |
type | Must be one of the following values: WORK_QUEUE_INPUT or WORK_QUEUE_OUTPUT | |
flags | May be zero to indicate no special handling, or any of the work_queue_file_flags_t or'd together The most common are: | |
recursive | Indicates whether just the directory (0) or the directory and all of its contents (1) should be included. | |
cache | Legacy parameter for setting file caching attribute. If both flags and cache are undefined, the file is cache. |
def work_queue::Task::specify_buffer | ( | self, | ||
buffer, | ||||
remote_name, | ||||
flags = None , |
||||
cache = None | ||||
) |
Add an input bufer to the task.
self | Reference to the current task object. | |
buffer | The contents of the buffer to pass as input. | |
remote_name | The name of the remote file to create. | |
flags | May take the same values as specify_file. | |
cache | Legacy parameter for setting buffer caching attribute. If both flags and cache are undefined (i.e., None), the buffer is cached. |
def work_queue::Task::specify_max_retries | ( | self, | ||
max_retries | ||||
) |
Indicate the number of times the task should be retried.
If 0 (the default), the task is tried indefinitely. A task that did not succeed after the given number of retries is returned with result WORK_QUEUE_RESULT_MAX_RETRIES.
def work_queue::Task::specify_cores | ( | self, | ||
cores | ||||
) |
Indicate the number of cores required by this task.
def work_queue::Task::specify_memory | ( | self, | ||
memory | ||||
) |
Indicate the memory (in MB) required by this task.
def work_queue::Task::specify_disk | ( | self, | ||
disk | ||||
) |
Indicate the disk space (in MB) required by this task.
def work_queue::Task::specify_priority | ( | self, | ||
priority | ||||
) |
Indicate the the priority of this task (larger means better priority, default is 0).
def work_queue::Task::tag | ( | self | ) |
Get the user-defined logical name for the task.
Note: This is defined using property decorator. So it must be called without parentheses (). For example:
>>> print t.tag
def work_queue::Task::category | ( | self | ) |
Get the category name for the task.
Note: This is defined using property decorator. So it must be called without parentheses (). For example:
>>> print t.category
def work_queue::Task::command | ( | self | ) |
Get the shell command executed by the task.
Note: This is defined using property decorator. So it must be called without parentheses (). For example:
>>> print t.command
def work_queue::Task::priority | ( | self | ) |
Get the priority of the task.
Note: This is defined using property decorator. So it must be called without parentheses (). For example:
>>> print t.priority
def work_queue::Task::algorithm | ( | self | ) |
Get the algorithm for choosing worker to run the task.
Note: This is defined using property decorator. So it must be called without parentheses (). For example:
>>> print t.algorithm
def work_queue::Task::output | ( | self | ) |
Get the standard output of the task.
Must be called only after the task completes execution. Note: This is defined using property decorator. So it must be called without parentheses (). For example:
>>> print t.output
def work_queue::Task::id | ( | self | ) |
Get the task id number.
Must be called only after the task was submitted. Note: This is defined using property decorator. So it must be called without parentheses (). For example:
>>> print t.id
def work_queue::Task::return_status | ( | self | ) |
Get the exit code of the command executed by the task.
Must be called only after the task completes execution. Note: This is defined using property decorator. So it must be called without parentheses (). For example:
>>> print t.return_status
def work_queue::Task::result | ( | self | ) |
Get the result of the task, such as successful, missing file, etc.
See work_queue_result_t for possible values. Must be called only after the task completes execution. Note: This is defined using property decorator. So it must be called without parentheses (). For example:
>>> print t.result
def work_queue::Task::total_submissions | ( | self | ) |
Get the number of times the task has been resubmitted internally.
Must be called only after the task completes execution. Note: This is defined using property decorator. So it must be called without parentheses (). For example:
>>> print t.total_submissions
def work_queue::Task::host | ( | self | ) |
Get the address and port of the host on which the task ran.
Must be called only after the task completes execution. Note: This is defined using property decorator. So it must be called without parentheses (). For example:
>>> print t.host
def work_queue::Task::hostname | ( | self | ) |
Get the name of the host on which the task ran.
Must be called only after the task completes execution. Note: This is defined using property decorator. So it must be called without parentheses (). For example:
>>> print t.hostname
def work_queue::Task::submit_time | ( | self | ) |
Get the time at which this task was submitted.
Must be called only after the task completes execution. Note: This is defined using property decorator. So it must be called without parentheses (). For example:
>>> print t.submit_time
def work_queue::Task::finish_time | ( | self | ) |
Get the time at which this task was finished.
Must be called only after the task completes execution. Note: This is defined using property decorator. So it must be called without parentheses (). For example:
>>> print t.finish_time
def work_queue::Task::app_delay | ( | self | ) |
Get the time spent in upper-level application (outside of work_queue_wait).
Must be called only after the task completes execution. Note: This is defined using property decorator. So it must be called without parentheses (). For example:
>>> print t.app_delay
def work_queue::Task::send_input_start | ( | self | ) |
Get the time at which the task started to transfer input files.
Must be called only after the task completes execution. Note: This is defined using property decorator. So it must be called without parentheses (). For example:
>>> print t.send_input_start
def work_queue::Task::send_input_finish | ( | self | ) |
Get the time at which the task finished transferring input files.
Must be called only after the task completes execution. Note: This is defined using property decorator. So it must be called without parentheses (). For example:
>>> print t.send_input_finish
def work_queue::Task::execute_cmd_start | ( | self | ) |
The time at which the task began.
Must be called only after the task completes execution. Note: This is defined using property decorator. So it must be called without parentheses (). For example:
>>> print t.execute_cmd_start
def work_queue::Task::execute_cmd_finish | ( | self | ) |
Get the time at which the task finished (discovered by the master).
Must be called only after the task completes execution. Note: This is defined using property decorator. So it must be called without parentheses (). For example:
>>> print t.execute_cmd_finish
def work_queue::Task::receive_output_start | ( | self | ) |
Get the time at which the task started to transfer output files.
Must be called only after the task completes execution. Note: This is defined using property decorator. So it must be called without parentheses (). For example:
>>> print t.receive_output_start
def work_queue::Task::receive_output_finish | ( | self | ) |
Get the time at which the task finished transferring output files.
Must be called only after the task completes execution. Note: This is defined using property decorator. So it must be called without parentheses (). For example:
>>> print t.receive_output_finish
def work_queue::Task::total_bytes_received | ( | self | ) |
Get the number of bytes received since task started receiving input data.
Must be called only after the task completes execution. Note: This is defined using property decorator. So it must be called without parentheses (). For example:
>>> print t.total_bytes_received
def work_queue::Task::total_bytes_sent | ( | self | ) |
Get the number of bytes sent since task started sending input data.
Must be called only after the task completes execution. Note: This is defined using property decorator. So it must be called without parentheses (). For example:
>>> print t.total_bytes_sent
def work_queue::Task::total_bytes_transferred | ( | self | ) |
Get the number of bytes transferred since task started transferring input data.
Must be called only after the task completes execution. Note: This is defined using property decorator. So it must be called without parentheses (). For example:
>>> print t.total_bytes_transferred
def work_queue::Task::total_transfer_time | ( | self | ) |
Get the time comsumed in microseconds for transferring total_bytes_transferred.
Must be called only after the task completes execution. Note: This is defined using property decorator. So it must be called without parentheses (). For example:
>>> print t.total_transfer_time
def work_queue::Task::cmd_execution_time | ( | self | ) |
Time spent in microseconds for executing the command until completion on a single worker.
Must be called only after the task completes execution. Note: This is defined using property decorator. So it must be called without parentheses (). For example:
>>> print t.cmd_execution_time
def work_queue::Task::total_cmd_execution_time | ( | self | ) |
Accumulated time spent in microseconds for executing the command on any worker, regardless of whether the task finished (i.e., this includes time running on workers that disconnected).
Must be called only after the task completes execution. Note: This is defined using property decorator. So it must be called without parentheses (). For example:
>>> print t.total_cmd_execution_time
def work_queue::Task::resources_measured | ( | self | ) |
Get the resources measured for the task execution if resource monitoring is enabled.
Must be called only after the task completes execution. Valid fields:
start: microseconds at the start of execution end: microseconds at the end of execution wall_time: microseconds spent during execution cpu_time: user + system time of the execution cores: number of cores. Computed as cpu_time/wall_time max_concurrent_processes: the maximum number of processes running concurrently total_processes: count of all of the processes created virtual_memory: maximum virtual memory across all processes resident_memory: maximum resident size across all processes swap_memory: maximum swap usage across all processes bytes_read: number of bytes read from disk bytes_written: number of bytes written to disk bytes_received: number of bytes read from the network bytes_send: number of bytes written to the network bandwidth: maximum network bits/s (average over one minute) workdir_num_files: total maximum number of files and directories of all the working directories in the tree workdir_footprint: size in MB of all working directories in the tree
>>> print t.resources_measured.resident_memory
def work_queue::Task::resources_requested | ( | self | ) |
Get the resources the task requested to run.
For valid fields see .