cctools
ndcctools.work_queue.RemoteTask Class Reference

Python RemoteTask object. More...

Inheritance diagram for ndcctools.work_queue.RemoteTask:
ndcctools.work_queue.Task

Public Member Functions

def __init__
 Create a new remote task specification. More...
 
def specify_fn_args
 Specify function arguments. More...
 
def specify_exec_method
 Specify how the remote task should execute. More...
 
- Public Member Functions inherited from ndcctools.work_queue.Task
def __init__
 Create a new task specification. More...
 
def clone
 Return a copy of this task. More...
 
def specify_command
 Set the command to be executed by the task. More...
 
def specify_coprocess
 Set the coprocess at the worker that should execute the task's command. More...
 
def specify_algorithm
 Set the worker selection algorithm for task. More...
 
def specify_tag
 Attach a user defined logical name to the task. More...
 
def specify_category
 Label the task with the given category. More...
 
def specify_feature
 Label the task with the given user-defined feature. More...
 
def specify_file
 Add a file to the task. More...
 
def specify_url
 Add a url to the task which will be provided as an input file. More...
 
def specify_file_command
 Add an input file produced by a Unix shell command. More...
 
def specify_file_piece
 Add a file piece to the task. More...
 
def specify_input_file
 Add a input file to the task. More...
 
def specify_output_file
 Add a output file to the task. More...
 
def specify_directory
 Add a directory to the task. More...
 
def specify_buffer
 Add an input bufer to the task. More...
 
def specify_snapshot_file
 When monitoring, indicates a json-encoded file that instructs the monitor to take a snapshot of the task resources. More...
 
def specify_max_retries
 Indicate the number of times the task should be retried. More...
 
def specify_cores
 Indicate the number of cores required by this task. More...
 
def specify_memory
 Indicate the memory (in MB) required by this task. More...
 
def specify_disk
 Indicate the disk space (in MB) required by this task. More...
 
def specify_gpus
 Indicate the number of GPUs required by this task. More...
 
def specify_priority
 Indicate the the priority of this task (larger means better priority, default is 0). More...
 
def specify_environment_variable
 Set this environment variable before running the task. More...
 
def specify_monitor_output
 Set a name for the resource summary output directory from the monitor. More...
 
def tag
 Get the user-defined logical name for the task. More...
 
def category
 Get the category name for the task. More...
 
def command
 Get the shell command executed by the task. More...
 
def priority
 Get the priority of the task. More...
 
def algorithm
 Get the algorithm for choosing worker to run the task. More...
 
def std_output
 Get the standard output of the task. More...
 
def output
 Get the standard output of the task. More...
 
def id
 Get the task id number. More...
 
def return_status
 Get the exit code of the command executed by the task. More...
 
def result
 Get the result of the task as an integer code, such as successful, missing file, etc. More...
 
def result_str
 Return a string that explains the result of a task. More...
 
def total_submissions
 Get the number of times the task has been resubmitted internally. More...
 
def exhausted_attempts
 Get the number of times the task has been failed given resource exhaustion. More...
 
def host
 Get the address and port of the host on which the task ran. More...
 
def hostname
 Get the name of the host on which the task ran. More...
 
def submit_time
 Get the time at which this task was submitted. More...
 
def finish_time
 Get the time at which this task was finished. More...
 
def total_cmd_exhausted_execute_time
 Get the total time the task executed and failed given resource exhaustion. More...
 
def app_delay
 Get the time spent in upper-level application (outside of work_queue_wait). More...
 
def send_input_start
 Get the time at which the task started to transfer input files. More...
 
def send_input_finish
 Get the time at which the task finished transferring input files. More...
 
def execute_cmd_start
 The time at which the task began. More...
 
def execute_cmd_finish
 Get the time at which the task finished (discovered by the manager). More...
 
def receive_output_start
 Get the time at which the task started to transfer output files. More...
 
def receive_output_finish
 Get the time at which the task finished transferring output files. More...
 
def total_bytes_received
 Get the number of bytes received since task started receiving input data. More...
 
def total_bytes_sent
 Get the number of bytes sent since task started sending input data. More...
 
def total_bytes_transferred
 Get the number of bytes transferred since task started transferring input data. More...
 
def total_transfer_time
 Get the time comsumed in microseconds for transferring total_bytes_transferred. More...
 
def cmd_execution_time
 Time spent in microseconds for executing the command until completion on a single worker. More...
 
def total_cmd_execution_time
 Accumulated time spent in microseconds for executing the command on any worker, regardless of whether the task finished (i.e., this includes time running on workers that disconnected). More...
 
def resources_measured
 Get the resources measured for the task execution if resource monitoring is enabled. More...
 
def limits_exceeded
 Get the resources the task exceeded. More...
 
def resources_requested
 Get the resources the task requested to run. More...
 
def resources_allocated
 Get the resources allocated to the task in its latest attempt. More...
 

Detailed Description

Python RemoteTask object.

This class is used to create a task that will execute on a worker running a coprocess

Constructor & Destructor Documentation

def ndcctools.work_queue.RemoteTask.__init__ (   self,
  fn,
  coprocess,
  args,
  kwargs 
)

Create a new remote task specification.

Parameters
selfReference to the current remote task object.
fnThe name of the function to be executed on the coprocess
coprocessThe name of the coprocess which has the function you wish to execute. The coprocess should have a name() method that returns this
argspositional arguments used in function to be executed by task. Can be mixed with kwargs
kwargskeyword arguments used in function to be executed by task.

Member Function Documentation

def ndcctools.work_queue.RemoteTask.specify_fn_args (   self,
  args = [],
  kwargs = {} 
)

Specify function arguments.

Accepts arrays and dictionarys. This overrides any arguments passed during task creation

Parameters
selfReference to the current remote task object
argsAn array of positional args to be passed to the function
kwargsA dictionary of keyword arguments to be passed to the function
def ndcctools.work_queue.RemoteTask.specify_exec_method (   self,
  remote_task_exec_method 
)

Specify how the remote task should execute.

Parameters
selfReference to the current remote task object
remote_task_exec_methodCan be one of "fork", "direct", or "thread". Fork creates a child process to execute the function, direct has the worker directly call the function, and thread spawns a thread to execute the function

The documentation for this class was generated from the following file: