cctools
|
Public Member Functions | |
def | __init__ |
Creates a new python task. More... | |
def | submit_finalize |
Finalizes the task definition once the manager that will execute is run. More... | |
def | enable_temp_output |
Marks the output of this task to stay at the worker. More... | |
def | set_output_cache |
Set the cache behavior for the output of the task. More... | |
def | output_file |
Returns the ndcctools.taskvine.file.File object that represents the output of this task. More... | |
def | output |
returns the result of a python task as a python variable More... | |
def | disable_output_serialization |
Disables serialization of results to disk when writing to a file for transmission. More... | |
![]() | |
def | __init__ |
Create a new task specification. More... | |
def | submit_finalize |
Finalizes the task definition once the manager that will execute is run. More... | |
def | clone |
Return a copy of this task. More... | |
def | set_command |
Set the command to be executed by the task. More... | |
def | set_coprocess |
Set the coprocess at the worker that should execute the task's command. More... | |
def | set_scheduler |
Set the worker selection scheduler for task. More... | |
def | set_tag |
Attach a user defined logical name to the task. More... | |
def | set_category |
Label the task with the given category. More... | |
def | add_feature |
Label the task with the given user-defined feature. More... | |
def | add_input |
Add any input object to a task. More... | |
def | add_output |
Add any output object to a task. More... | |
def | set_snapshot_file |
When monitoring, indicates a json-encoded file that instructs the monitor to take a snapshot of the task resources. More... | |
def | add_environment |
Adds an execution environment to the task. More... | |
def | set_retries |
Indicate the number of times the task should be retried. More... | |
def | set_cores |
Indicate the number of cores required by this task. More... | |
def | set_memory |
Indicate the memory (in MB) required by this task. More... | |
def | set_disk |
Indicate the disk space (in MB) required by this task. More... | |
def | set_gpus |
Indicate the number of GPUs required by this task. More... | |
def | set_priority |
Indicate the the priority of this task (larger means better priority, default is 0). More... | |
def | set_env_var |
Set this environment variable before running the task. More... | |
def | set_monitor_output |
Set a name for the resource summary output directory from the monitor. More... | |
def | tag |
Get the user-defined logical name for the task. More... | |
def | category |
Get the category name for the task. More... | |
def | command |
Get the shell command executed by the task. More... | |
def | std_output |
Get the standard output of the task. More... | |
def | output |
Get the standard output of the task. More... | |
def | id |
Get the task id number. More... | |
def | exit_code |
Get the exit code of the command executed by the task. More... | |
def | result |
Return a string that explains the result of a task. More... | |
def | completed |
Return True if task executed and its command terminated normally. More... | |
def | successful |
Return True if task executed successfully, (i.e. More... | |
def | get_metric |
Return various integer performance metrics about a completed task. More... | |
def | addrport |
Get the address and port of the host on which the task ran. More... | |
def | hostname |
Get the address and port of the host on which the task ran. More... | |
def | resources_measured |
Get the resources measured for the task execution if resource monitoring is enabled. More... | |
def | limits_exceeded |
Get the resources the task exceeded. More... | |
def | resources_requested |
Get the resources the task requested to run. More... | |
def | resources_allocated |
Get the resources allocated to the task in its latest attempt. More... | |
def ndcctools.taskvine.task.PythonTask.__init__ | ( | self, | |
func, | |||
args, | |||
kwargs | |||
) |
Creates a new python task.
self | Reference to the current python task object |
func | python function to be executed by task |
args | arguments used in function to be executed by task |
kwargs | keyword arguments used in function to be executed by task |
def ndcctools.taskvine.task.PythonTask.submit_finalize | ( | self, | |
manager | |||
) |
Finalizes the task definition once the manager that will execute is run.
This function is run by the manager before registering the task for execution.
self | Reference to the current python task object |
manager | Manager to which the task was submitted |
def ndcctools.taskvine.task.PythonTask.enable_temp_output | ( | self | ) |
Marks the output of this task to stay at the worker.
Functions that consume the output of this tasks have to add self.output_file as an input, and cloudpickle.load() it.
E.g.:
where fn_with_tmp may look something like this:
self | Reference to the current python task object |
def ndcctools.taskvine.task.PythonTask.set_output_cache | ( | self, | |
cache = False |
|||
) |
Set the cache behavior for the output of the task.
cache | If True or 'workflow', cache the file at workers for reuse until the end of the workflow. If 'always', the file is cache until the end-of-life of the worker. Default is False (file is not cache). |
def ndcctools.taskvine.task.PythonTask.output_file | ( | self | ) |
Returns the ndcctools.taskvine.file.File object that represents the output of this task.
def ndcctools.taskvine.task.PythonTask.output | ( | self | ) |
returns the result of a python task as a python variable
self | reference to the current python task object |
def ndcctools.taskvine.task.PythonTask.disable_output_serialization | ( | self | ) |
Disables serialization of results to disk when writing to a file for transmission.
WARNING: Only do this if the function itself encodes the output in a way amenable for serialization.
self | Reference to the current python task object |