cctools
Public Member Functions
resource_monitor.Categories Class Reference

Class to encapsule all the categories in a workflow. More...

Public Member Functions

def __init__ (self, all_categories_name='(all)')
 Create an empty set of categories. More...
 
def category_names (self)
 Returns a lists of the category categories. More...
 
def first_allocation (self, mode, category)
 Compute and return the first allocations for the given category. More...
 
def maximum_seen (self, category)
 Return the maximum resource values so far seen for the given category. More...
 
def accumulate_summary (self, summary)
 Add the summary (a dictionary) to the respective category. More...
 
def waste (self, category, field, allocation)
 Return the waste (unit x time) that would be produced if the accumulated summaries were run under the given allocation. More...
 
def wastepercentage (self, category, field, allocation)
 Return the percentage of wasted resources that would be produced if the accumulated summaries were run under the given allocation. More...
 
def throughput (self, category, field, allocation)
 Return the throughput that would be obtained if the accumulated summaries were run under the given allocation. More...
 
def retries (self, category, field, allocation)
 Return the number of tasks that would be retried if the accumulated summaries were run under the given allocation. More...
 
def count (self, category)
 Return the number of summaries in a particular category. More...
 

Detailed Description

Class to encapsule all the categories in a workflow.

cs = Categories()
cs.accumulate_summary( { 'category': 'some_category', 'wall_time': 60, 'cores': 1, ... } )
print(cs.first_allocation(mode = 'throughput', category = 'some_category'))

Constructor & Destructor Documentation

◆ __init__()

def resource_monitor.Categories.__init__ (   self,
  all_categories_name = '(all)' 
)

Create an empty set of categories.

Parameters
selfReference to the current object.
all_categories_nameName of the general category that holds all of the summaries.

References resource_monitor.Categories.all_categories_name, and resource_monitor.Categories.categories.

Member Function Documentation

◆ category_names()

def resource_monitor.Categories.category_names (   self)

Returns a lists of the category categories.

List sorted lexicographicaly, with the exception of self.all_categories_name, which it is always the last entry.

Parameters
selfReference to the current object.

References resource_monitor.Categories.all_categories_name, and resource_monitor.Categories.categories.

◆ first_allocation()

def resource_monitor.Categories.first_allocation (   self,
  mode,
  category 
)

Compute and return the first allocations for the given category.

Note: wall_time needs to be defined in the resource summaries to be considered in this optimization.

Parameters
selfReference to the current object.
modeOptimization mode. One of 'throughput', 'waste', or 'fixed'.
categoryName of the category
cs = Categories()
fa = cs.first_allocation(mode = 'throughput, category = 'some_category')
print(fa['cores'])
print(fa['memory'])
print(fa['disk'])

References resource_monitor.Categories._category().

◆ maximum_seen()

def resource_monitor.Categories.maximum_seen (   self,
  category 
)

Return the maximum resource values so far seen for the given category.

Parameters
selfReference to the current object.
categoryName of the category
cs = Categories()
fa = cs.maximum_seen('some_category')
print(fa['cores'])
print(fa['memory'])
print(fa['disk'])

References resource_monitor.Categories._category().

◆ accumulate_summary()

def resource_monitor.Categories.accumulate_summary (   self,
  summary 
)

Add the summary (a dictionary) to the respective category.

At least both the 'category' and 'wall_time' keys should be defined.

cs = Categories()
cs.accumulate_summary( { 'category': 'some_category', 'wall_time': 50, 'cores': 1, ... } )

References resource_monitor.Categories._category(), and resource_monitor.Categories.all_categories_name.

◆ waste()

def resource_monitor.Categories.waste (   self,
  category,
  field,
  allocation 
)

Return the waste (unit x time) that would be produced if the accumulated summaries were run under the given allocation.

Parameters
selfReference to the current object.
categoryName of the category
fieldName of the resource (e.g., cores, memory, or disk)
allocationValue of allocation to test.

References resource_monitor.Categories._category().

◆ wastepercentage()

def resource_monitor.Categories.wastepercentage (   self,
  category,
  field,
  allocation 
)

Return the percentage of wasted resources that would be produced if the accumulated summaries were run under the given allocation.

Parameters
selfReference to the current object.
categoryName of the category
fieldName of the resource (e.g., cores, memory, or disk)
allocationValue of allocation to test.

References resource_monitor.Categories._category().

◆ throughput()

def resource_monitor.Categories.throughput (   self,
  category,
  field,
  allocation 
)

Return the throughput that would be obtained if the accumulated summaries were run under the given allocation.

Parameters
selfReference to the current object.
categoryName of the category
fieldName of the resource (e.g., cores, memory, or disk)
allocationValue of allocation to test.

References resource_monitor.Categories._category().

◆ retries()

def resource_monitor.Categories.retries (   self,
  category,
  field,
  allocation 
)

Return the number of tasks that would be retried if the accumulated summaries were run under the given allocation.

Parameters
selfReference to the current object.
categoryName of the category
fieldName of the resource (e.g., cores, memory, or disk)
allocationValue of allocation to test.

References resource_monitor.Categories._category().

◆ count()

def resource_monitor.Categories.count (   self,
  category 
)

Return the number of summaries in a particular category.

Parameters
selfReference to the current object.
categoryName of the category

References resource_monitor.Categories._category(), and resource_monitor.Categories.categories.


The documentation for this class was generated from the following file: