wrench::SimulationOutput Class Reference

A class that contains post-mortem simulation-generated data. More...

#include <SimulationOutput.h>

Public Member Functions

template<class T >
void addTimestamp (T *timestamp)
 Append a simulation timestamp to a simulation output trace. More...
 
void addTimestampDiskReadCompletion (std::string hostname, std::string mount, double bytes, int unique_sequence_number)
 Add a file read completion timestamp. More...
 
void addTimestampDiskReadFailure (std::string hostname, std::string mount, double bytes, int unique_sequence_number)
 Add a file read failure timestamp. More...
 
void addTimestampDiskReadStart (std::string hostname, std::string mount, double bytes, int unique_sequence_number)
 Add a file read start timestamp. More...
 
void addTimestampDiskWriteCompletion (std::string hostname, std::string mount, double bytes, int unique_sequence_number)
 Add a file write completion timestamp. More...
 
void addTimestampDiskWriteFailure (std::string hostname, std::string mount, double bytes, int unique_sequence_number)
 Add a file write failure timestamp. More...
 
void addTimestampDiskWriteStart (std::string hostname, std::string mount, double bytes, int unique_sequence_number)
 Add a file write start timestamp. More...
 
void addTimestampEnergyConsumption (std::string hostname, double joules)
 Add an energy consumption timestamp. More...
 
void addTimestampFileCopyCompletion (WorkflowFile *file, std::shared_ptr< FileLocation > src, std::shared_ptr< FileLocation > dst)
 Add a file copy completion timestamp. More...
 
void addTimestampFileCopyFailure (WorkflowFile *file, std::shared_ptr< FileLocation > src, std::shared_ptr< FileLocation > dst)
 Add a file copy failure timestamp. More...
 
void addTimestampFileCopyStart (WorkflowFile *file, std::shared_ptr< FileLocation > src, std::shared_ptr< FileLocation > dst)
 Add a file copy start timestamp. More...
 
void addTimestampFileReadCompletion (WorkflowFile *file, FileLocation *src, StorageService *service, WorkflowTask *task=nullptr)
 Add a file read completion timestamp. More...
 
void addTimestampFileReadFailure (WorkflowFile *file, FileLocation *src, StorageService *service, WorkflowTask *task=nullptr)
 Add a file read failure timestamp. More...
 
void addTimestampFileReadStart (WorkflowFile *file, FileLocation *src, StorageService *service, WorkflowTask *task=nullptr)
 Add a file read start timestamp. More...
 
void addTimestampFileWriteCompletion (WorkflowFile *file, FileLocation *src, StorageService *service, WorkflowTask *task=nullptr)
 Add a file write completion timestamp. More...
 
void addTimestampFileWriteFailure (WorkflowFile *file, FileLocation *src, StorageService *service, WorkflowTask *task=nullptr)
 Add a file write failure timestamp. More...
 
void addTimestampFileWriteStart (WorkflowFile *file, FileLocation *src, StorageService *service, WorkflowTask *task=nullptr)
 Add a file write start timestamp. More...
 
void addTimestampLinkUsage (std::string linkname, double bytes_per_second)
 Add a link usage timestamp. More...
 
void addTimestampPstateSet (std::string hostname, int pstate)
 Add a pstate change/set timestamp. More...
 
void addTimestampTaskCompletion (WorkflowTask *task)
 Add a task start completion. More...
 
void addTimestampTaskFailure (WorkflowTask *task)
 Add a task start failure. More...
 
void addTimestampTaskStart (WorkflowTask *task)
 Add a task start timestamp. More...
 
void addTimestampTaskTermination (WorkflowTask *task)
 Add a task start termination. More...
 
void dumpDiskOperationsJSON (std::string file_path, bool writing_file=true)
 Writes a JSON file containing disk operation information as a JSON array. More...
 
void dumpHostEnergyConsumptionJSON (std::string file_path, bool writing_file=true)
 Writes a JSON file containing host energy consumption information as a JSON array. More...
 
void dumpLinkUsageJSON (std::string file_path, bool writing_file=true)
 
void dumpPlatformGraphJSON (std::string file_path, bool writing_file=true)
 Writes a JSON file containing all hosts, network links, and the routes between each host. More...
 
void dumpUnifiedJSON (Workflow *workflow, std::string file_path, bool include_platform=false, bool include_workflow_exec=true, bool include_workflow_graph=false, bool include_energy=false, bool generate_host_utilization_layout=false, bool include_disk=false, bool include_bandwidth=false)
 Function that generates a unified JSON file containing the information specified by boolean arguments. More...
 
void dumpWorkflowExecutionJSON (Workflow *workflow, std::string file_path, bool generate_host_utilization_layout=false, bool writing_file=true)
 Writes WorkflowTask execution history for each task to a file, formatted as a JSON array. More...
 
void dumpWorkflowGraphJSON (wrench::Workflow *workflow, std::string file_path, bool writing_file=true)
 Writes a JSON graph representation of the Workflow to a file. More...
 
void enableBandwidthTimestamps (bool enabled)
 Enable or Disable the insertion of link-usage-related timestamps in the simulation output (enabled by default) More...
 
void enableDiskTimestamps (bool enabled)
 Enable or Disable the insertion of disk-related timestamps in the simulation output (enabled by default) More...
 
void enableEnergyTimestamps (bool enabled)
 Enable or Disable the insertion of energy-related timestamps in the simulation output (enabled by default) More...
 
void enableFileReadWriteCopyTimestamps (bool enabled)
 Enable or Disable the insertion of file-related timestamps in the simulation output (enabled by default) More...
 
void enableWorkflowTaskTimestamps (bool enabled)
 Enable or Disable the insertion of task-related timestamps in the simulation output (enabled by default) More...
 
template<class T >
std::vector< SimulationTimestamp< T > * > getTrace ()
 Retrieve a copy of a simulation output trace once the simulation has completed. More...
 

Detailed Description

A class that contains post-mortem simulation-generated data.

Member Function Documentation

◆ addTimestamp()

template<class T >
void wrench::SimulationOutput::addTimestamp ( T *  timestamp)
inline

Append a simulation timestamp to a simulation output trace.

Template Parameters
aparticular SimulationTimestampXXXX class (defined in SimulationTimestampTypes.h)
Parameters
timestampa pointer to a SimulationTimestampXXXX object

◆ addTimestampDiskReadCompletion()

void wrench::SimulationOutput::addTimestampDiskReadCompletion ( std::string  hostname,
std::string  mount,
double  bytes,
int  unique_sequence_number 
)

Add a file read completion timestamp.

Parameters
hostnamehostname being read from
mountmountpoint of disk
bytesnumber of bytes read
unique_sequence_numberan integer id

◆ addTimestampDiskReadFailure()

void wrench::SimulationOutput::addTimestampDiskReadFailure ( std::string  hostname,
std::string  mount,
double  bytes,
int  unique_sequence_number 
)

Add a file read failure timestamp.

Parameters
hostnamehostname being read from
mountmountpoint of disk
bytesnumber of bytes read
unique_sequence_numberan integer id

◆ addTimestampDiskReadStart()

void wrench::SimulationOutput::addTimestampDiskReadStart ( std::string  hostname,
std::string  mount,
double  bytes,
int  unique_sequence_number 
)

Add a file read start timestamp.

Parameters
hostnamehostname being read from
mountmountpoint of disk
bytesnumber of bytes read
unique_sequence_numberan integer id

◆ addTimestampDiskWriteCompletion()

void wrench::SimulationOutput::addTimestampDiskWriteCompletion ( std::string  hostname,
std::string  mount,
double  bytes,
int  unique_sequence_number 
)

Add a file write completion timestamp.

Parameters
hostnamehostname being read from
mountmountpoint of disk
bytesnumber of bytes read
unique_sequence_numberan integer id

◆ addTimestampDiskWriteFailure()

void wrench::SimulationOutput::addTimestampDiskWriteFailure ( std::string  hostname,
std::string  mount,
double  bytes,
int  unique_sequence_number 
)

Add a file write failure timestamp.

Parameters
hostnamehostname being read from
mountmountpoint of disk
bytesnumber of bytes read
unique_sequence_numberan integer id

◆ addTimestampDiskWriteStart()

void wrench::SimulationOutput::addTimestampDiskWriteStart ( std::string  hostname,
std::string  mount,
double  bytes,
int  unique_sequence_number 
)

Add a file write start timestamp.

Parameters
hostnamehostname being read from
mountmountpoint of disk
bytesnumber of bytes read
unique_sequence_numberan integer id

◆ addTimestampEnergyConsumption()

void wrench::SimulationOutput::addTimestampEnergyConsumption ( std::string  hostname,
double  joules 
)

Add an energy consumption timestamp.

Parameters
hostnamea hostname
joulesconsumption in joules

◆ addTimestampFileCopyCompletion()

void wrench::SimulationOutput::addTimestampFileCopyCompletion ( WorkflowFile file,
std::shared_ptr< FileLocation src,
std::shared_ptr< FileLocation dst 
)

Add a file copy completion timestamp.

Parameters
filea workflow file
srcthe source location
dstthe target location

◆ addTimestampFileCopyFailure()

void wrench::SimulationOutput::addTimestampFileCopyFailure ( WorkflowFile file,
std::shared_ptr< FileLocation src,
std::shared_ptr< FileLocation dst 
)

Add a file copy failure timestamp.

Parameters
filea workflow file
srcthe source location
dstthe target location

◆ addTimestampFileCopyStart()

void wrench::SimulationOutput::addTimestampFileCopyStart ( WorkflowFile file,
std::shared_ptr< FileLocation src,
std::shared_ptr< FileLocation dst 
)

Add a file copy start timestamp.

Parameters
filea workflow file
srcthe source location
dstthe target location

◆ addTimestampFileReadCompletion()

void wrench::SimulationOutput::addTimestampFileReadCompletion ( WorkflowFile file,
FileLocation src,
StorageService service,
WorkflowTask task = nullptr 
)

Add a file read completion timestamp.

Parameters
filea workflow file
srcthe source location
servicethe source storage service
taskthe workflow task for which this read is done (or nullptr);

◆ addTimestampFileReadFailure()

void wrench::SimulationOutput::addTimestampFileReadFailure ( WorkflowFile file,
FileLocation src,
StorageService service,
WorkflowTask task = nullptr 
)

Add a file read failure timestamp.

Parameters
filea workflow file
srcthe source location
servicethe source storage service
taskthe workflow task for which this read is done (or nullptr);

◆ addTimestampFileReadStart()

void wrench::SimulationOutput::addTimestampFileReadStart ( WorkflowFile file,
FileLocation src,
StorageService service,
WorkflowTask task = nullptr 
)

Add a file read start timestamp.

Parameters
filea workflow file
srcthe source location
servicethe source storage service
taskthe workflow task for which this read is done (or nullptr);

◆ addTimestampFileWriteCompletion()

void wrench::SimulationOutput::addTimestampFileWriteCompletion ( WorkflowFile file,
FileLocation src,
StorageService service,
WorkflowTask task = nullptr 
)

Add a file write completion timestamp.

Parameters
filea workflow file
srcthe target location
servicethe target storage service
taskthe workflow task for which this write is done (or nullptr);

◆ addTimestampFileWriteFailure()

void wrench::SimulationOutput::addTimestampFileWriteFailure ( WorkflowFile file,
FileLocation src,
StorageService service,
WorkflowTask task = nullptr 
)

Add a file write failure timestamp.

Parameters
filea workflow file
srcthe target location
servicethe target storage service
taskthe workflow task for which this write is done (or nullptr);

◆ addTimestampFileWriteStart()

void wrench::SimulationOutput::addTimestampFileWriteStart ( WorkflowFile file,
FileLocation src,
StorageService service,
WorkflowTask task = nullptr 
)

Add a file write start timestamp.

Parameters
filea workflow file
srcthe target location
servicethe target storage service
taskthe workflow task for which this write is done (or nullptr);

◆ addTimestampLinkUsage()

void wrench::SimulationOutput::addTimestampLinkUsage ( std::string  linkname,
double  bytes_per_second 
)

Add a link usage timestamp.

Parameters
linknamea linkname
bytes_per_secondlink usage in bytes_per_second

◆ addTimestampPstateSet()

void wrench::SimulationOutput::addTimestampPstateSet ( std::string  hostname,
int  pstate 
)

Add a pstate change/set timestamp.

Parameters
hostnamea hostname
pstatea pstate index

◆ addTimestampTaskCompletion()

void wrench::SimulationOutput::addTimestampTaskCompletion ( WorkflowTask task)

Add a task start completion.

Parameters
taska workflow task

◆ addTimestampTaskFailure()

void wrench::SimulationOutput::addTimestampTaskFailure ( WorkflowTask task)

Add a task start failure.

Parameters
taska workflow task

◆ addTimestampTaskStart()

void wrench::SimulationOutput::addTimestampTaskStart ( WorkflowTask task)

Add a task start timestamp.

Parameters
taska workflow task

◆ addTimestampTaskTermination()

void wrench::SimulationOutput::addTimestampTaskTermination ( WorkflowTask task)

Add a task start termination.

Parameters
taska workflow task

◆ dumpDiskOperationsJSON()

void wrench::SimulationOutput::dumpDiskOperationsJSON ( std::string  file_path,
bool  writing_file = true 
)

Writes a JSON file containing disk operation information as a JSON array.

>>>>>NOTE<<<<< The timestamps the JSON is generated from are disabled by default. Enable them with SimulationOutput::enableDiskTimestamps() to use.

The JSON array has the following format:

{
 "disk_operations": {
     "io_host": {                        <--- Hostname
         "/": {                          <--- Mount
            "reads": [
                 {
                  "bytes": <double>,
                  "end": <double>,
                  "failed": <double>,
                  "start": <double>
                 },
                 {
                    ...
                 }
                 ],
            "writes": [
                {
                 "bytes": <double>,
                 "end": <double>,
                 "failed": <double>,
                 "start": <double>
                 },
                 {
                  ...
                 }
                 ]
             }
         }
  }
}
Parameters
file_path- path to save JSON at
writing_file- boolean, default true, to write the JSON to the specified file path. Used for unified output.
Exceptions
invalid_argument

◆ dumpHostEnergyConsumptionJSON()

void wrench::SimulationOutput::dumpHostEnergyConsumptionJSON ( std::string  file_path,
bool  writing_file = true 
)

Writes a JSON file containing host energy consumption information as a JSON array.

The JSON array has the following format:

{
     "energy_consumption": {
     {
         hostname: <string>,
         pstates: [                 <-- if this host is a single core host, items in this list will be formatted as
             {                          the first item, else if this is a multi core host, items will be formatted as
                 pstate: <int>,         the second item
                 idle: <double>,
                 running: <double>,   <-- if single core host
                 speed: <double>
             },
             {
                 pstate: <int>,
                 idle: <double>,
                 one_core: <double>,  <-- if multi core host
                 all_cores: <double>, <-- if multi core host
                 speed: <double>
             } ...
         ],
         watts_off: <double>,
         pstate_trace: [
             {
                 time: <double>,
                 pstate: <int>
             }, ...
         ],
         consumed_energy_trace: [
             {
                 time: <double>,
                 joules: <double>
             }, ...
         ]
     }, ...
}
Parameters
file_paththe path to write the file
writing_filewhether or not the file is written, true by default but will be false when utilized as part of dumpUnifiedJSON
Exceptions
std::invalid_argument
std::runtime_error

◆ dumpLinkUsageJSON()

void wrench::SimulationOutput::dumpLinkUsageJSON ( std::string  file_path,
bool  writing_file = true 
)

Writes a JSON file containing link usage information as a JSON array.

This information will not be generated without using the bandwidth meter service and providing it with linknames to monitor.

 {
  "link_usage": {
      "links": [
                  {
                   "link_usage_trace": [
                          {
                           "bytes per second": <double>,
                           "time": <double>
                          },
                          {
                              ...
                          },
                      ],
                      "linkname": <string>
                  },
                  {
                      ...
                  }
              ]
   }
 }
 
Parameters
file_pathpath where json file is written
writing_filewhether to write file to disk. Enabled by default.
Exceptions
std::invalid_argument
std::runtime_error

◆ dumpPlatformGraphJSON()

void wrench::SimulationOutput::dumpPlatformGraphJSON ( std::string  file_path,
bool  writing_file = true 
)

Writes a JSON file containing all hosts, network links, and the routes between each host.

The JSON array has the following format:

{
    "platform":{
      vertices: [
           {
               type: <"host">,
               id: <string>,
             flop_rate: <double (flops per second)>,
             memory_manager_service: <double (bytes)>,
             cores: <unsigned_long>
          },
          {
               type: <"link">,
               id: <string>,
              bandwidth: <double (bytes per second)>,
            latency: <double (in seconds)>
           }, . . .
      ],
       edges: [
           {
               source: {
                   type: <string>,
                   id: <string>
             }
               target: {
                   type: <string>,
                   id: <string>
           }
           }, . . .
     ],
     routes: [
          {
              source: <string>,
              target: <string>,
              latency: <double (in seconds)>
              route: [
                 link_id, ...
             ]
       }
   ],
  }
}
Parameters
file_paththe path to write the file
writing_filewhether or not the file is written, true by default but will be false when utilized as part of dumpUnifiedJSON
Exceptions
std::invalid_argument

◆ dumpUnifiedJSON()

void wrench::SimulationOutput::dumpUnifiedJSON ( Workflow workflow,
std::string  file_path,
bool  include_platform = false,
bool  include_workflow_exec = true,
bool  include_workflow_graph = false,
bool  include_energy = false,
bool  generate_host_utilization_layout = false,
bool  include_disk = false,
bool  include_bandwidth = false 
)

Function that generates a unified JSON file containing the information specified by boolean arguments.

  JSON Structure:
  {
      "disk_operations": {
           ...
       },
       "energy_consumption": {
           ...
       },
       "link_usage": {
           ...
       },
       "platform": {
           ...
       },
       "workflow_execution": {
           ...
       },
       "workflow_graph": {
           ...
       }
  }
*

Any pieces not specified in the arguments are left out. For full structure see documentation of specific sections.

Parameters
workflowa pointer to the Workflow
file_pathpath for generated JSON
include_platformboolean whether to include platform in JSON
include_workflow_execboolean whether to include workflow execution in JSON
include_workflow_graphboolean whether to include workflow graph in JSON
include_energyboolean whether to include energy consumption in JSON
generate_host_utilization_layoutboolean specifying whether or not you would like a possible host utilization layout to be generated
include_diskboolean specifying whether to include disk operation in JSON (disk timestamps must be enabled)
include_bandwidthboolean specifying whether to include link bandwidth measurements in JSON

◆ dumpWorkflowExecutionJSON()

void wrench::SimulationOutput::dumpWorkflowExecutionJSON ( Workflow workflow,
std::string  file_path,
bool  generate_host_utilization_layout = false,
bool  writing_file = true 
)

Writes WorkflowTask execution history for each task to a file, formatted as a JSON array.

The JSON array has the following format:

    {
     "workflow_execution": {
        "tasks": [
            {
               "compute": {
                    "end": <double>,
                    "start": <double>
               },
               "execution_host": {
                    "cores": <unsigned_long>,
                    "flop_rate": <double>,
                    "hostname": <string>,
                    "memory_manager_service": <double>
               },
               "failed": <double>,
               "num_cores_allocated": <unsigned_long>,
               "read": [
                    {
                        "end": <double>,
                        "start": <double>
                    },
                    {
                        ...
                    }
               ],
               "task_id": <string>,
               "color": <string>,
               "terminated": <double>,
               "whole_task": {
                    "end": <double>,
                    "start": <double>
                },
                "write": [
                    {
                        "end": <double>,
                        "start": <double>
                    },
                    {
                        ...
                    }
                ],
            },
            {
                ...
            }
            ]
        }
    }

If generate_host_utilization_layout is set to true, a recursive function searches for a possible host utilization layout where tasks are assumed to use contiguous numbers of cores on their execution hosts. Note that each ComputeService does not enforce this, and such a layout may not exist for some workflow executions. In this situation, the function will go through the entire search space until all possible layouts are evaluated. For a large Workflow, this may take a very long time.

If a host utilization layout is able to be generated, the 'vertical_position' values will be set for each task run, and the task can be plotted as a rectangle on a graph where the y-axis denotes the number of cores - 1, and the x-axis denotes the workflow execution timeline. The vertical_position specifies the bottom of the rectangle. num_cores_allocated specifies the height of the rectangle.

Parameters
workflowa pointer to the Workflow
file_paththe path to write the file
generate_host_utilization_layoutboolean specifying whether or not you would like a possible host utilization layout to be generated
writing_filewhether or not the file is written, true by default but will be false when utilized as part of dumpUnifiedJSON
Exceptions
std::invalid_argument

◆ dumpWorkflowGraphJSON()

void wrench::SimulationOutput::dumpWorkflowGraphJSON ( wrench::Workflow workflow,
std::string  file_path,
bool  writing_file = true 
)

Writes a JSON graph representation of the Workflow to a file.

A node is added for each WorkflowTask and WorkflowFile. A WorkflowTask will have the type "task" and a WorkflowFile will have the type "file". A directed link is added for each dependency in the Workflow.

{
     "workflow_graph": {
         vertices: [
             {
                 type: <"task">,
                 id: <string>,
                 flops: <double>,
                 min_cores: <unsigned_long>,
                 max_cores: <unsigned_long>,
                 memory_manager_service: <double>,
             },
             {
                 type: <"file">,
                 id: <string>,
                 size: <double>
             }, . . .
         ],
         edges: [
             {
                 source: <string>,
                 target: <string>
             }, . . .
         ]
     }
 }
 
Parameters
workflowa pointer to the workflow
file_paththe path to write the file
writing_filewhether or not the file is written, true by default but will be false when utilized as part of dumpUnifiedJSON
Exceptions
std::invalid_argument

◆ enableBandwidthTimestamps()

void wrench::SimulationOutput::enableBandwidthTimestamps ( bool  enabled)

Enable or Disable the insertion of link-usage-related timestamps in the simulation output (enabled by default)

Parameters
enabledtrue to enable, false to disable

◆ enableDiskTimestamps()

void wrench::SimulationOutput::enableDiskTimestamps ( bool  enabled)

Enable or Disable the insertion of disk-related timestamps in the simulation output (enabled by default)

Parameters
enabledtrue to enable, false to disable

◆ enableEnergyTimestamps()

void wrench::SimulationOutput::enableEnergyTimestamps ( bool  enabled)

Enable or Disable the insertion of energy-related timestamps in the simulation output (enabled by default)

Parameters
enabledtrue to enable, false to disable

◆ enableFileReadWriteCopyTimestamps()

void wrench::SimulationOutput::enableFileReadWriteCopyTimestamps ( bool  enabled)

Enable or Disable the insertion of file-related timestamps in the simulation output (enabled by default)

Parameters
enabledtrue to enable, false to disable

◆ enableWorkflowTaskTimestamps()

void wrench::SimulationOutput::enableWorkflowTaskTimestamps ( bool  enabled)

Enable or Disable the insertion of task-related timestamps in the simulation output (enabled by default)

Parameters
enabledtrue to enable, false to disable

◆ getTrace()

template<class T >
std::vector<SimulationTimestamp<T> *> wrench::SimulationOutput::getTrace ( )
inline

Retrieve a copy of a simulation output trace once the simulation has completed.

Template Parameters
aparticular SimulationTimestampXXXX class (defined in SimulationTimestampTypes.h)
Returns
a vector of pointers to SimulationTimestampXXXX instances

The documentation for this class was generated from the following files: