A class that contains post-mortem simulation-generated data. More...

#include <SimulationOutput.h>

Public Member Functions
void	dumpDiskOperationsJSON (std::string file_path, bool writing_file=true)
	Writes a JSON file containing disk operation information as a JSON array. More...

void	dumpHostEnergyConsumptionJSON (std::string file_path, bool writing_file=true)
	Writes a JSON file containing host energy consumption information as a JSON array. More...

void	dumpLinkUsageJSON (std::string file_path, bool writing_file=true)

void	dumpPlatformGraphJSON (std::string file_path, bool writing_file=true)
	Writes a JSON file containing all hosts, network links, and the routes between each host. More...

void	dumpUnifiedJSON (std::shared_ptr< Workflow > workflow, std::string file_path, bool include_platform=false, bool include_workflow_exec=true, bool include_workflow_graph=false, bool include_energy=false, bool generate_host_utilization_layout=false, bool include_disk=false, bool include_bandwidth=false)
	Function that generates a unified JSON file containing the information specified by boolean arguments. More...

void	dumpWorkflowExecutionJSON (std::shared_ptr< Workflow > workflow, std::string file_path, bool generate_host_utilization_layout=false, bool writing_file=true)
	Writes WorkflowTask execution history for each task to a file, formatted as a JSON array. More...

void	dumpWorkflowGraphJSON (std::shared_ptr< Workflow > workflow, std::string file_path, bool writing_file=true)
	Writes a JSON graph representation of the Workflow to a file. More...

void	enableBandwidthTimestamps (bool enabled)
	Enable or Disable the insertion of link-usage-related timestamps in the simulation output (enabled by default) More...

void	enableDiskTimestamps (bool enabled)
	Enable or Disable the insertion of disk-related timestamps in the simulation output (enabled by default) More...

void	enableEnergyTimestamps (bool enabled)
	Enable or Disable the insertion of energy-related timestamps in the simulation output (enabled by default) More...

void	enableFileReadWriteCopyTimestamps (bool enabled)
	Enable or Disable the insertion of file-related timestamps in the simulation output (enabled by default) More...

void	enableWorkflowTaskTimestamps (bool enabled)
	Enable or Disable the insertion of task-related timestamps in the simulation output (enabled by default) More...

template<class T >
std::vector< SimulationTimestamp< T > * >	getTrace ()
	Retrieve a copy of a simulation output trace once the simulation has completed. More...

Detailed Description

A class that contains post-mortem simulation-generated data.

Member Function Documentation

◆ dumpDiskOperationsJSON()

void wrench::SimulationOutput::dumpDiskOperationsJSON	(	std::string	file_path,
		bool	writing_file = `true`
	)

Writes a JSON file containing disk operation information as a JSON array.

>>>>>NOTE<<<<< The timestamps the JSON is generated from are disabled by default. Enable them with SimulationOutput::enableDiskTimestamps() to use.

The JSON array has the following format:

{
 "disk_operations": {
     "io_host": {                        <--- Hostname
         "/": {                          <--- Mount
            "reads": [
                 {
                  "bytes": <double>,
                  "end": <double>,
                  "failed": <double>,
                  "start": <double>
                 },
                 {
                    ...
                 }
                 ],
            "writes": [
                {
                 "bytes": <double>,
                 "end": <double>,
                 "failed": <double>,
                 "start": <double>
                 },
                 {
                  ...
                 }
                 ]
             }
         }
  }
}

Parameters

file_path	- path to save JSON at
writing_file	- boolean, default true, to write the JSON to the specified file path. Used for unified output.

Exceptions

invalid_argument

◆ dumpHostEnergyConsumptionJSON()

void wrench::SimulationOutput::dumpHostEnergyConsumptionJSON	(	std::string	file_path,
		bool	writing_file = `true`
	)

Writes a JSON file containing host energy consumption information as a JSON array.

The JSON array has the following format:

{
     "energy_consumption": {
     {
         hostname: <string>,
         pstates: [
             {
                 pstate: <int>,
                 idle: <double>,
                 epsilon: <double>,
                 all_cores: <double>,
                 speed: <double>
             },
             {
                 pstate: <int>,
                 idle: <double>,
                 epsilon: <double>,
                 all_cores: <double>,
                 speed: <double>
             } ...
         ],
         watts_off: <double>,
         pstate_trace: [
             {
                 time: <double>,
                 pstate: <int>
             }, ...
         ],
         consumed_energy_trace: [
             {
                 time: <double>,
                 joules: <double>
             }, ...
         ]
     }, ...
}

Parameters

file_path	the path to write the file
writing_file	whether or not the file is written, true by default but will be false when utilized as part of dumpUnifiedJSON

Exceptions

std::invalid_argument
std::runtime_error

◆ dumpLinkUsageJSON()

void wrench::SimulationOutput::dumpLinkUsageJSON	(	std::string	file_path,
		bool	writing_file = `true`
	)

Writes a JSON file containing link usage information as a JSON array.

This information will not be generated without using the bandwidth meter service and providing it with linknames to monitor.

 {
  "link_usage": {
      "links": [
                  {
                   "link_usage_trace": [
                          {
                           "bytes per second": <double>,
                           "time": <double>
                          },
                          {
                              ...
                          },
                      ],
                      "linkname": <string>
                  },
                  {
                      ...
                  }
              ]
   }
 }

Parameters

file_path	path where json file is written
writing_file	whether to write file to disk. Enabled by default.

Exceptions

std::invalid_argument
std::runtime_error

◆ dumpPlatformGraphJSON()

void wrench::SimulationOutput::dumpPlatformGraphJSON	(	std::string	file_path,
		bool	writing_file = `true`
	)

Writes a JSON file containing all hosts, network links, and the routes between each host.

The JSON array has the following format:

{
    "platform":{
      vertices: [
           {
               type: <"host">,
               id: <string>,
             flop_rate: <double (flops per second)>,
             memory_manager_service: <double (bytes)>,
             cores: <unsigned_long>
          },
          {
               type: <"link">,
               id: <string>,
              bandwidth: <double (bytes per second)>,
            latency: <double (in seconds)>
           }, . . .
      ],
       edges: [
           {
               source: {
                   type: <string>,
                   id: <string>
             }
               target: {
                   type: <string>,
                   id: <string>
           }
           }, . . .
     ],
     routes: [
          {
              source: <string>,
              target: <string>,
              latency: <double (in seconds)>
              route: [
                 link_id, ...
             ]
       }
   ],
  }
}

Parameters

file_path	the path to write the file
writing_file	whether or not the file is written, true by default but will be false when utilized as part of dumpUnifiedJSON

Exceptions

std::invalid_argument

◆ dumpUnifiedJSON()

void wrench::SimulationOutput::dumpUnifiedJSON	(	std::shared_ptr< Workflow >	workflow,
		std::string	file_path,
		bool	include_platform = `false`,
		bool	include_workflow_exec = `true`,
		bool	include_workflow_graph = `false`,
		bool	include_energy = `false`,
		bool	generate_host_utilization_layout = `false`,
		bool	include_disk = `false`,
		bool	include_bandwidth = `false`
	)

Function that generates a unified JSON file containing the information specified by boolean arguments.

  JSON Structure:
  {
      "disk_operations": {
           ...
       },
       "energy_consumption": {
           ...
       },
       "link_usage": {
           ...
       },
       "platform": {
           ...
       },
       "workflow_execution": {
           ...
       },
       "workflow_graph": {
           ...
       }
  }
*

Any pieces not specified in the arguments are left out. For full structure see documentation of specific sections.

Parameters

workflow	a pointer to the Workflow
file_path	path for generated JSON
include_platform	boolean whether to include platform in JSON
include_workflow_exec	boolean whether to include workflow execution in JSON
include_workflow_graph	boolean whether to include workflow graph in JSON
include_energy	boolean whether to include energy consumption in JSON
generate_host_utilization_layout	boolean specifying whether or not you would like a possible host utilization layout to be generated
include_disk	boolean specifying whether to include disk operation in JSON (disk timestamps must be enabled)
include_bandwidth	boolean specifying whether to include link bandwidth measurements in JSON

◆ dumpWorkflowExecutionJSON()

void wrench::SimulationOutput::dumpWorkflowExecutionJSON	(	std::shared_ptr< Workflow >	workflow,
		std::string	file_path,
		bool	generate_host_utilization_layout = `false`,
		bool	writing_file = `true`
	)

Writes WorkflowTask execution history for each task to a file, formatted as a JSON array.

The JSON array has the following format:

    {
     "workflow_execution": {
        "tasks": [
            {
               "compute": {
                    "end": <double>,
                    "start": <double>
               },
               "execution_host": {
                    "cores": <unsigned_long>,
                    "flop_rate": <double>,
                    "hostname": <string>,
                    "memory_manager_service": <double>
               },
               "failed": <double>,
               "num_cores_allocated": <unsigned_long>,
               "read": [
                    {
                        "end": <double>,
                        "start": <double>
                    },
                    {
                        ...
                    }
               ],
               "task_id": <string>,
               "color": <string>,
               "terminated": <double>,
               "whole_task": {
                    "end": <double>,
                    "start": <double>
                },
                "write": [
                    {
                        "end": <double>,
                        "start": <double>
                    },
                    {
                        ...
                    }
                ],
            },
            {
                ...
            }
            ]
        }
    }

If generate_host_utilization_layout is set to true, a recursive function searches for a possible host utilization layout where tasks are assumed to use contiguous numbers of cores on their execution hosts. Note that each ComputeService does not enforce this, and such a layout may not exist for some workflow executions. In this situation, the function will go through the entire search space until all possible layouts are evaluated. For a large Workflow, this may take a very long time.

If a host utilization layout is able to be generated, the 'vertical_position' values will be set for each task run, and the task can be plotted as a rectangle on a graph where the y-axis denotes the number of cores - 1, and the x-axis denotes the workflow execution timeline. The vertical_position specifies the bottom of the rectangle. num_cores_allocated specifies the height of the rectangle.

Parameters

workflow	a pointer to the Workflow
file_path	the path to write the file
generate_host_utilization_layout	boolean specifying whether or not you would like a possible host utilization layout to be generated
writing_file	whether or not the file is written, true by default but will be false when utilized as part of dumpUnifiedJSON

Exceptions

std::invalid_argument

◆ dumpWorkflowGraphJSON()

void wrench::SimulationOutput::dumpWorkflowGraphJSON	(	std::shared_ptr< Workflow >	workflow,
		std::string	file_path,
		bool	writing_file = `true`
	)

Writes a JSON graph representation of the Workflow to a file.

A node is added for each WorkflowTask and DataFile. A WorkflowTask will have the type "task" and a DataFile will have the type "file". A directed link is added for each dependency in the Workflow.

{
     "workflow_graph": {
         vertices: [
             {
                 type: <"task">,
                 id: <string>,
                 flops: <double>,
                 min_cores: <unsigned_long>,
                 max_cores: <unsigned_long>,
                 memory_manager_service: <double>,
             },
             {
                 type: <"file">,
                 id: <string>,
                 size: <double>
             }, . . .
         ],
         edges: [
             {
                 source: <string>,
                 target: <string>
             }, . . .
         ]
     }
 }

Parameters

workflow	a pointer to the workflow
file_path	the path to write the file
writing_file	whether or not the file is written, true by default but will be false when utilized as part of dumpUnifiedJSON

Exceptions

std::invalid_argument

◆ enableBandwidthTimestamps()

void wrench::SimulationOutput::enableBandwidthTimestamps ( bool enabled )

Enable or Disable the insertion of link-usage-related timestamps in the simulation output (enabled by default)

Parameters