Logger

class Kilosim::Logger

A Logger is used to save HDF5 files containing parameters and continuous state information for multiple simulation trials.

State information is logged through aggregator functions, which reduce the state of the robots to a vector. This could be a single average value over all the robots (e.g., mean observed ambient light) all the way to saving a value for every robot (e.g., each robot’s ambient light value as an element). Each aggregator is saved as an array, where each row is the output of the aggregator function. Whenever the state is logged, the simulator time is also saved as a timeseries.

The resulting file structure would look as follows:

logFile.h5
|__ trial_1  (group)
|   |__ time (dataset)  [1 x t]
|   |__ params (group)
|   |   |__ param1 (dataset)
|   |   |__ param2 (dataset)
|   |   |__ ...
|   |__ aggregator_1 (dataset)  [n x t]
|   |__ aggregator_2 (dataset)  [m x t]
|   |__ ...
|__ trial_2
|   |__ time (dataset)  [1 x t]
|   |__ params
|   |   |__ ...
|   |__ ...
...

where t is the number of time steps when data was logged, and aggregator_1 and aggregator_2 were specified by the user.

Logging (I/O in general) is one of the slowest parts of the simulation. As such, a high logging rate will significantly slow down your simulations. Try something like logging every 5-10 seconds, if you can get away with it.

If you delete an HDF5 file, it does NOT actually delete the data; it just removes the reference to it. Therefore, rather than extensive use of the overwrite flag, if you plan to completely redo a set of simulations, delete the file and let it regenerate. (However, there are tools for removing this pseudo-deleted data later.)

Note

The Logger does not provide functionality for reading/viewing log files once created. (It’s kind of a pain in C++. I recommend using h5py instead.)

Public Types

typedef std::vector<double> (*aggregatorFunc)(std::vector<Robot*> &robots)

A function mapping Robots to values This is used by log_state to compute values to save from all of the robots at a time step. The output will form a row in an H5::PacketTable. It may be one value per robot (e.g., the current motor command) or all the way to one combined value from all the robots (e.g., the mean light perceived).

Functions with this signature are the inputs to add_aggregator

Public Functions

Logger(World &world, const std::string file_id, const int trial_num, const bool overwrite_trials = false)

Construct a Logger that logs within the given HDF5 file and group trial_num

If the HDF5 file does not exist, it will be created.

Warning

You must create any directories in the filepath to your file_id before attempting to create a file with this constructor. If you attempt to create a log file in a location (directory) that does not exist, your program will terminate with an H5::FileIException.

Parameters

  • world – The world from which state and parameters will be logged

  • file_id – Name and location of the HDF5 file to log stuff in

  • trial_num – Number of the trial to save the data. Data will be saved in a group named “trial_#”, where # is trial_num.

  • overwrite_trials – Whether to overwrite data if the trial is already in the log file. If set to false, the program will exit if the trial already exists.

~Logger()

Destructor: closes the file when it goes out of scope.

void set_trial(uint const trial_num)

Set which trial is being logged.

This allows you to create a single logger and use it for many trials, without having to re-initialize and re-add your aggregators. You DO have to manually re-log parameters for each trial, if you want to do that.

The same overwrite_trials flag from initialization is still in effect.

Parameters

trial_num – New trial you want to log.

uint get_trial() const

Get/check which trial the Logger is currently set to log data/parameters for.

Returns

Current trial. Change with set_trial()

void add_aggregator(std::string const agg_name, aggregatorFunc const agg_func)

Add an aggregator function that will be run when log_state is called.

(When log_state is called, all aggregator functions will be called on the robots in the World.)

The output is saved as a row in a dataset named by agg_name

Parameters

  • agg_name – Name of the dataset in with to store the output of the agg_func. This exists within the trial_# group.

  • agg_func – Aggregator that saves values from the Robots in the World. Each output is saved as a row in the dataset.

void log_state() const

Log the aggregators at the given time mapped over all the given robots in the World. Every time this is called, the current time (in seconds) is added to the time series, and a row is appended to every aggregator array.

void log_config(ConfigParser &config, const bool show_warnings = true)

Log all of the values in the configuration as params in the HDF5 file/trial

Note

This only supports atomic datatypes (bool, int, uint, float, double, string). Any non-atomic data (arrays and objects) will be skipped (with a warning in the terminal).

Parameters

  • config – Loaded configuration for this experiment/trial

  • show_warnings – Whether or not to print out warnings when there are non-atomic datatypes in the config that cannot be saved

void log_param(const std::string name, const json val)

Log a single parameter name and value

To log the contents loaded from a configuration file, use log_config; this is for logging additional/computed parameters that are not part of a ConfigParser.

Note

This only supports atomic datatypes (bool, int, uint, float, double, string). Any non-atomic data (arrays and objects) will be skipped (with a warning in the terminal).

Parameters

  • name – Key/variable name to save the data under in the log file

  • val – Atomic data value to save as a parameter

void log_vector(const std::string name, const std::vector<double> val_vec)

Save a vector of values to the HDF5 file.