HDF5 Utilities

The logging framework uses the HDF5 standard to store log information. You can find more documentation on HDF5 at http://www.h5py.org/

HDF5 Container Class

class HDF5LogContainer(file_handle, name=None, compression=None)

Class to define an HDF5 log container.

Parameters:

• file_handle (h5py.File()) – Handle of the HDF5 file
• name (str, optional) – Name of the HDF5 group of the log container
• compression (bool, optional) – HDF5 compression setting on the log container

When an HDF5LogContainer is created, the underlying HDF5 file will not be modified unless one of the write_* methods are called.

is_valid()

Check that the HDF5 Log Container is valid.

Returns:

• True –> This is a valid HDF5 log file
• False –> This is NOT a valid HDF5 log file

Return type:is_valid (bool)

write_log_data(log_data, append=True)

Write the log data to the log container.

Parameters:

• log_data (bytes) – Binary data from a WlanExpNode log
• append (bool, optional) – Append to (True) or Overwrite (False) the current log data

write_log_index(log_index=None)

Write the log index to the log container.

Parameters:log_index (dict) – Log index generated from wlan_exp log data

If the log index currently exists in the HDF5 file, that log index will be replaced with this new log index. If log_index is provided then that log index will be written to the log container. Otherwise, a raw log index will be generated and added to the log container.

write_attr_dict(attr_dict)

Add the given attribute dictionary to the opened log container.

Parameters:attr_dict (dict) – A dictionary of user provided attributes that will be added to the HDF5 group.

get_log_data_size()

Get the current size of the log data in the log container.

Returns:Number of bytes of log data in the log container Return type:size (int)

get_log_data()

Get the log data from the log container.

Returns:Bytes object of the log data in the container Return type:log_data (bytes)

get_log_index(gen_index=True)

Get the raw log index from the log container.

Parameters:gen_index (bool, optional) – Generate the raw log index if the log index does not exist in the log container. Returns:Log index from the log container Return type:log_index (dict)

get_attr_dict()

Get the attribute dictionary from the log container.

Returns:The dictionary of user provided attributes in the log container. Return type:attr_dict (dict)

HDF5 Utility Functions

These utility functions wrap the interactions with the HDF5 Container class.

hdf5_open_file(filename, readonly=False, append=False, print_warnings=True)

Open an HDF5 file.

Parameters:

• filename (str) – Filename of the HDF5 file to open
• readonly (bool, optional) – Open the file in read-only mode
• append (bool, optional) – Append to the data in the current file
• print_warnings (bool, optional) – Print warning messages

Returns:

Handle for the HDF5 file

Return type:

file_handle (h5py.File)

Behavior of input attributes:

readonly	append	Behavior
True	T/F	File opened in read-only mode
False	True	File opened in append mode; created if it does not exist
False	False	If file with filename exists, then a new filename is generated using the log utilities. The new file is then created by the h5py File method (DEFAULT)

hdf5_close_file(file_handle)

Close an HDF5 file.

Parameters:file_handle (h5py.File) – Handle for the HDF5 file

log_data_to_hdf5(log_data, filename, attr_dict=None, gen_index=True, overwrite=False)

Create an HDF5 file that contains the log_data, a raw_log_index, and any user attributes.

Parameters:

• log_data (bytes) – Binary data from a WlanExpNode log
• filename (str) – Filename of the HDF5 file to appear on disk
• attr_dict (dict, optional) – A dictionary of user provided attributes that will be added to the HDF5 group.
• gen_index (bool, optional) – Generate the raw_log_index from the log_data and store it in the file.
• overwrite (bool, optional) – If True method will overwrite existing file with filename

If the requested filename already exists and overwrite==True this method will replace the existing file, destroying any data in the original file.

If the filename already exists and overwrite==False this method will print a warning, then create a new filename with a unique date-time suffix.

hdf5_to_log_data(filename, group_name=None)

Extract the log_data from an HDF5 Log Container

Parameters:

• filename (str) – Name of HDF5 file to open
• group_name (str, optional) – Name of Group within the HDF5 file object (defaults to “”)

Returns:

Log data in the HDF5 file

Return type:

log_data (bytes)

hdf5_to_log_index(filename, group_name=None, gen_index=True)

Extract the log_index from an HDF5 Log Container

Parameters:

• filename (str) – Name of HDF5 file to open
• group_name (str, optional) – Name of Group within the HDF5 file object (defaults to “”)
• gen_index (bool, optional) – Generate the raw_log_index from the log_data and store it in the file if the log_index is not in the file.

Returns:

Either the log_index from HDF5 file or a generated raw_log_index from log_data in HDF5 file

Return type:

log_index (dict)

hdf5_to_attr_dict(filename=None, group_name=None)

Extract the attribute dictionary from an HDF5 Log Container.

Parameters:

• filename (str) – Name of HDF5 file to open
• group_name (str, optional) – Name of Group within the HDF5 file object (defaults to “”)

Returns:

The dictionary of user provided attributes in the HDF5 file

Return type:

attr_dict (dict)

np_arrays_to_hdf5(filename, np_log_dict, attr_dict=None, compression=None)

Generate an HDF5 file from numpy arrays.

Parameters:

• filename (str) – Name of HDF5 file to open
• np_log_dict (Numpy Array) – Numpy array to add to the HDF5 file
• attr_dict (dict, optional) – A dictionary of user provided attributes that will be added to the HDF5 file.
• compression (bool, optional) – HDF5 compression setting on the log container

The np_log_dict input must be either:

1. A dictionary with numpy record arrays as values; each array will be a dataset in the HDF5 file root group
2. A dictionary of dictionaries like (1); each top-level value will be a group in the root HDF5 group, each numpy array will be a dataset in the group.

attr_dict is optional. If provied, values in attr_dict will be copied to HDF5 group and dataset attributes. attr_dict values with keys matching np_log_dict keys will be used as dataset attributes named '<the_key>_INFO'. Attribute dictionary entries may have an extra value with key '/', which will be used as the value for a group attribute named 'INFO'.

Examples:

# No groups - all datasets in root group
np_log_dict = {
    'RX_OFDM':  np_array_of_rx_etries,
    'TX_HIGH':  np_array_of_tx_entries
}

attr_dict = {
    '/':        'Data from some_log_file.bin, node serial number W3-a-00001, written on 2014-03-18',
    'RX_OFDM':  'Filtered Rx OFDM events, only good FCS receptions',
    'TX_HIGH':  'Filtered Tx events, only DATA packets'
}

# Two groups, with two datasets in each group
np_log_dict = {
    'Log_Node_A': {
        'RX_OFDM':  np_array_of_rx_etries_A,
        'TX_HIGH':  np_array_of_tx_entries_A
    },
    'Log_Node_B': {
        'RX_OFDM':  np_array_of_rx_etries_B,
        'TX_HIGH':  np_array_of_tx_entries_B
    }
}

attr_dict = {
    '/':        'Written on 2014-03-18',
    'Log_Node_A': {
        '/':        'Data from node_A_log_file.bin, node serial number W3-a-00001',
        'RX_OFDM':  'Filtered Rx OFDM events, only good FCS receptions',
        'TX_HIGH':  'Filtered Tx events, only DATA packets'
    }
    'Log_Node_B': {
        '/':        'Data from node_B_log_file.bin, node serial number W3-a-00002',
        'RX_OFDM':  'Filtered Rx OFDM events, only good FCS receptions',
        'TX_HIGH':  'Filtered Tx events, only DATA packets'
    }
}