Event Log Utilities

Common utilities for the WLAN Experiment logging framework.

Log Utility Functions

gen_raw_log_index(log_data)[source]

Parses binary wlan_exp log data by recording the byte index of each entry.

Parameters:log_data (bytes) – Binary data from a WlanExpNode log
Returns:Dictionary that corresponds 1-to-1 with what is in the given log_data of the form: { <int> : [<offsets>] }
Return type:raw_log_index (dict)

The byte indexes are returned in a dictionary with the entry type IDs as keys. This method does not unpack or interpret each log entry and does not change any values in the log file itself (the log_data array argument can be read-only).

Format of log entry header:

typedef struct{
    u32 delimiter;
    u16 entry_type;
    u16 entry_length;
} entry_header;

fmt_log_hdr = 'I H H' # Using struct.unpack
filter_log_index(log_index, include_only=None, exclude=None, merge=None, verbose=False)[source]

Parses a log index to generate a filtered log index.

Parameters:
  • log_index (dict) – Log index dictionary (can be either a ‘raw_log_index’ or a previously processed ‘log_index’)
  • include_only (list of WlanExpLogEntryType, optional) – All WlanExpLogEntryType to include in the output log index. This takes precedence over ‘exclude’.
  • exclude (list of WlanExpLogEntryType, optional) – All WlanExpLogEntryType to exclude in the output log index. This will not be used if include != None.
  • merge (dict, optional) – Dictionary of the form: {'WlanExpLogEntryType name': [List of 'WlanExpLogEntryTypes name' to merge]}
Returns:

Filtered log index dictionary based on the given parameters
Return type:

log_index (dict)

Consumers, in general, cannot operate on a raw log index since that has not been converted in to log entry types. The besides filtering a log index, this method will also convert any raw index entries (ie entries with keys of type int) in to the corresponding WlanExpLogEntryTypes.

Using the ‘merge’ argument can combine the indexes of WlanExpLogEntryTypes to create super-sets of entries. For example, to create a log index that contains all the receive events use: {'RX_ALL': ['RX_OFDM', 'RX_DSSS']} as long as the names ‘RX_ALL’, ‘RX_OFDM’, and ‘RX_DSSS’ are valid WlanExpLogEntryTypes.

The filter follows the following basic rules:
  1. Every requested output (either through ‘include_only’ or ‘merge’) has a key in the output dictionary
  2. All input and output keys must refer to the ‘name’ property of valid WlanExpLogEntryType instances

Examples:

Assume:
  • ‘A’, ‘B’, ‘C’, ‘D’, ‘M’ are valid WlanExpLogEntryType instance names
  • The log_index = {‘A’: [A0, A1, A2], ‘B’: [B0, B1], ‘C’: []}
  • include_only:

    All names specified in ‘include_only’ are included as part of the output dictionary. It is then up to the consumer to check if the number of entries for a given ‘name’ is zero (ie the list is empty).

    x = filter_log_index(log_index, include_only=['A'])
x == {'A': [A0, A1, A2]}

x = filter_log_index(log_index, include_only=['A',B'])
x == {'A': [A0, A1, A2], 'B': [B0, B1]}

x = filter_log_index(log_index, include_only=['C'])
x == {'C': []]}

x = filter_log_index(log_index, include_only=['D'])
x == {'D': []]}
  • exclude:

    All names specified in ‘exclude’ are removed from the output dictionary. However, there is no guarentee what other WlanExpLogEntryTypes are in the output dictionary. That depends on the entries in the input log index.

    x = filter_log_index(log_index, exclude=['B'])
x == {'A': [A0, A1, A2]}, 'C': []}

x = filter_log_index(log_index, exclude=['D'])
WARNING:  D does not exist in log index.  Ignoring for exclude.
x == {'A': [A0, A1, A2]}, 'B': [B0, B1], 'C': []}
  • merge:

    All names specified in the ‘merge’ are included as part of the output dictionary. It is then up to the consumer to check if the number of entries for a given ‘name’ is zero (ie the list is empty).

    x = filter_log_index(log_index, merge={'D': ['A', 'B']}
x == {'A': [A0, A1, A2],
      'B': [B0, B1],
      'C': [],
      'D': [A0, A1, A2, B0, B1]}

x = filter_log_index(log_index, merge={'M': ['C', 'D']}
x == {'A': [A0,A1,A2]},
      'B': [B0,B1],
      'C': [],
      'M': []}
  • Combined:

    Combining the behavior of ‘include_only’, ‘exclude’, and ‘merge’

    x = filter_log_index(log_index, include_only=['M'], merge={'M': ['A','C']}
x == {'M': [A0, A1, A2]}

x = filter_log_index(log_index, include_only=['M'], merge={'M': ['A','D']}
WARNING:  D does not exist in log index.  Ignoring for merge.
x == {'M': [A0, A1, A2]}

x = filter_log_index(log_index, include_only=['M'], merge={'M': ['C','D']}
WARNING:  D does not exist in log index.  Ignoring for merge.
x == {'M': []}
log_data_to_np_arrays(log_data, log_index)[source]

Generate numpy structured arrays using log_data and a log_index.

Parameters:
  • log_data (bytes) – Binary data from a WlanExpNode log
  • log_index (dict) – Log index dictionary
Returns:

Numpy structured arrays corresponding to the log_data and log_index
Return type:

np_array (Numpy Array)

Misc Utility Functions

print_log_index_summary(log_index, title=None)[source]

Prints a summary of the log_index.

Parameters:
  • log_index (dict) – Log index dictionary
  • title (str, optional) – Title to be used for the log_index (default is ‘Log Index Summary:’)
calc_tx_time(mcs, phy_mode, payload_length, phy_samp_rate)[source]

Calculates the duration of an 802.11 transmission given its rate and payload length. Returns duration of PHY transmission in microseconds.

Parameters:
  • mcs (int or list of ints) – Modulation and coding scheme (MCS) index
  • phy_mode (str, int or list of strs or ints) – PHY mode (from util.phy_modes)
  • payload_length (int or list of ints) – Nnumber of bytes in the payload
  • phy_sample_rate (int or list of ints) – PHY sample rate; only (10, 20, 40) are valid

This method accounts only for PHY overhead (preamble, SIGNAL field, etc.). It does not account for MAC overhead. The payload_length argument must include any MAC fields (typically a 24-byte MAC header plus 4 byte FCS).

All 4 arguments are required. The dimensions of the 4 arguments must match. To calculate the duration of a single packet, call this method with scalaer integer arguments. To calculate the duration of many packets, call this method with iterables (typically Numpy arrays) of integer values. When calling this method with arrays the lengths of the 4 arrays must be equal.

find_overlapping_tx_low(src_tx_low, int_tx_low)[source]

Finds TX_LOW entries in the source that are overlapped by the TX_LOW entries in other flow.

Parameters:
  • src_tx_low (Numpy Array) – Source TX_LOW numpy array of entries
  • int_tx_low (Numpy Array) – Other TX_LOW numpy array of entries
  • phy_sample_rate (int) – Sample rate of the PHY
Returns:

Tuple containing indexes into the provided arrays indicating which entries overlapped
Return type:

indexes (tuple)
convert_datetime_to_log_time_str(datetime_obj)[source]

Convert a datetime object to a log time string.

Parameters:datetime_obj (DateTime()) – Python DateTime() object
Returns:String format of the DateTime() object to be used in HDF5 files
Return type:log_time_str (str)
convert_log_time_str_to_datetime(log_time_str)[source]

Convert a log time string to a datetime object.

Parameters:log_time_str (str) – String format of the DateTime() object to be used in HDF5 files
Returns:Python DateTime() object
Return type:datetime_obj (DateTime())
get_now_as_log_time_str()[source]

Get the current time as a log time string.

This should be used instead of datetime.datetime.now() because it automatically handles timezones.

Returns:String format of the datetime.datetime.now() to be used in HDF5 files
Return type:log_time_str (str)
merge_log_indexes(dest_index, src_index, offset)[source]

Merge log indexes.

Both the dest_index and src_index have log entry offsets that are relative to the beginning of the log data from which they were generated. If the log data used to generate the log indexes are being merged, then move the log entry offsets in the src_index to their absolute offset in the merged log index. For each of the log entry offsets in the src_index, the following translation will occur:

<Offset in merged log index> = <Offset in src_index> + offset
Parameters:
  • dest_index (dict) – Destination log index to merge src_index into
  • src_index (dict) – Source log index to merge into destination log index
  • offset (int) – Offset of src_index into dest_index
calc_next_entry_offset(log_data, raw_log_index)[source]

Calculates the offset of the next log entry given the log data and the raw log index.

The log data does not necessarily end on a log entry boundary. Therefore, it is necessary to be able to calculate the offset of the next log entry so that it is possible to continue index generation when reading the log in multiple pieces.

Parameters:
  • log_data (bytes) – Binary data from a WlanExpNode log
  • log_index (dict) – Raw log index dictionary
Returns:

Offset of next log entry
Return type:

offset (int)
overwrite_entries_with_null_entry(log_data, byte_offsets)[source]

Overwrite the entries in byte_offsets with NULL entries.

This is an in-place modification of log_data.

Parameters:
  • log_data (bytes) – Binary data from a WlanExpNode log
  • byte_offsets (list of int) – List of offsets corresponding to the entries to overwrite
overwrite_payloads(log_data, byte_offsets, payload_offsets=None)[source]

Overwrite any payloads with zeros.

Parameters:
  • log_data (bytes) – Binary data from a WlanExpNode log
  • byte_offsets (list of int) – List of offsets corresponding to the entries to be modified
  • payload_offsets (dict) – Dictionary of { entry_type_id : <payload offset> }

By default, if payload_offsets is not specified, the method will iterate through all the entry types and calculate the defined size of the entry (ie it will use calcsize on the struct format of the entry). Sometimes, this is not the desired behavior and calling code would want to specify a different amount of the payload to keep. For example, for data transmissions / receptions, it might be desired to also keep the SNAP headers and potentially the IP headers. In this case, the calling code would get the appropriate set of byte_offsets and then create a payload_offsets dictionary with the desired “size” of the entry for those byte_offsets. This will result in the calling code potentially calling this function multiple times with different payload_offsets for a given entry_type_id.

This method relies on the fact that for variable length log entries, the variable length data, ie the payload, is always at the end of the entry. The code also knows, based on the entry type, the size of the entry without the payload. Therefore, from the entry header, the code can determine how many payload bytes are after the defined fields and zero them out.

This is an in-place modification of log_data.