Python module to reading in data from both 12-bit and 4-bit TBW files. This module defines the following classes for storing the TBW data found in a file:
object that contains all data associated with a particular TBW frame. The primary consituents of each frame are:
- FrameHeader - the TBW frame header object and
- FrameData - the TBW frame data object.
Combined, these two objects contain all of the information found in the original TBW frame.
For reading in data, use the readFrame function. It takes a python file- handle as an input and returns a fully-filled Frame object. readFrame is designed to work with both 4-bit and 12-bit observations.
For describing the format of data in the file, two function are provided:
read in the first several frames to see how many stands are found in the data. .. note:
This function is a little flaky on TBW data sets that have less
than a full complement or 12M (36M) samples.
Changed in version 0.4.0: Switched over from pure Python readers to the new C-base Go Fast! readers.
Class that stores the information found in the header of a TBW frame. All three fields listed in the DP ICD version H are stored as well as the original binary header data.
Function to parse the TBW ID field and return the size of number of bits that comprise the data. 12 is returned for 12-bit data, and 4 for 4-bit data.
Function to check if the data is really TBW and not TBN by examining the TBW ID field. Returns True if the data is TBW, false otherwise.
Function to parse the TBW ID field and return the stand number.
Class that stores the information found in the data section of a TBW frame. Both fields listed in the DP ICD version H are stored.
Function to convert the time tag from samples since the UNIX epoch (UTC 1970-01-01 00:00:00) to seconds since the UNIX epoch.
Class that stores the information contained within a single TBW frame. It’s properties are FrameHeader and FrameData objects.
Convenience wrapper for the Frame.FrameHeader.getDataBits function.
Convenience wrapper for the Frame.FrameData.getTime function.
Convenience wrapper for the Frame.FrameHeader.parseID function.
Function to read in a single TBW frame (header+data) and store the contents as a Frame object. This function wraps readerHeader and readData[(12)|4].
Find out the number of data bits used in the file be reading in the first frame.
Find out how many frames are present per observation by examining the first frames for what would be 260 stands. This is done by reading two frames and then skipping the next 30,000.
Note
Post-IOC it is probably simpler to adopt a value of the number of frames per observation of 260 rather than try to find it from the file.
Python module for reading data in from TBN files.This module defines the following classes for storing the TBN data found in a file:
object that contains all data associated with a particular TBN frame. The primary constituents of each frame are:
- FrameHeader - the TBN frame header object and
- FrameData - the TBN frame data object.
Combined, these two objects contain all of the information found in the original TBN frame.
In addition to storing the data available in the frame, the Frame object also has attributes for holding information about the gain, central frequency, and filter code used for the observations.
For reading in data, use the readFrame function. It takes a python file- handle as an input and returns a fully-filled Frame object. The readBlock function reads in a (user-defined) number of TBN frames and returns a ObservingBlock object.
For describing the format of data in the file, two function are provided:
Changed in version 0.4.0: Switched over from pure Python readers to the new C-base Go Fast! readers.
Changed in version 0.5.0: Support for ECR 11 TBN header format change.
Class that stores the information found in the header of a TBW frame. All three fields listed in the DP ICD version H are stored as well as the original binary header data.
Changed in version 0.5.0: Added various attributes to retrieve the central frequnecy and gain that are part of the ECR 11 changes.
Convert the tuning word to a frequency in Hz.
Function to convert the sample rate in Hz to a filter code.
Get the current TBN gain for this frame.
Function to check if the data is really TBN and not TBW by examining the TBN ID field. Returns True if the data is TBN, false otherwise.
Function to parse the TBN ID field and return a tuple of the stand number and polarization.
Function to set the sample rate of the TBN data in Hz.
Class that stores the information found in the data section of a TBN frame. Both fields listed in the DP ICD version H are stored.
Changed in version 0.5.0: Removed various attributes related to storing a central frequnecy and gain that aren’t needed with ECR 11.
Function to convert the time tag from samples since the UNIX epoch (UTC 1970-01-01 00:00:00) to seconds since the UNIX epoch.
Class that stores the information contained within a single TBN frame. It’s properties are FrameHeader and FrameData objects.
Changed in version 0.5.0: Removed various attributes related to storing a central frequnecy and gain that aren’t needed with ECR 11.
Convenience wrapper for the Frame.FrameHeader.getCentralFreq function.
Convenience wrapper for the Frame.FrameData.getFilterCode function.
Convenience wrapper for the Frame.FrameHeader.getGain function.
Convenience wrapper for the Frame.FrameData.getTime function.
Convenience wrapper for the Frame.FrameHeader.parseID function.
Convenience wrapper for the Frame.FrameData.setSampleRate function.
Function to read in a single TBN frame (header+data) and store the contents as a Frame object.
Function to read in a single TBN block (frames set by the nFrames keyword) and store the contents as a ObservingBlock object. This function wraps readFrame.
Find out what the sampling rate/filter code is from consecutive sets of observations. By default, the rate in Hz is returned. However, the corresponding filter code can be returned instead by setting the FilterCode keyword to True.
Find out how many frames are present per observation by examining the first 2,080 TBN frames. Return the number of frames per observations as a two- element tuple, one for each polarization.
So many TBN frames are read in order to try to compensate for the inter- leaving of the packets from the various DP1 boards during the recording.
Note
Post-IOC it is probably simpler to adopt a value of the number of frames per observation of 520 rather than try to find it from the file.
Python module to read in DRX data. This module defines the following classes for storing the DRX data found in a file:
object that contains all data associated with a particular DRX frame. The primary constituents of each frame are:
- FrameHeader - the DRX frame header object and
- FrameData - the DRX frame data object.
Combined, these two objects contain all of the information found in the original DRX frame.
For reading in data, use the readFrame function. It takes a python file- handle as an input and returns a fully-filled Frame object. The readBlock function reads in a (user-defined) number of DRX frames and returns a ObservingBlock object.
For describing the format of data in the file, two function are provided:
Finally, there are also two experimental functions defined in this file for averaging DRX observations. averageObservations performs a straight channel- by-channel average of a collection of DRX frames, which averageObservations2 allows for both temporal and spatial averaging.
Changed in version 0.4.0: Switched over from pure Python readers to the new C-base Go Fast! readers.
Class that stores the information found in the header of a DRX frame. All six fields listed in the DP ICD version H are stored as well as the original binary header data.
Function to convert the sample rate in Hz to a filter code.
Return the sample rate of the data in samples/second.
Parse the DRX ID into a tuple containing the beam (1 through 4), tunning (1 and 2), and polarization (0 and 1).
Class that stores the information found in the data section of a DRX frame. All three fields listed in the DP ICD version H are stored.
Function to set the central frequency of the DRX data in Hz.
Function to set the gain of the DRX data.
Class that stores the information contained within a single DRX frame. It’s properties are FrameHeader and FrameData objects.
Convenience wrapper for the Frame.FrameData.getCentralFreq function.
Convenience wrapper for the Frame.FrameHeader.getFilterCode function.
Convenience wrapper for the Frame.FrameHeader.getSampleRate function.
Function to convert the time tag from samples since the UNIX epoch (UTC 1970-01-01 00:00:00) to seconds since the UNIX epoch.
Convenience wrapper for the Frame.FrameHeader.parseID function.
Convenience wrapper for the Frame.FrameData.setGain function.
Class that stores all frames associates with a particular beam at a particular time.
Convenience wrapper for the Frame.FrameData.getCentralFreq function.
Note
This returned a two-element tuple giving the frequncies for both tunings.
Convenience wrapper for the Frame.FrameData.getFilterCode function.
Convenience wrapper for the Frame.FrameData.getTime function.
Convenience wrapper for the Frame.FrameData.setGain function.
Function to read in a single DRX frame (header+data) and store the contents as a Frame object. This function wraps readerHeader and readData.
Function to read in a single DRX block (four frames) and store the contents as a ObservingBlock object. This function wraps readFrame.
Find out what the sampling rate/filter code is from a single observations. By default, the rate in Hz is returned. However, the corresponding filter code can be returned instead by setting the FilterCode keyword to true.
This function is included to make easier to write code for TBN analysis and modify it for DRX data.
Find out how many beams are present by examining the first 16 DRX records. Return the number of beams found.
Find out how many frames are present per beam by examining the first 16 DRX records. Return the number of frames per observations as a four- element tuple, one for each beam.
New in version 0.5.
Python module to read in DR spectrometer data. This module defines the following classes for storing the spectra found in a file:
object that contains all data associated with a particular spectrometer frame. The primary constituents of each frame are:
- FrameHeader - the spectrometer frame header object and
- FrameData - the spectral data object.
Combined, these two objects contain all of the information found in the original spectrometer frame.
For reading in data, use the readFrame function. It takes a python file- handle as an input and returns a fully-filled Frame object For describing the format of data in the file, three function are provided:
- getSampleRate - get the sample rate in the file
- getFrameSize - get the total (header + data) frame size
- getTransformSize - get the FFT length
Note
This reader works with the most current version of the DR spectrometer data format as specified in the version 1.7. To read data created with previous versions of the DR spectrometer, use LSL version 0.5.2.
Class that stores the information found in the header of a DR spectrometer/DRX frame.
Function to convert the sample rate in Hz to a filter code.
Return the sample rate of the data in samples/second.
Return the beam the frame corresponds to.
Class that stores the information found in the data section of a DR spectrometer/ DRX frame.
Changed in version 0.5.3: Added the saturations field to keep up with saturation counts.
Function to set the central frequency of the DRX data in Hz.
Class that stores the information contained within a single DR spectrometer/ DRX frame. It’s properties are FrameHeader and FrameData objects.
Convenience wrapper for the Frame.FrameData.getCentralFreq function.
Convenience wrapper for the Frame.FrameHeader.getFilterCode function.
Convenience wrapper for the Frame.FrameHeader.getSampleRate function.
Function to convert the time tag from samples since the UNIX epoch (UTC 1970-01-01 00:00:00) to seconds since the UNIX epoch.
Convenience wrapper for the Frame.FrameHeader.parseID function.
Convenience wrapper for the Frame.FrameData.setGain function.
Function to read in a single DR spectrometer/DRX frame (header+data) and store the contents as a Frame object.
Find out what the sampling rate/filter code is from a single observations. By default, the rate in Hz is returned. However, the corresponding filter code can be returned instead by setting the FilterCode keyword to true.
Find out what the frame size in a file is at the current file location. Returns the frame size in bytes.
Find out what the transform size in a file is at the current file location.
Module that contains the error classes for the DRX, TBN, and TBW readers. These errors are currently meant to deal with file I/O problems.
Base class for file I/O problem during numpy.fromfile calls and out-of- sync Mark5C headers.
Extension to the base class for dealing with EOF errors. The error code is 1.
Extension to the base class for dealing with errors that occur when numpy.fromfile tried to read more data can exists. This is a specialized form of EOF error. The error code is 2.
Extension to the base class for dealing with Mark 5C header sync word problems. If the sync word doesn’t match what is expected. The error code is 3.
Extenstion to the base class for dealing with trying to read in TBW data with a TBN reader. The error code is 4.
Extenstion to the base class for dealing with trying to read in TBN data with a TBW reader. The error code is 5.
Function to provide a list of errors defined in this file. It alternatively takes an error code using the ‘errno’ keyword and returns its description.
Module to provide direct access to the files stored on a DRSU.
Warning
The direct access module also only compiles on Linux systems and may not work with 32-bit Linux installation.
Object to provide direct access to a file stored on a DRSU. The File object supports:
- open
- seek
- tell
- read
- close
Close the file object.
Return the underlying file object associated with an open file. None is returned if the file isn’t open.
Open the ‘file’ on the device and ready the File object for reading.
Read in a specified amount of data.
Seek in the file. All three ‘whence’ modes are supported.
Return the current position in the file by using the internal ‘bytesRead’ attribute value.
Function to return a list of File instances describing the files on a the specified DRSU device.
Note
Currently, the user needs to have read/write privileges to the device in question. This typically means running the script calling this function via sudo.
Function to return a File instance corresponding to the specified filename on the provided device. None is returned if the filename does not exists on the device.
Given a (open) File object and a reader module (e.g., lsl.reader.tbn), read in a single Frame instance. This function wraps the reader’s readFrame method and ‘shepherds’ the reading such that the file size specified by the File.size attribute is enforced on the reading process. This ensures that data beyond the ‘official’ end of the file are not read in.
Note
Currently, the user needs to have read/write privileges to the device in question. This typically means running the script calling this function via sudo.
Buffer for dealing with out-of-order/missing frames.
Changed in version 0.5: Removed support for DRX FrameBuffers since they shouldn’t be needed.
Changed in version 0.6: Removed support for TBW FrameBuffers since they didn’t really work.
Bases: object
Frame buffer for re-ordering TBW and TBN frames in time order. This class is filled with frames and a returns a frame list when the ‘nSegments’ starts filling. In that case, the oldest segment is returned.
The buffer also keeps track of what has already been read out so that tardy frames are just dropped. For buffers that are read out, missing frames are replaced with frames filled with zeros.
Note
Due to the nature of the buffer, it is possible that there will still be ‘nSegments’-1 segements in the buffer that are either full or partially full. This can be retrieved using the buffer’s ‘flush()’ function.
Note
DRX should _not_ need a ring buffer since the four beam outputs are, effectively, on different networks.
Append a new frame to the buffer with the appropriate time tag. True is returned if the frame was added to the buffer and False if the frame was dropped because it belongs to a buffer that has already been returned.
Figure of merit for storing/sorting frames in the ring buffer.
This will be overridden by sub-classes of FrameBuffer.
Return a list of lists containing all remaining frames in the buffer from buffers that are considered ‘full’. Afterwards, delete all buffers. This is useful for emptying the buffer after reading in all of the data.
Note
It is possible for this function to return list of packets that are mostly invalid.
Return a list of frames that consitute a ‘full’ buffer. Afterwards, delete that buffer and mark it as closed so that any missing frames that are recieved late are dropped. If none of the buffers are ready to be dumped, None is returned.
Print out the status of the buffer. This contains information about: 1. The current buffer fill level 2. The numer of full and partial buffer dumps preformed 3. The number of missing frames that fill packets needed to be created
for
Bases: lsl.reader.buffer.FrameBuffer
A sub-type of FrameBuffer specifically for dealing with TBN frames. See lsl.reader.buffer.FrameBuffer for a description of how the buffering is implemented.
Keywords: stands
list of stand to expect packets for
The number of segements in the ring can be converted to a buffer time in seconds:
| Segments | TBN Filter Code | ||||||
| 1 | 2 | 3 | 4 | 5 | 6 | 7 | |
| 10 | 5.1 | 1.6 | 0.8 | 0.4 | 0.2 | 0.1 | 0.05 |
| 20 | 10.2 | 3.3 | 1.6 | 0.8 | 0.4 | 0.2 | 0.10 |
| 30 | 15.4 | 4.9 | 2.5 | 1.2 | 0.6 | 0.3 | 0.15 |
| 40 | 20.5 | 6.6 | 3.3 | 1.6 | 0.8 | 0.4 | 0.20 |
| 50 | 25.6 | 8.2 | 4.1 | 2.0 | 1.0 | 0.5 | 0.26 |
| 100 | 51.2 | 16.4 | 8.2 | 4.1 | 2.0 | 1.0 | 0.51 |
Figure of merit for sorting frames. For TBN it is: <frame count of frame>