wiki:Commissioning

Version 11 (modified by jayce, 5 years ago) (diff)

Updated the information about the TBW scripts.

LWA1 Commissioning Scripts

Overview and Requirements

This is a collection of scripts use for commissioning and testing at the LWA1 station. All of the scripts depend on the LSL 0.6.x branch.

The Commissioning scripts build off of the LSL 0.6.x framework for working with LWA data and also depends on the following python modules:

  • h5py
  • pytz
  • wxPython

No installation (e.g., python setup.py install) is required to use the software. Simply run the scripts in the Commissioning directory.

Obtaining

The easiest way to obtain the latest version of the commissioning scripts is via subversion export/checkout:

svn checkout http://fornax.phys.unm.edu/lwa/subversion/trunk/Commissioning

Contents

General

checkDRSU.sh

Shell script to run on Linux machines with attached and powered-on DRSU to report the firmware version of Seagate drives with model number "ST31000525SV". This is useful for identifying disks with the newer (and less reliable) firmware version "CV12".

time2time.py

Given a date/time in the format of "YYYY/MM/DD HH:MM:SS[.SSS]" in local time, compute the corresponding MJD and MPM in UTC. If no date/time is specified, the current local date/time is used for the conversion.

mjd2local.py

For a given MJD value or list of MJD values, return the range of local times associated with that MJD.

astroevents2.py

List the rise, transit, and set times for the brightest radio objects in the sky.

estimateData.py

Script to estimate the data volume of TBN and DRX observations.

findMyStand.py

Simple script to locate and label where stands are in the array.

plotShelterTemp.py

Short script to read in a shelter.txt file (log of shelter temperature with each line consisting of a UNIX time stamp and a shelter temperature) and plot all of the available data.

Update: This script is also now compatiable with the /data/thermometer##.txt files created by the new SHL MCS.

plotPDUVoltages.py

Short script to plot the input PDU voltages over time from the /data/rack##.txt files generated by the new SHL MCS software.

plotBoardTemps.py

Short script to read in a temp.txt file (log of DP FPGA temperatures) and plot all of the available data.

plotChassisTemps.py

Script to plot colormaps of the FPGA temperatures as a function of time, chassis, and physical slot.

TBW

checkTimetags.py

Script for checking for missing frames and bad time tags in a full DP TBW capture. If problems are found the error are noted and printed out. It shouldn't be too hard to modify this for fewer inputs.

tbw2hdf.py

Export select stands from a TBW file to HDF5.

stationMaster.py

Modified version of tbwSpectra.py that is included with LSL (version >= 0.4.0) that estimated the frequency (in MHz) of each dipole's resonance point and saves that information along with time-average spectra to a NPZ file.

tinyMaster.py

Modified version of stationMaster.py that uses Numpy memory maps to work on systems with less than 16 GB of memory.

stationMaster2.py

Modified version of stationMaster.py that uses the 'ClipLevel?' keyword in LSL >= 0.4.2 to blank impulsive RFI events.

tinyMaster2.py

Modified version of stationMaster2.py that uses Numpy memory maps to work on systems with less than 16 GB of memory.

smGUI.py

GUI that interfaces with a NPZ file created by stationMaster.py that makes looking for problems with dipoles/mappings/etc. a point-and-click exercise.

rfiCheck.py

Script to take a single TBW capture and create a RFI-centered HDF5 file for stands 1, 10, 54, 248, 251, and 258 (the outlier). These stands correspond to the four corners of the array, the center, and the outlier. The HDF5 contains values for the spectral kurtosis estimated from the data and various statistics about the timeseries (mean, std. dev., percentiles, etc.)

plotRFI.py

Script to take the output of rfiCheck.py and make a series of plots to look at.

TBN

checkTimetags.py

Simple script for making sure that time tags update properly on the prototype DP system. This script won't work for the full DP system because of how the packets are interleaved.

checkTimetags2.py

Modified version of checkTimetags.py that uses the LSL TBN ring buffer to reorder the interleaved packets. The script not only checks time tags like its predecessor but also checks for dropped packets and synchronization problems.

eyeDiagram.py

Create an eye diagram for some portion of a TBN file.

eyeDiagram2.py

Look for glitches in a TBN file by fitting a quantized sine wave to the data.

tbnTimeseries.py

Alpha-version script to plot out time series I/Q data from a TBN file.

checkMissing.py

Look at what is coming out of the TBN ring buffer and plot of what frames are missing, if any, as a function of time in the file.

DRX

decodeStatus.py

Take the status code returned by a DP_ BOARD_STAT query and break it down into its constituent parts.

gain.py

JPL-created script for creating gain files to be used by DRX.

delay.py

JPL-create script for created delay files to be used by DRX.

test_analog_in2.py

Script for setting up a test DRX observation with a variety of tuning, frequency, bandwidth, and gain settings. Needs gain.py and delay.py to work.

generateDelays.py

Generate gain and delay files for a particular topocentric pointing (azimuth and elevation in degrees) at a specified frequency.

genernateDelays.py generates text files for gains and delays that need to be converted to binary packed files with the MCS/Executive commands megfg and medfg.

generateDelaysMixed.py

Modified version of generateDelays.py that forms the beam only with the X pol. and leaves only the outlier (#258) on the Y pol.

fringeSets.py

Read in SSMIF file and create a set of DRX gain files (zero-delay for dipoles-only) that puts a single dipole or beam on the X pol and the outlier on the other. The gain files are constructed such that all data is from X pol.

estimateBeam.py

Use the LSL misc.beamformer's integer delay and sum to estimate the beam shape towards a particular azimuth and elevation and save the beam to a Numpy NPZ file.

driftcurve.py

Using the beam created by estimateBeam.py, generate a temperature as a function of time for that pointing over an entire day. This script is based on the LSL driftcurve.py script.

checkTimetags.py

Simple script to check the time tag sequence in a DRX file and look for strange jumps and other badness. It can also be used to check the DRX decimate (sample rate) field and if the time has been set correctly on the various boards.

getTuningOffset.py

Check for an offset in the time tags associated with the two tunings beyond what is controlled by the time offset stored in the frame header. This should read in the first complete set of frame from all four beamtunepols and compare tunings 1 and 2. Any differences between the times are noted.

checkTuningCoherency.py

Given a a DRX file with both tunings set to the same parameters, check for coherency across the tunings via cross-correlation. For each set of data, print out the lag (in DRX samples) where the cross-correlation function has its peak and the normalized correlation magnitude relative to the previous set of tuning 1 data.

beamCoherency.py

Given two or more DRX files from different beams, check for coherency between the beams and make sure that the beams agree with the T_NOM values. For each beam/tuning/pair, print out the lag (converted to ticks), the time tag differences (both raw and T_NOM corrected) and the normalized correlation magnitude relative to the same-beam tuning cross-correlation.

drxTimeseries.py

Script for plotting time series data from a DRX file. This file is also available as part of the LSL package (version >= 0.4.0) but is included here since this is the script we actually used at JPL for pre-acceptance testing.

drxPower.py

Modified version of drxTimeseries.py that plots the power for each DRX sample instead of the raw I/Q values. The power is summed for the integration time (listed as average) and displayed for the duration that it is computed.

drxSpectra.py

Script for reading in DRX data and created time-averaged power spectral density plots. Like drxTimeseries.py, this file is also available as part of the LSL package (version >= 0.4.0) but is included here since this is the script we actually used at JPL for pre-acceptance testing.

drxWaterfall.py

Read in DRX files and create a collection of time-averaged spectra. These spectra are saved to a NPZ file called <filename>-waterfall.npz.

plotWaterfall.py

Given the NPZ output of drxWaterfall, plot it in an interative way.

eyeDiagram.py

Create an eye diagram for some portion of a DRX file.

eyeDiagram2.py

Look for glitches in a DRX file by fitting a quantized sine wave to the data.