Robust Statistics

Changed in version 0.4.0: Function names have been changed in this module and new function added. See the below for more details.

Small collection of robust statistical estimators based on functions from Henry Freudenriech (Hughes STX) statistics library (called ROBLIB) that have been incorporated into the AstroIDL User’s Library. Function included are:

• biweightMean - biweighted mean estimator
• mean - robust estimator of the mean of a data set
• std - robust estimator of the standard deviation of a data set
• checkfit - return the standard deviation and biweights for a fit in order to determine its quality
• linefit - outlier resistant fit of a line to data
• polyfit - outlier resistant fit of a polynomial to data

For the fitting routines, the coefficients are returned in the same order as numpy.polyfit, i.e., with the coefficient of the highest power listed first.

For additional information about the original IDL routines, see:

http://idlastro.gsfc.nasa.gov/contents.html#C17

lsl.statistics.robust.biweightMean(inputData)

Calculate the mean of a data set using bisquare weighting.

Based on the biweight_mean routine from the AstroIDL User’s Library.

lsl.statistics.robust.mean(inputData, Cut=3.0)

Robust estimator of the mean of a data set. Based on the resistant_mean function from the AstroIDL User’s Library.

See also

lsl.misc.mathutil.robustmean()

lsl.statistics.robust.std(inputData, Zero=False)

Robust estimator of the standard deviation of a data set.

Based on the robust_sigma function from the AstroIDL User’s Library.

lsl.statistics.robust.checkfit(inputData, inputFit, epsilon, delta, BisquareLimit=6.0)

Determine the quality of a fit and biweights. Returns a tuple with elements:

0. Robust standard deviation analog
1. Fractional median absolute deviation of the residuals
2. Number of input points given non-zero weight in the calculation
3. Bisquare weights of the input points
4. Residual values scaled by sigma

This function is based on the rob_checkfit routine from the AstroIDL User’s Library.

lsl.statistics.robust.linefit(inputX, inputY, iterMax=25, Bisector=False, BisquareLimit=6.0, CloseFactor=0.03)

Outlier resistance two-variable linear regression function.

Based on the robust_linefit routine in the AstroIDL User’s Library.

lsl.statistics.robust.polyfit(inputX, inputY, order, iterMax=25)

Outlier resistance two-variable polynomial function fitter.

Based on the robust_poly_fit routine in the AstroIDL User’s Library.

Unlike robust_poly_fit, two different polynomial fitters are used because numpy.polyfit does not support non-uniform weighting of the data. For the weighted fitting, the SciPy Orthogonal Distance Regression module (scipy.odr) is used.

Statistical Tests

New in version 0.4.0.

Collection of statistical tests not found in any of the common python libraries.

lsl.statistics.stattests.waldwolfowitz(inputData)

Wald-Wolfowitz test of randomness. Given a numpy array of values compute the probability that the values are mutially independent.

See also

http://en.wikipedia.org/wiki/Wald%E2%80%93Wolfowitz_runs_test