Dataplot Vol 2 Vol 1

# F TEST

Name:
F TEST
Type:
Analysis Command
Purpose:
Perform a two sample F test that the ratio of two standard deviations is equal to 1 (i.e., the standard deviations are equal).
Description:
This F test can be either an upper tailed test or a two-sided test.

The two-sided hypothesis test is:

 H0: $$\frac{\sigma_1} {\sigma_2} = 1$$ Ha: $$\frac{\sigma_1} {\sigma_2} \ne 1$$ Test Statistic: $$\mbox{F} = \frac{s_{1}^{2}} {s_{2}^{2}}$$ with $$s_1$$ and $$s_2$$ denoting the sample standard deviations Alpha: Typically set to .05 Critical Region: $$\mbox{F} < f(\alpha/2)(\nu_1,\nu_2)$$ or $$\mbox{F} > f(1-\alpha/2)(\nu_1,\nu_2)$$ and where the critical region is determined from the F distribution function with $$\nu_1-1$$ and $$\nu_2-1$$ degrees of freedom Conclusion: Reject the null hypothesis if F in critical region

Note that for the upper tailed test, the larger of $$s_1$$ and $$s_2$$ is in the numerator and the smaller is in the denominator of the test statistic.

The upper tailed hypothesis test is:

 H0: $$\frac{max(\sigma_1,\sigma_2)} {min(\sigma_1,\sigma_2)} = 1$$ Ha: $$\frac{max(\sigma_1,\sigma_2)} {min(\sigma_1,\sigma_2)} > 1$$ Test Statistic: $$\mbox{F} = \frac{max(s_1,s_2)^2} {min(s_1,s_2)^2}$$ with $$s_1$$ and $$s_2$$ denoting the sample standard deviations Alpha: Typically set to .05 Critical Region: $$\mbox{F} > f(1-\alpha,\nu_1,\nu_2)$$ where the critical region is determined from the F distribution function with $$\nu_1 - 1$$ and $$\nu_2 - 1$$ degrees of freedom Conclusion: Reject null hypothesis if F in critical region

To specify the upper tailed test (the default), enter

SET F TEST TYPE UPPER TAILED

To specify the two-tailed test, enter

SET F TEST TYPE TWO SIDED

The test conclusions are given for several values of alpha.

Note that the F test is known to be quite senstive to departures from normality. There have been several proposals in the literature to make this test more robust. This is discussed further in the "Note:" section below.

Syntax 1:
F TEST <y1> <y2>             <SUBSET/EXCEPT/FOR qualification>
where <y1> is the first response variable;
<y2> is the second response variable;
and where the <SUBSET/EXCEPT/FOR qualification> is optional.
Syntax 2:
F TEST <y1> ... <yk>             <SUBSET/EXCEPT/FOR qualification>
where <y1> ... <yk> is a list of two or more response variables;
and where the <SUBSET/EXCEPT/FOR qualification> is optional.

This syntax performs all the two-way F tests for the listed variables. This syntax supports the TO syntax.

Examples:
F TEST Y1 Y2
F TEST Y1 Y2 Y3
F TEST Y1 TO Y10
F TEST Y1 Y2 SUBSET Y2 > 0

SET F TEST TYPE TWO SIDED
F TEST Y1 Y2

SET F TEST TYPE UPPER TAILED
F TEST Y1 Y2

Note:
The F test is known to be quite sensitive to departures from normality.

Shoemaker (2003) proposed the following adjustment to the degrees of freedom to make the F test more robust against non-normality.

The degrees of freedom are

$$r_{i} = \frac{2 n_{i}} {\frac{\mu_{4}}{\sigma^{4}} - \frac{n_{i}-3}{n_{i}-1}}$$

where

$$\mu_{4} = \frac{\sum_{i=1}^{2}{\sum_{j=1}^{n_i} {(y_{ij} - \bar{y}_{i})^{4}}}} {n_{1} + n_{2}}$$

$$\sigma^{2} = \frac{(n_{1} - 1) s_{1}^{2} + (n_{2} - 1) s_{2}^{2}} {n_{1} + n_{2}}$$

with $$y_{ij}$$, $$n_{i}$$, $$s_{i}$$ and $$\bar{y}_{i}$$ denoting the data, the sample size, the standard deviation and the mean, respectively, of the i-th variable.

This typically results in fractional degrees of freedom. The Dataplot FCDF, FPDF and FPPF routines were updated to support fractional degrees of freedom (fractional degrees of freedom less than 1 are set to 1).

The justification and derivation of this modification is given in Shoemaker's paper. Simulations by Shoemaker indicated that the standard F test tends to be too conservative for light tailed distributions and too liberal for heavy tailed and skewed distributions. Using the Shoemaker modifications for degrees of freedom generally had good coverage properties with the exception of heavily skewed distributions with small samples where it was still too liberal.

Bonett (2006) suggested that rounding the numerator degrees of freedom up to the nearest integer degrees of freedom and rounding the denominator degrees of freedom down to the nearest integer degrees of freedom (but not less than 1) prevented some anomalous behavior while reducing coverage performance only slightly.

Bonett proposed the following statistic:

$$\exp{ \frac{ \ln{ \frac{c s_{1}^{2}} {s_{2}^{2}} }} {se}}$$

where

 $$\gamma_{4}$$ = $$(n_1 + n_2) \frac{ \sum_{i=1}^{2}{\sum_{j=1}^{n_i}{(y_{ij} - m_{i})^{4}}}} {(\sum_{i=1}^{2}{\sum_{j=1}^{n_i}{(y_{ij} - \mu_{i})^{2}}} )^{2}}$$ $$m_{i}$$ = sample trimmed mean of the i-th variable with trimming proportion $$\frac{1}{\sqrt{2 (n{i} - 4)}}$$ $$\bar{y_{i}}$$ = sample mean of the i-th variable se = $$\sqrt{ \frac{\gamma_{4} - k_{1}} {n_{1} - 1} + \frac{\gamma_{4} - k_{2}} {n_{2} - 1} }$$ $$k_1$$ = $$\frac{n_{1}-3}{n_{1}}$$ $$k_2$$ = $$\frac{n_{2}-3}{n_{2}}$$ c = $$\frac{\frac{n_{1}} {n_{1} - \Phi^{-1}(\alpha/2)}} {\frac{n_{2}} {n_{2} - \Phi^{-1}(\alpha/2)}}$$ $$\Phi^{-1}$$ = the percent point function of the normal distribution

The parameter c is a small sample adjustment to help equalize the tail probabilities. It equals 1 when n1 and n2 equal and approaches 1 as n1 and n2 get large. For unequal sample sizes, the value of the test statistic will vary slightly depending on the value of alpha.

The above formula is for the variance. For the standard deviation, take the square root.

This test statistic is compared to a standard normal distribution.

The justification and derivation of this test is given in the Bonett paper. Based on his simulations, he claims this method improves somewhat on the Shoemaker modification for heavy-tailed (particularly skewed) distributions.

To specify Shoemaker's modification with fractional degrees of freedom, enter

SET F TEST DEGREES OF FREEDOM SHOEMAKER

To specify Shoemaker's modification with integral degrees of freedom as suggested by Bonett, enter

SET F TEST DEGREES OF FREEDOM SHOEMAKER ROUNDED

To specify Bonett's method, enter

SET F TEST DEGREES OF FREEDOM BONETT
Note:
Shoemaker suggests that when more than two groups of data are available, it can improve accuracy to use all of the groups in the estimates of the pooled location (mu4) and pooled variance (sigma**2) instead of just the two groups being tested. That is,

$$\mu_{4} = \frac{\sum_{i=1}^{k}{\sum_{j=1}^{n_i} {(y_{ij} - \bar{y}_{i})^{4}}}} {n_{1} + n_{2} + ... + n_{k}}$$

$$\sigma^{2} = \frac{(n_{1} - 1) s_{1}^{2} + (n_{2} - 1) s_{2}^{2} + ... + (n_{k} - 1) s_{k}} {n_{1} + n_{2} + ... + n_{k}}$$

To use these values, enter the commands

SET SHOEMAKER F TEST POOLED MU <value>
SET SHOEMAKER F TEST POOLED VARIANCE <value>

To reset the default of Dataplot computing these from the two groups being tested, enter

SET SHOEMAKER F TEST POOLED MU
SET SHOEMAKER F TEST POOLED VARIANCE 0
Note:
The following parameters are saved after the F test is performed.

 STATVAL - value of the test statistic STATCDF - CDF of the test statistic PVALUE - p-value of the test statistic STATNU1 - degrees of the freedom of the numerator variable STATNU2 - degrees of the freedom of the denominator variable POOLSD - pooled standard deviation

For an upper tailed test, the following parameters are saved

 CUTUPP50 - upper critical value for 50% test CUTUPP75 - upper critical value for 50% test CUTUPP90 - upper critical value for 90% test CUTUPP95 - upper critical value for 95% test CUTUPP99 - upper critical value for 99% test CUTUP999 - upper critical value for 99.9% test

For a two sided test, the following are saved

 CUTLOW80 - lower critical value for 80% test CUTUPP80 - upper critical value for 80% test CUTLOW90 - lower critical value for 90% test CUTUPP90 - upper critical value for 90% test CUTLOW95 - lower critical value for 95% test CUTUPP95 - upper critical value for 95% test CUTLOW99 - lower critical value for 99% test CUTUPP99 - upper critical value for 99% test CUTLW999 - lower critical value for 99.9% test CUTUP999 - upper critical value for 99.9% test

If the Bonett method is specified, a two sided interval will always be used. Also, STATNU1, STATNU2 and POOLSD will not be saved.

Note:
In addition to the F TEST command, the following commands can also be used:

LET STATVAL = F TEST Y1 Y2
LET STATCDF = F TEST CDF Y1 Y2
LET PVALUE = F TEST PVALUE Y1 Y2

In addition to the above LET commands, built-in statistics are supported for 30+ different commands (enter HELP STATISTICS for details).

Note:
Default:
An upper tailed test without the Shoemaker or Bonett modifications is performed.
Synonyms:
None
Related Commands:
 RATIO OF SD CONFIDENCE LIMITS = Compute the confidence limtis for the ratio of two standard deviations. SIEGEL TUKEY TEST = Perform a ranks based test that two standard deviations are equal. BARTLETT TEST = Perform a k-sample Bartlett test for homogeneous variances. LEVENE TEST = Perform a k-sample Levene test for homogeneous variances. SQUARED RANKS TEST = Perform a k-sample squared ranks test for homogeneous variances. KLOTZ TEST = Perform a k-sample Klotz test for homogeneous variances. SD CONFIDENCE LIMITS = Compute the confidence limits for the standard deviation. CHI-SQUARE TEST = Performs a one sample chi-square test that the standard deviation is equal to a given value. T TEST = Performs a two-sample t test for equal means. CONFIDENCE LIMITS = Compute the confidence limits for the mean of a sample. STANDARD DEVIATION = Computes the standard deviation of a variable.
Reference:
F tests are discussed in most introductory statistics books.

Shoemaker (2003), "Fixing the F-Test for Equal Variances," The American Statistician, Vol. 57, pp. 105-114.

Bonett (2006), "Robust Confidence Intervals for a Ratio of Standard Deviations," Applied Pyschological Measurement, Vol. 30, No. 5, pp. 432-439.

Applications:
Confirmatory Data Analysis
Implementation Date:
1994/02
2023/06: Added support for Shoemaker and Bonett modifications
2023/06: Added support for two sided tests
Program 1:


SKIP 25
DELETE Y2 SUBSET Y2 < 0
.
F TEST Y1 Y2

The following output is generated

Two Sample F-Test for Equal Standard Deviations

First Response Variable:  Y1
Second Response Variable: Y2

H0: Sigma1/Sigma2 = 1
Ha: Sigma1/Sigma2 > 1

Sample One Summary Statistics:
Number of Observations:                             249
Sample Mean:                                   20.14458
Sample Standard Deviation:                      6.41470

Sample Two Summary Statistics:
Number of Observations:                              79
Sample Mean:                                   30.48101
Sample Standard Deviation:                      6.10771

Test:
Standard Deviation (Numerator):                 6.41470
Standard Deviation (Denomerator):               6.10771
Degrees of Freedom (Numerator):                     248
Degrees of Freedom (Denomerator):                    78
Pooled Standard Deviation:                      6.34260
F-Test Statistic Value:                         1.10305
F-Test CDF Value:                               0.69032
F-Test P-Value:                                 0.30968

Conclusions (Upper 1-Tailed Test)

H0: sigma1/sigma2 = 1; Ha: sigma1/sigma2 > 1
------------------------------------------------------------
Null
Significance           Test       Critical     Hypothesis
Level      Statistic    Region (>=)     Conclusion
------------------------------------------------------------
50.0%        1.10305        1.00590         REJECT
75.0%        1.10305        1.14047         ACCEPT
90.0%        1.10305        1.28015         ACCEPT
95.0%        1.10305        1.37347         ACCEPT
97.5%        1.10305        1.46106         ACCEPT
99.0%        1.10305        1.57145         ACCEPT
99.9%        1.10305        1.83565         ACCEPT


Program 2:

. Step 1:   Read the data
.
skip 25
let y x = stack bottom mid surface
let temp1 = cross tabulate mean y x
let temp2 = (y - temp1)**4
let num = sum temp2
let den = size y
let mu4 = num/den
let y1var = variance bottom
let y2var = variance mid
let y3var = variance surface
let n1 = size bottom
let n2 = size mid
let n3 = size surface
let num   = (n1 - 1)*y1var + (n2 -1)*y2var + (n3 - 1)*y3var
let poolvar = num/den
.
. Step 2:   Generate a box plot
.
character box plot
line box plot
xlimits 1 3
major xtic mark number 3
minor xtic mark number 0
xtic mark offset 0.5 0.5
x1tic mark label format alpha
x1tic mark label content Bottom Middepth Surface
tic mark label case asis
set box plot fences on
box plot y x

.
. Step 3:   Default F-test
.
set f test degrees of freedom default
print "Standard F Test"
f test bottom mid
let statval = f test        bottom mid
let statcdf = f test cdf    bottom mid
let pvalue  = f test pvalue bottom mid
print statval statcdf pvalue

The following output is generated
Standard F Test

Two Sample F-Test for Equal Standard Deviations

First Response Variable:  BOTTOM
Second Response Variable: MID

H0: Sigma1/Sigma2 = 1
Ha: Sigma1/Sigma2 > 1

Sample One Summary Statistics:
Number of Observations:                              10
Sample Mean:                                    6.02100
Sample Standard Deviation:                      1.58184

Sample Two Summary Statistics:
Number of Observations:                              10
Sample Mean:                                    5.01900
Sample Standard Deviation:                      1.10440

Test:
Standard Deviation (Numerator):                 1.58184
Standard Deviation (Denomerator):               1.10440
Degrees of Freedom (Numerator):                       9
Degrees of Freedom (Denomerator):                     9
Pooled Standard Deviation:                      1.36417
F-Test Statistic Value:                         2.05152
F-Test CDF Value:                               0.85031
F-Test P-Value:                                 0.14969

Conclusions (Upper 1-Tailed Test)

H0: sigma1/sigma2 = 1; Ha: sigma1/sigma2 > 1
------------------------------------------------------------
Null
Significance           Test       Critical     Hypothesis
Level      Statistic    Region (>=)     Conclusion
------------------------------------------------------------
50.0%        2.05152        1.00000         REJECT
75.0%        2.05152        1.59090         REJECT
90.0%        2.05152        2.44034         ACCEPT
95.0%        2.05152        3.17889         ACCEPT
97.5%        2.05152        4.02599         ACCEPT
99.0%        2.05152        5.35113         ACCEPT
99.9%        2.05152       10.10663         ACCEPT

THE COMPUTED VALUE OF THE CONSTANT STATVAL       =    2.051516

THE COMPUTED VALUE OF THE CONSTANT STATCDF       =   0.8503082

THE COMPUTED VALUE OF THE CONSTANT PVALUE        =   0.1496918

PARAMETERS AND CONSTANTS--

STATVAL --        2.05152
STATCDF --        0.85031
PVALUE  --        0.14969

.
. Step 4:   Shoemaker F-test
.
set f test degrees of freedom shoemaker
print "Shoemaker degrees of freedom F Test"
f test bottom mid
print "Use pooled location and variance for Shoemaker"
set shoemaker f test pooled variance poolvar
set shoemaker f test pooled mu       mu4
f test bottom mid
print "Shoemaker rounded degrees of freedom F Test"
set f test degrees of freedom shoemaker rounded
f test bottom mid
set shoemaker f test pooled variance 0
set shoemaker f test pooled mu

The following output is generated
Shoemaker degrees of freedom F Test

Two Sample F-Test for Equal Standard Deviations
(Use Shoemaker Modifications to Degrees of Freedom)

First Response Variable:  BOTTOM
Second Response Variable: MID

H0: Sigma1/Sigma2 = 1
Ha: Sigma1/Sigma2 > 1

Sample One Summary Statistics:
Number of Observations:                              10
Sample Mean:                                    6.02100
Sample Standard Deviation:                      1.58184

Sample Two Summary Statistics:
Number of Observations:                              10
Sample Mean:                                    5.01900
Sample Standard Deviation:                      1.10440

Test:
Standard Deviation (Numerator):                 1.58184
Standard Deviation (Denomerator):               1.10440
Degrees of Freedom (Numerator):                12.42315
Degrees of Freedom (Denomerator):              12.42315
Pooled Standard Deviation:                      1.36417
F-Test Statistic Value:                         2.05152
F-Test CDF Value:                               0.89036
F-Test P-Value:                                 0.10964

Conclusions (Upper 1-Tailed Test)

H0: sigma1/sigma2 = 1; Ha: sigma1/sigma2 > 1
------------------------------------------------------------
Null
Significance           Test       Critical     Hypothesis
Level      Statistic    Region (>=)     Conclusion
------------------------------------------------------------
50.0%        2.05152        1.00000         REJECT
75.0%        2.05152        1.47952         REJECT
90.0%        2.05152        2.11767         ACCEPT
95.0%        2.05152        2.63791         ACCEPT
97.5%        2.05152        3.20501         ACCEPT
99.0%        2.05152        4.04346         ACCEPT
99.9%        2.05152        6.73615         ACCEPT

Use pooled location and variance for Shoemaker

THE FORTRAN COMMON SCALAR SHOEF    HAS JUST BEEN SET TO   0.1251562E+01

THE FORTRAN COMMON SCALAR SHOEF    HAS JUST BEEN SET TO   0.4585221E+01

Two Sample F-Test for Equal Standard Deviations
(Use Shoemaker Modifications to Degrees of Freedom)

First Response Variable:  BOTTOM
Second Response Variable: MID

H0: Sigma1/Sigma2 = 1
Ha: Sigma1/Sigma2 > 1

Sample One Summary Statistics:
Number of Observations:                              10
Sample Mean:                                    6.02100
Sample Standard Deviation:                      1.58184

Sample Two Summary Statistics:
Number of Observations:                              10
Sample Mean:                                    5.01900
Sample Standard Deviation:                      1.10440

Test:
Standard Deviation (Numerator):                 1.58184
Standard Deviation (Denomerator):               1.10440
Degrees of Freedom (Numerator):                 9.30474
Degrees of Freedom (Denomerator):               9.30474
Pooled Standard Deviation:                      1.36417
F-Test Statistic Value:                         2.05152
F-Test CDF Value:                               0.85454
F-Test P-Value:                                 0.14546

Conclusions (Upper 1-Tailed Test)

H0: sigma1/sigma2 = 1; Ha: sigma1/sigma2 > 1
------------------------------------------------------------
Null
Significance           Test       Critical     Hypothesis
Level      Statistic    Region (>=)     Conclusion
------------------------------------------------------------
50.0%        2.05152        1.00000         REJECT
75.0%        2.05152        1.57799         REJECT
90.0%        2.05152        2.40166         ACCEPT
95.0%        2.05152        3.11256         ACCEPT
97.5%        2.05152        3.92318         ACCEPT
99.0%        2.05152        5.18299         ACCEPT
99.9%        2.05152        9.64550         ACCEPT

Shoemaker rounded degrees of freedom F Test

THE FORTRAN COMMON CHARACTER VARIABLE F   TEST HAS JUST BEEN SET TO SHO2

Two Sample F-Test for Equal Standard Deviations
(Shoemaker Modifications with Rounded Degrees of Freedom)

First Response Variable:  BOTTOM
Second Response Variable: MID

H0: Sigma1/Sigma2 = 1
Ha: Sigma1/Sigma2 > 1

Sample One Summary Statistics:
Number of Observations:                              10
Sample Mean:                                    6.02100
Sample Standard Deviation:                      1.58184

Sample Two Summary Statistics:
Number of Observations:                              10
Sample Mean:                                    5.01900
Sample Standard Deviation:                      1.10440

Test:
Standard Deviation (Numerator):                 1.58184
Standard Deviation (Denomerator):               1.10440
Degrees of Freedom (Numerator):                      10
Degrees of Freedom (Denomerator):                     9
Pooled Standard Deviation:                      1.37649
F-Test Statistic Value:                         2.05152
F-Test CDF Value:                               0.85275
F-Test P-Value:                                 0.14725

Conclusions (Upper 1-Tailed Test)

H0: sigma1/sigma2 = 1; Ha: sigma1/sigma2 > 1
------------------------------------------------------------
Null
Significance           Test       Critical     Hypothesis
Level      Statistic    Region (>=)     Conclusion
------------------------------------------------------------
50.0%        2.05152        1.00774         REJECT
75.0%        2.05152        1.58634         REJECT
90.0%        2.05152        2.41632         ACCEPT
95.0%        2.05152        3.13728         ACCEPT
97.5%        2.05152        3.96387         ACCEPT
99.0%        2.05152        5.25654         ACCEPT
99.9%        2.05152        9.89430         ACCEPT

.
. Step 5:   Bonett
.
set f test degrees of freedom bonett
print "Bonett method"
f test bottom mid

The following output is generated
Bonett method

Two Sample F-Test for Equal Standard Deviations
(Bonett Method for Robustness)

First Response Variable:  BOTTOM
Second Response Variable: MID

H0: Sigma1/Sigma2 = 1
Ha: Sigma1/Sigma2 > 1

Sample One Summary Statistics:
Number of Observations:                              10
Sample Mean:                                    6.02100
Sample Standard Deviation:                      1.58184

Sample Two Summary Statistics:
Number of Observations:                              10
Sample Mean:                                    5.01900
Sample Standard Deviation:                      1.10440

Test:
Standard Deviation (Numerator):                 1.58184
Standard Deviation (Denomerator):               1.10440
Test Statistic Value (alpha = 0.05):            3.23288
Test CDF Value (alpha = 0.05):                  0.99939
Test P-Value (alpha = 0.05):                    0.00123

Conclusions

H0: sigma1/sigma2 = 1; Ha: sigma1/sigma2 <> 1
---------------------------------------------------------------------------
Lower          Upper           Null
Significance           Test       Critical       Critical     Hypothesis
Level      Statistic    Region (<=)    Region (>=)     Conclusion
---------------------------------------------------------------------------
50.0%        3.23288       -0.67449        0.67449         REJECT
80.0%        3.23288       -1.28155        1.28155         REJECT
90.0%        3.23288       -1.64485        1.64485         REJECT
95.0%        3.23288       -1.95996        1.95996         REJECT
99.0%        3.23288       -2.57583        2.57583         REJECT
99.9%        3.23288       -3.29053        3.29053         ACCEPT


Date created: 06/07/2023
Last updated: 06/07/2023