Statistical test procedures use numerical data sets which can have two types of representation:

For the standard type representation separate data sets are stored in the data table in separate columns. For the key-type representation the two data sets are stored in one 'united' column while an additional key column indicates the sample for the particular element. The key column must contain a variable with only two possible outcomes (either categorical or numerical).

Null (missing) values in the data set are supported and the missed observations are omitted.

All statistical tests are accessible from the script level. In order to apply a particular statistical test to sample data a script should be created.

The following commands are used for implementing the tests:

runStatTest(test, table)  or
runStatTest(test, table, col1)  or
runStatTest(test, table, col1, col2)
                

The general form of the command is

runStatTest(test, alias.table, col1, col2)
                

where

If no column names are given, the test procedure will take the first column from the specified table and try to implement the test.

The test object is created in two steps:

1) creating the corresponding test settings

2) creating the selected test object using the commands:

settings = TestNameSettings()
test = TestName(settings)
                

where settings and test are the names of the corresponding objects.

A list of all elements of settings for each statistical test can be found in the Settings section for the corresponding test. TestName /TestNameSettings can be any of the following:

The results of the test are stored as an object with the chosen name:

res = runStatTest(test, alias.table, col1, col2)
                

Here res is the name of the resulting object of the test implementation for the set (test, alias.table, col1, col2) . The result of the test contains four elements: the value of the test statistic, the critical value(s) for the given significance level, the p-value for the calculated test statistic value and the resulting hypothesis (null hypothesis is rejected or accepted), which can be printed to the output window using the following commands:

                    res.printResults()
                

to print all results or

                    res.getHypothesis()
                    res.getStatistic()
                    res.getCriticalValue()
                    res.getPValue()
                

to extract the particular elements separately.

The getHypothesis() command prints the resulting hypothesis as 0/1, which corresponds to null hypothsis/alternative hypothesis.

Below is a list of all commands for the definition of the separate elements of a selected settings object (the default parameter values can be found in the settings table for each test):

The admissible values for each element of the settings can be found in the description of each test in the Settings section. For viewing the setting values the following commands can be used:

table 'statTest':
    a b

    1 2
    5 6
    1 7
    2 4
    5 8


settingsMW = MannWhitneyTestSettings()
settingsMW.setAlpha(0.1)
testMW = MannWhitneyTest(settingsMW)
resMW = runStatTest(testMW, 'statTest', 'a', 'b')
resMW.printResults()

        StatTestResult: 

        1) Critical value:         1.6448536269514715        
        2) Statistic:        -1.6711454971746993
        3) p-value:        0.09469293918766253
        4) Hypothesis:        null hypotesis is rejected

The Anderson-Darling test is defined as:

Hypothesis:

Test Statistic: The Anderson-Darling test statistic is defined as

and is calculated as , where

Here are the ordered data and is the sample size, is the cumulative distribution function of the specified distribution, is the empirical distribution function, which estimates the distribution function .

For every the value is the proportion of observations less than or equal to , while is the probability of an observation less than or equal to .

Confidence Level:

Critical Region:

The normal and lognormal distributions require the following modification for small samples:

Then the test statistic has to be compared to the appropriate critical value.

For the Weibull and Gumbel distributions the test statistic for small samples is calculated as follows:

As for the exponential distribution, it is a special case of the Weibull distribution with the scale parameter equal to the mean parameter of the exponential distribution and the shape parameter equal to 1.

The probability of a larger test statistic is obtained by linear interpolation within the range of the simulated critical values (see D'Agostino and Stephens 1986).

P-Value:

The P-value is equal to the probability of a random variable with Andeson-Darling distribution to be greater than or equal to the observed value of the AD statistic under the null hypothesis.

Data Requirements:

The Anderson-Darling test is performed for one sample of numerical data, which can be stored either in one column (standard representation) or in two columns (key-type representation).

This test can be either two-sided test or one-sided. The two-sided version tests against the alternative that the true variance is not equal to the specified value. The one-sided version tests only in one direction: the true variance is less than the predetermined value (left-sided test) or the true variance is greater than the predetermined value (right-sided test). The choice of the two-sided versus one-sided test is determined by the problem.

Hypothesis:

Test Statistic:

The test statistic has the form

where is the sample size and is the sample variance. It can be seen from this formula that the test compares the ratio of the sample variance to the target variance. The more this ratio deviates from 1, the more likely we are to reject the null hypothesis.

Under the null hypothesis the test statistic has a chi-square distribution with degrees of freedom.

Confidence Level:

Critical Region:

The null hypothesis is rejected if:

Two-sided test :

Left-sided test :

Right-sided test :

Here and are respectively the lower and upper quantile of a chi-square distribution with degrees of freedom.

P-Value:

The P-value is equal to the probability that a random variable with a chi-square distribution with degrees of freedom will be greater or equal to the calculated value of the test statistic under the null hypothesis.

Data Requirements:

The Chi-square test can be performed for numerical samples in the standard or key-type representations.

There are two versions of this test: two-tailed and one-tailed. The two-tailed test is used to test whether standard deviations are equal. The one-tailed version tests in one direction: the left-sided variant checks whether the standard deviation for the first population is less than the one for the second population and the right-sided variant checks whether the standard deviation for the first population is greater than the one for the second population. The choice is determined by the problem.

The F-test checks the ratio of the variances of the two samples. The more this ratio deviates from 1, the more likely the null hypothesis will be rejected.

Hypothesis:

Test Statistic:

The F-test statistic has the following form:

where is the variance of the first sample and is the variance of the second sample.

As seen from this formula the F-test checks the ratio of the variances of the two samples . The more this ratio deviates from 1, the more likely the null hypothesis will be rejected.

Confidence Level:

Critical Region:

The null hypothesis is rejected if:

Two-sided test :

Left-sided test :

Right-sided test:

where are the quantiles of an F-distribution with degrees of freedom, is the first sample size and is the second sample size.

P-Value:

Under the null hypothesis the P-value is equal to the probability that a random variable with a Fisher-Snedecor F-distribution with degrees of freedom will be greater or equal to the calculated value of the test statistic

Data Requirements:

The F-test is performed for two numerical data samples which can have a standard or key-type representation.

The Kolmogorov-Smirnov (KS) test is based on the empirical distribution function (EDF).

Hypothesis: One-sample test

The following distributions are implemented: Normal, Exponential, Logistic distribution, Beta-distribution, Gamma-distribution.

Test Statistic:

The KS test statistic is defined as:

For the one-sample test:

where is the theoretical cumulative distribution of the distribution being tested, which must be a continuous distribution (i.e. no discrete distributions such as the binomial or Poisson) and fully specified (i.e. all distribution parameters should be specified, as in the test routine the distribution parameters cannot be estimated from the data), and is the EDF for the sample.

Note

Due to the properties of EDF the one sample test statistic is computed according to the following formula:

where is the sample size, are the ordered distinct observations, and it is assumed that .

The asymptotic statisic value is obtained from the formula

For the two-samples test:

where are the pulled ordered observations from the two samples, is the total sample size, is the first sample size, is the second sample size and and are EDF-s for the first and second sample accordingly.

The value of the asymptotic statisic is obtained from the equation

Confidence Level:

Critical Region:

The null hypothesis is rejected if the calculated sample value of the KS test statistic greater than the corresponding critical value of the distribution of the Kolmogorov-Smirnov test statistic. This distribution depends on sample size and confidence level

P-Value:

The P-value is equal to the probability of random variable with Kolmogorov-Smirnov distribution to be greater than or equal to the observed value of the KS test statistic under the null hypothesis. P-value is implemented only for Asymptotic Kolmogorov-Smirnov's distribution. c.d.f for this distribution is given by:

Data Requirements:

The Kolmogorov-Smirnov test can be performed for one or two data samples with a standard or key-type representation.

Kuiper's test is based on the empirical distribution function (EDF).

Hypothesis: One sample test

Two samples test:

The following distributions are implemented: Normal, Exponential, Logistic distribution, Beta distribution, Gamma distribution.

Test statistic:

The statistic for the one sided Kuiper's test is calculated according to the formulas:

where are the ordered distinct observations, is the sample size, is the theoretical cumulative distribution being tested and is the EDF for the sample.

The asymptotic statistic value is calculated as

The statistic for the two sided Kuiper's test is calculated according to the formulas:

where is the first sample size, is the second sample size, , and and are the EDF-s for the firs and second sample respectively.

The asymptotic statistic value for the two sided test is calculated as

Confidence level:

Critical region:

The null hypothesis is rejected if the value of the Kuipr's test statistic calculated for the given sample is greater than the corresponding critical value of the distribution of the Kuiper's test statistic. This distribution depends on sample size and confidence level.

P-value:

The P-value is equal to the probability of a random variable with Kuiper's distribution to be greater or equal to the observed value of Kuiper's test statistic under the null hypothesis. The P-value is implemented only for the asymptotic Kuiper's distribution. The CDF for distribution is given by the formula

.

Data requirements:

Kuiper's test can be performed for one or two data samples with a standard or key-type representation

There are three variants of Levene's test: mean-based, median-based and 10%-trimmed mean based. In each case the absolute deviations from the above values are used.

The originally proposed test used mean values of the sample. Brown and Forsythe, 1974 performed Monte Carlo simulations and showed that using the trimmed mean performed best when the underlying data followed a Cauchy distribution and the median performed best when the underlying data followed a chi-square distribution. Using the mean provided the best power for symmetric distributions without heavy tails. Although the optimal choice of the test variant depends on the underlying distribution, the variant based on the median is recommended as the choice that provides good robustness against many types of non-normal data while retaining good power. Knowledge of the underlying distribution of the data may suggest using one of the other variants.

Hypothesis:

Test Statistic:

Levene's test statistic is calculated as follows:

with calculated as follows:

1. For the mean-based test:

where denotes the mean value of the sample i.

2. For the median-based test:

where denotes the median value of the sample i.

3. For the 10%-trimmed mean based test:

where denotes the 10%-trimmed mean of the sample i.

In the above denotes the in-sample (over j) mean value of and denotes the overall mean of .

These three choices for calculating determine the robustness and power of Levene's test.

Confidence Level:

Critical Region:

The null hypothesis is rejected if the calculated value of the test statistic is greater than the corresponding critical value:

where is the upper critical value of an F-distribution with degrees of freedom.

P-Value:

The P-value is equal to probability of a random variable with an F-distribution with degrees of freedom to be greater than or equal to the observed value of the Levene's test statistic under the null hypothesis.

Data Requirements:

Levene's test is performed for two numerical data samples with a standard or key-type representation.

The t-test is the standard method for testing whether the population means for two non-paired samples are equal. If the populations are non-normal, particularly for small samples, then the t-test may not be valid. The Mann-Whitney test is an alternative that can be applied when the distributional assumptions are suspect. However, it is not as powerful as the t-test when the distributional assumptions are in fact valid. To perform the Mann-Whitney test the first step is to rank the combined samples. Then the sum of the ranks for sample one and the sum of the ranks for sample two are computed and these ranks are used for the calculation of the test statistic. For more details see for example Lehmann 1975.

Hypothesis:

Test Statistic:

To perform the Mann-Whitney test the first step is to rank the combined samples. Then the sum of the ranks for sample one ( ) is computed. The following statistics are calculated:

where and are the sample sizes.

Two-sided test :

If the sample sizes are equal, the Mann-Whitney test statistic is the minimum of the two values:

Left-sided test :

Right-sided test :

For sufficiently large samples (sample size greater than 20), the following normal approximation is used:

where

For small sample sizes it is necessary to use special tables of critical values.

Confidence Level:

Critical Region:

The null hypothesis is rejected if:

Two-sided test : the calculated value of the test statistic is greater than the critical value

One-sided test s: the calculated value of the test statistic is greater than the critical value , where is the standard normal distribution quantile.

P-Value:

The P-value is equal to the probability of a random variable with a standard normal distribution to be greater or equal to the observed value of the Mann-Whitney test statistic under the null hypothesis.

Data Requirements:

The Mann-Whitney test is performed for two numerical data samples with a standard or key-type representation.

# Mann-Whitney test example


table 'MannWhitney':
    mw1 mw2

    43   99
    97   98
    102  94
    101  86
    12   56
    97   103
    103  99
    104  89
    109  102
    59   104
    90   103
    98   96
    86   67
    92   90
    95   105
    101  79
    89   87

settings = MannWhitneyTestSettings()
test = MannWhitneyTest(settings)

res = runStatTest(test, 'MannWhitney', 'mw1', 'mw2')
res.printResults()

        StatTestResult: 

        1) Critical value:         1.9599639845400536        
        2) Statistic:        -0.08610882996604705
        3) p-value:        0.9313798362985003
        4) Hypothesis:        null hypotesis is accepted

The above example performs the Mann-Whitney test for the data samples 'mw1' and 'mw2' and checks whether these samples come from populations with equal means and at the default confidence level of 0.05 the test proves this assumption.

Pearson's product moment correlation is an example of a correlation coefficient. It is a measure of the linear association between two variables that have been measured on interval or ratio scales. However, one should remember that it can be misleadingly small when there is a relationship between the variables but it is not a linear one.

For two variables and Pearson's correlation coefficient is calculated as:

For two paired samples Pearson's sample correlation coefficient is calculated as:

where

and is the sample size.

Pearson's sample correlation coefficient has the following properties:

Hypothesis:

Test Statistic:

The Pearson's Test for the correlation coefficient is based on the statistic:

where is a sample Pearson's product moment correlation and is the sample size, which has the Student t-distribution with degrees of freedom.

Confidence Level:

Critical Region:

The null hypothesis is rejected if:

Two-sided test : the calculated absolute value of the test statistic is greater than the corresponding critical value:

and

Left-sided test : the calculated value of the test statistic is less than the corresponding critical value

Right-sided test : the calculated value of the test statistic is greater than the corresponding critical value

where are the upper quantiles of the Student t-distribution with degrees of freedom.

P-Value:

The P-value is equal to the probability of a random variable with a Student t-distribution with degrees of freedom to be greater or equal to the calculated value of the test statistic under the null hypothesis.

Data Requirements:

The Pearson's test is performed for two data samples of equal length with standard or key-type representation.

This example tests the proportion of good debtors Class in the german_credit dataset. Note that the categorical variable 'Class' needs to be transformed to a numerical binary variable before the test can be performed.

# prepare data sample for testing: add a binary
# numerical variable 'Class_num' based on the
# categorical variable "Class". 
trans 'german_credit_1' <- 'german_credit':
    if Class == 'good':
        Class_num = 1
    else:
        Class_num = 0

# prepare ProportionTestSettings and the test object
settings = ProportionTestSettings()
settings.setAlpha(0.05)
settings.setExpectedProportion(0.75)
settings.setDistinctValue(1)
settings.setSide(StatisticalTestsSettings.RIGHT_SIDED)

test = ProportionTest(settings)

# execute the test and print the results
res = runStatTest(test, 'german_credit_1', 'Class_num')
res.printResults()        
    

        StatTestResult: 

        1) Critical value:         1.6448536269514715        
        2) Statistic:        -3.6514837167011107
        3) p-value:        0.9998696351720902
        4) Hypothesis:        null hypotesis is rejected
    

The Sign test is used to test the hypothesis about the median of a population.

The sign test uses the following statistic:

Note that the value of this statistic depends only on the sign (positive or negative) of the difference between sample values and the tested median value. That is, the statistic simply counts the number of positive (or negative) signs among the above differences. If is too large, the null hypothesis will be rejected in favor of the alternative hypothesis. Since the observations are independent the properties of a binomial distribution are satisfied and therefore the statistic has a binomial distribution with the parameters {the number of sample elements} and p = 0.5. For samples with more than 10 elements the normal approximation for binomial distribution can be used since it performs reasonably well even for sample size equal to 10. For more details see for example Conover 1980, Hollander and Wolfe 1973.

Hypothesis:

Test Statistic:

For a sample , the Sign test statistic is calculated according to the following RULE:

is calculated for the two-sided test as

where

(here denotes the cardinality of the set )

For the left-side test

and for the right-sided test , where is the tested median value.

Confidence Level:

Critical Region:

The null hypothesis is rejected if:

Two-sided test : the absolute value of the calculated test statistic is greater than the corresponding critical value:

Left-sided test : the calculated test statistic is less than the corresponding critical value:

Right-sided test : the calculated test statistic is greater than the corresponding critical value:

Here denotes the upper percentile of the standard normal distribution.

P-Value:

The P-value is equal to the probability of a random variable with a standard normal distribution to be greater than or equal to the observed value of the Sign test statistic under the null hypothesis.

Data Requirements:

The Sign test can be performed on two numerical data samples with a standard or key-type representation.

Spearman's rank-order correlation is a nonparametric measure of dependency between two paired samples and is based on the ranks of the paired data. Spearman's test is used to check the significance of rank correlation coefficient.

For pairs of observations Spearman's rank-order correlation is calculated as

where

and

Here and are the ranks of pairs of observations. When there are no ties the above formula is reduced to

where is the difference between the pair ranks.

Hypothesis:

Test Statistic: Spearman's test statistic is defined as

The statistic has a Student t-distribution with degrees of freedom.

Confidence Level:

Critical Region:

The null hypothesis is rejected if:

Two-sided test : the absolute value of the test statistic is greater than the critical value; the critical value is equal to percentile of t-Student distribution with degrees of freedom;

Left-sided test : the value of the test statistic is less than the negative critical value;

Right-sided test : the value of the test statistic is greater than the critical value.

The critical value for the one-sided alternatives is equal to percentile of a Student t-distribution with degrees of freedom.

P-Value:

The P-value is equal to the probability that a random variable with a Student t-distribution with degrees of freedom will be greater than or equal to the calculated value of the test statistic.

Data Requirements:

Spearman's correlation test can be performed for two numerical data samples of equal length with a standard or key-type representation.

Student's t test for independent samples is used to determine whether two samples were drawn from populations with different means or whether the population mean is different from some predefined value. Student's t-test, often known simply as the t-test, is one of the most commonly used statistical tests. It comes in two versions: the paired t-test and the unpaired t-test. Both types are used to test the hypothesis that the mean values differ between the two populations, but the paired test is particularly useful when each observation in one sample corresponds to a matching observation in the other sample. The unpaired t-test is a more general technique that can be used to test whether the mean value differs between the two populations and does not require that the two samples be paired in any way or even have equal sizes.

For two unpaired samples if both of them are large enough, the separate or unequal variance version of the Student's t-test has many attractive features. The denominator of the test statistic correctly estimates the standard deviation of the numerator, while the Central Limit Theorem guarantees the validity of the test even if the populations are non-normal. "Large enough" sample sizes can be as small as 30 per sample if the two populations are nearly normally distributed. The more the populations depart from normality the larger the sample size needed for the Central Limit Theorem to support the test result. Note that 100 observations is often a quite sufficient sample size. For small and moderate sample sizes, the equal variances version of the test provides an exact test of the equality of the two population means. In this case the validity of the test demands that the samples be drawn from normally distributed populations with equal variances.

Hypothesis:

One-sample test :

Two-samples unpaired test :

Two-samples paired test :

Test Statistic:

For a sample Student's t-test statistic is calculated as:

where is the sample mean value and is the sample standard deviation, is the sample size.

For two paired samples the t-test statistic is calculated as:

where is the mean value of the differences and is the standard deviation of these differences, is the sample size.

For two unpaired samples under the unequal variances assumption the t-test statistic is calculated as:

where and are the mean values of the samples , and are the sample standard deviations, and

are the sample sizes.

For two unpaired samples under equal variances assumption the t-test statistic is calculated as:

where and are the mean values of the samples , and are the sample standard deviations, and are the sample sizes.

Confidence Level:

Critical Region:

The null hypothesis is rejected if:

Two-sided test : the absolute value of the test statistic is greater than the critical value; the critical value is equal to: percentile of Student's t-distribution with degrees of freedom;

Left-sided test : the value of the test statistic is less than the negative critical value;

Right-sided test : the value of the test statistic is greater than the critical value.

The critical value for the one-sided alternatives is equal to percentile of Student's t-distribution with degrees of freedom.

P-value:

The P-value is equal to the probability of a random variable with a Student's t-distribution with degrees of freedom to be greater or equal to the calculated absolute value of the test statistic under the null hypothesis.

Data Requirements:

The Student's t-test can be performed for one or two numerical data samples with a standard or sample-key representation.