# Baumgartner Weiss Schindler Test of Equal Distributions

Performs the 'Baumgartner-Weiss-Schindler' two-sample test of equal probability distributions, . Also performs similar rank-based tests for equal probability distributions due to Neuhauser and Murakami .

Performs the Baumgartner-Weiß-Schindler 2-sample test of equal probability distributions.

-- Steven E. Pav, [email protected]

## Installation

This package can be installed from CRAN, via drat, or from github:

# Basic Usage

The front end for the hypothesis test is the function `bws_test`. By default it supports the the classical Baumgartner-Weiß-Schindler test for a two-sided alternative, returning a `htest` object:

``````##
## 	two-sample BWS test
##
## data:  x vs. y
## B = 1, p-value = 0.2
## alternative hypothesis: true difference in survival functions is not equal to 0
``````
``````##
## 	two-sample BWS test
##
## data:  x vs. z
## B = 30, p-value <2e-16
## alternative hypothesis: true difference in survival functions is not equal to 0
``````

The code also supports alternative test statistics from Neuhäuser and Murakami, along with supporting one-sided alternatives for some of the tests:

``````##
## 	two-sample Neuhauser/Murakami test
##
## data:  x vs. z
## B_2 = -30, p-value <2e-16
## alternative hypothesis: true difference in survival functions is less than 0
``````
``````##
## 	two-sample Murakami test
##
## data:  x vs. z
## B_3 = 2, p-value = 0.009
## alternative hypothesis: true difference in survival functions is not equal to 0
``````

We should note that the `B3` through `B5` tests do not achieve nominal coverage for large sample sizes and should only be used on sample sizes of about 12 or fewer in each of the two samples.

## Checking the null

Doverai No Proverai (Trust, but verify.) -- Russian proverb.

Here we perform 5000 simulations of the BWS test under the null hypothesis, then compute the CDF of the test statistic. If the code is correct, the resultant p-values should be uniform. So I q-q plot under the uniform law:

Looks good to me!

## Under the alternative

Here we replicate figure 2A of Baumgartner et al. We draw two samples from the normal distribution, both with unit standard deviation, letting a be the difference in means. We check the empirical rejection rate at the 0.05 level for a few different tests. As in Baumgartner, we find that the lowly t-test is the most powerful in this case, with the BWS, Cramer-Von Mises, and Wilcoxon tests displaying similar power, then the KS test the least powerful. Note that the Kolmogorov Smirnov test does not appear to have nominal coverage under the null, probably due to the small sample size.

Here we replicate figure 2B of Baumgartner et al. We draw two samples from the normal distribution, both with zero mean, one with unit standard deviation, the other with standard deviation of sigma. We compute the empirical rejection rate at the 0.05 level, dropping the t-test since it is not relevant for this formulation. As in Baumgartner, we find the BWS test is the most powerful, followed by KS test, then Cramer-Von Mises, then Wilcoxon, which is basically useless in this simulation.

Here we replicate figure 3A of Baumgartner et al. We draw two samples from the exponential distribution, letting l be the ratio of the rate parameters of the two populations. We compute the empirical rejection rate at the 0.05 level. As in Baumgartner, we find the BWS test is the most powerful, followed by Wilcoxon, then Cramer-Von Mises, then the KS test.

Here we replicate figure 3B of Baumgartner et al. We draw two samples, one from the normal distribution with zero mean and variance one-twelth, the other uniformly on -0.5 to 0.5. We take equal sample sizes from these two populations, then vary the sample size, checking the empirical rejection rate at the 0.05 level. Since the first two moments are equal, the Wilcoxon test is useless here, and not applied. As in Baumgartner, we find the BWS test is the most powerful, followed by the KS test and Cramer-Von Mises tests. Based on the power plots here, I theorize that Baumgartner et al. are plotting the total sample sizes on the x axis, that is, drawing n from both distributions, then plotting empirical power versus 2n. We follow that convention, which makes the plots match those of Baumgartner.

## Murakami tests

Neuhäuser and Murakami described some modifications to the original test of Baumgartner. Neuhäuser's test allows one to test against directional alternatives. Murakami enumerated some modifications to the weighting scheme. These are available via the `murakami_stat` function, where the `flavor` corresponds to the test number from Murakami's paper, namely

• 0 corresponds to the original BWS statistic.
• 1 corresponds to Murakami's first modification.
• 2 corresponds to Neuhäuser's statistic, which can take negative values.
• 3 through 5 correspond to different weighting schemes of Murakami's.

Here we take these through the paces as above.

### Under the null

As above, we draw samples under the null and compare to the putative CDF function

While these all look fine, they are based on small sample sizes. The CDF is approximated by evaluating all the permutations (with memoisation to tame the computational requirements), but this can only be done up to some reasonably small sample size. If the test statistic does not converge beyond that sample size, the CDF approximation will not be accurate. This appears to be the case for flavors 3 through 5, as demonstrated below:

So that's not so great.

## Under the alternative

We can perform the power comparisons of Baumgartner et al. using the Murakami test statistics. Here we consider the setup of figure 2A, samples of size 10 from the normal distribution with equal variances but different means. Neuhäuser's test is, as expected, more powerful for detecting this one-sided alternative, probably on par with the t-test, while the BWS test (basically equivalent to Murakami 1) is next, then 5, 3, 4.

Now we consider figure2B, with two samples of equal size from the normal distribution with zero mean, but different standard deviations. Neuhäuser's test is nearly useless here because the means of the populations are equal. Murakami 3, 4, and 5 are the most powerful, followed by BWS.

Now to figure 3A, with two equal sized samples from the exponential distribution, letting l be the ratio of the rate parameters of the two populations. The BWS test is the most powerful here, apparently.

Now figure 3B, with a normal sample and a uniform sample, with equal first and second moments, varying the sample size. Because flavors 3 through 5 are not well behaved, we drop them here. Once again the BWS test is the most powerful.

# Future work

I anticipate the following:

• Modifications of flavors 3 through 5 so that they converge for large sample sizes.

# Reference manual

install.packages("BWStest")

0.2.2 by Steven E. Pav, 3 years ago

https://github.com/shabbychef/BWStest

Report a bug at https://github.com/shabbychef/BWStest/issues

Browse source code at https://github.com/cran/BWStest

Authors: Steven E. Pav [aut, cre]

Documentation:   PDF Manual