3.1 Fisher’s exact test

Let us denote a generic two-by-two contingency table under the null hypothesis using random variables W₁ and W₀ such as in Table 3. Here, the null hypothesis for Fisher’s exact test is as follows:

H₀: Y(1) = Y(0) for all subjects,

which is referred to as the sharp causal null hypothesis [12].

Under this null hypothesis, subjects are limited to those with (Y(1), Y(0)) = (0, 0) or (1, 1), which implies that an outcome for a subject is constant regardless of the assigned group. Then, subjects with Y = 1 are those with(Y(1), Y(0)) = (1, 1), and similarly subjects with Y = 0 are those with(Y(1), Y(0)) = (0, 0). Therefore, under the sharp causal null hypothesis, w₀ + w₁ = a + c and n–w₀–w₁ = b + d from Tables 1 and 3.

The probability of W₁ = w₁ under the sharp causal null hypothesis is given by the hypergeometric distribution as follows:

$p_{w_1}=\left(\begin{array}{c}a+c\\ w_1\end{array}\right)\left(\begin{array}{c}b+d\\ a+b-w_1\end{array}\right)/\left(\begin{array}{c}n\\ a+b\end{array}\right)$

where $\left(\begin{array}{c}j\\ k\end{array}\right)={}_{j}C{}_k=\frac{j!}{k!(j-k)!}$ and max {0, a–d} ≤ w₁≤ min {a + b, a + c}. This is the probability that w₁ subjects of (a + c) subjects who experienced the event, and (a + b–w₁) subjects of (b + d) subjects who did not experience the event are selected in the treatment group, when (a + b) subjects of the total n subjects are selected in the treatment group.

Table 3: Generic two-by-two contingency table under the null hypothesis with realized values of random variables W₀ and W₁.

Because the risk difference under the null hypothesis can be expressed as

${\text{RD}}_{\text{N}}:=\frac{w_1}{a+b}-\frac{w_0}{c+d}$

the one-sided p-value can be calculated by

$\begin{array}{c}p={\displaystyle \sum _{w_1=\max \{0,a-d\}}^{\min (a+b,a+c)}I(z)\left(\begin{array}{c}a+c\\ w_1\end{array}\right)\left(\begin{array}{c}b+d\\ a+b-w_1\end{array}\right)/\left(\begin{array}{c}n\\ a+b\end{array}\right)}\\ ={\displaystyle \sum _{w_1=a}^{\min (a+b,a+c)}\left(\begin{array}{c}a+c\\ w_1\end{array}\right)\left(\begin{array}{c}b+d\\ a+b-w_1\end{array}\right)/\left(\begin{array}{c}n\\ a+b\end{array}\right)},\end{array}$

where I(z) := RD_N– RD_O = 1 if z≥ 0 and I(z) = 0 if z< 0 with z = RD_N– RD_O. The second equation is derived because w₀ = a + c–w₁ and thus

$\begin{array}{c}{\text{RD}}_{\text{N}}-{\text{RD}}_{\text{O}}=\left(\frac{w_1}{a+b}-\frac{w_0}{c+d}\right)-\left(\frac{a}{a+b}-\frac{c}{c+d}\right)\\ =\left(\frac{1}{a+b}+\frac{1}{c+d}\right)(w_1-a).\end{array}$

This is the p-value for Fisher’s exact test. For the hypothetical data listed in Table 2, we have

$p={\displaystyle \sum _{w_1=3}^4\left(\begin{array}{c}4\\ w_1\end{array}\right)\left(\begin{array}{c}6\\ 5-w_1\end{array}\right)/\left(\begin{array}{c}10\\ 5\end{array}\right)}=0.2619$

3.2 Barnard’s exact test

Let the response probability for the group with X = x be Pr(Y = 1 | X = x) = π_x. Then, the probability of observing w₁ and w₀ is given by the following product of two binomial probabilities:

$p_{{\pi }_1,{\pi }_0}=\left(\begin{array}{c}a+b\\ w_1\end{array}\right){\pi }_1^{w_1}{(1-{\pi }_1)}^{a+b-w_1}\left(\begin{array}{c}c+d\\ w_0\end{array}\right){\pi }_0^{w_0}{(1-{\pi }_0)}^{c+d-w_0}$

The null hypothesis for Barnard’s exact test is as follows:

H₀: π₁ = π₀.

Therefore, for π₁ = π₀ = π, the one-sided p-value can be calculated by

$p_{\pi }={\displaystyle \sum _{w_1=0}^{a+b}{\displaystyle \sum _{w_0=0}^{c+d}I(z)\left(\begin{array}{c}a+b\\ w_1\end{array}\right)\left(\begin{array}{c}c+d\\ w_0\end{array}\right){\pi }^{w_0+w_1}{(1-\pi )}^{n-w_0-w_1}}}$

Unfortunately, because this calculation of the p-value includes a nuisance parameter π, we cannot yield the p-value immediately. Thus, we yield the p-value by calculating the p-values for all possible π and choosing the maximum value; i.e., p = sup{p_π}. This is the p-value for Barnard’s exact test. For the hypothetical data in Table 2, we have

$p=p_{0.4950}={\displaystyle \sum _{w_1=0}^5{\displaystyle \sum _{w_0=0}^{5}I(z)\left(\begin{array}{c}5\\ w_1\end{array}\right)\left(\begin{array}{c}5\\ w_0\end{array}\right){0.4950}^{w_0+w_1}{(1-0.4950)}^{10-w_0-w_1}}}=\mathrm{0.1719.}$

An SAS program to yield the p-values for Fisher’s and Barnard’s exact tests is given in the appendix.

3.3 Chiba’s exact test

Let n_st denote the number of subjects with (Y(1), Y(0)) = (s, t), where s, t = 0, 1. If all subjects are assigned to the treatment group (X = 1), then Pr(Y(1) = 1) = (n₁₁ + n₁₀) / n, because only subjects with type (i) or (ii) would experience the event. Likewise, if all subjects are assigned to the control group (X = 0), then Pr(Y(0) = 1) = (n₁₁ + n₀₁) /n, because only subjects with type (i) or (iii) would experience the event.

The null hypothesis for Chiba’s exact test is as follows:

H₀: n₁₀ = n₀₁.

This null hypothesis corresponds to Pr(Y(1) = 1) = Pr(Y(0) = 1), which is referred to as the weak causal null hypothesis [12].

Let us assume that of the n_st subjects, n_st,1 subjects were assigned to the treatment group (X = 1), and n_st,0 subjects were assigned to the control group (X = 0) by random assignment at a 1:r ratio. This leads to the two-by-two contingency table shown in Table 4. Applying the binomial probabilities, the one-sided p-value can be calculated by

$p_{n_{11},n_{10},n_{01},n_{00}}={\displaystyle \sum _{n_{11,1}=0}^{n_{11}}{\displaystyle \sum _{n_{10,1}=0}^{n_{10}}{\displaystyle \sum _{n_{01,1}=0}^{n_{01}}{\displaystyle \sum _{n_{00,1}=0}^{n_{00}}{I}^{\prime }(z){\displaystyle \prod _{s=0}^1{\displaystyle \prod _{t=0}^1\left(\begin{array}{c}{n}_{st}\\ n_{st,1}\end{array}\right){\left(\frac{1}{1+r}\right)}^{n_{st,1}}{\left(\frac{r}{1+r}\right)}^{n_{st,0}}}}}}}}$ (1)

where I'(Z)=1 if z≥ 0 and I'(Z)=0 if z< 0 with , z=RD'_N- RD_O and

${\text{RD′}}_{\text{N}}:=\frac{n_{11,1}+n_{10,1}}{n_{11,1}+n_{10,1}+n_{01,1}+n_{00,1}}-\frac{n_{11,0}+n_{01,0}}{n_{11,0}+n_{10,0}+n_{01,0}+n_{00,0}}$

where n_st (s, t = 0, 1) must satisfy the following conditions:

$n_{10}=n_{01},{\displaystyle \sum _{s=0}^1{\displaystyle \sum _{t=0}^1{n}_{st}}}=n,\{\begin{array}{c}{n}_{11}\le a+c\\ n_{10}\le a+d\\ n_{01}\le b+c\\ n_{00}\le b+d\end{array}and\{\begin{array}{c}a\le n_{11}+n_{10}\le n-b\\ c\le n_{11}+n_{01}\le n-d\\ d\le n_{00}+n_{10}\le n-c\\ b\le n_{00}+n_{01}\le n-a\end{array}$ (2)

Table 4: Generic two-by-two contingency table with the numbers for the four types of subjects defining the four principal strata.

Similar to Barnard’s exact test, because this calculation of the p-value includes the nuisance parameters n₁₁, n₁₀, n₀₁ and n₀₀, we cannot yield the p-value directly. Thus, we yield the p-value by calculating the p-values for all possible combinations of (n₁₁, n₁₀, n₀₁ and n₀₀) and choosing the maximum value; i.e., p = sup{ pn₁₁,n₁₀,n₀₁,n₀₀}. This is the p-value for Chiba’s unconditional exact test. For the hypothetical data in Table 2, we have

$p=p_{4,0,0,6}={\displaystyle \sum _{n_{11,1}=3}^4\left(\begin{array}{c}4\\ n_{11,1}\end{array}\right)\left(\begin{array}{c}6\\ 5-n_{11,1}\end{array}\right){\left(\frac{1}{2}\right)}^{10}}=0.1592$

For Chiba’s conditional exact test,

$p_{n_{11},n_{10},n_{01},n_{00}}={\displaystyle \sum _{n_{11,1}=0}^{n_{11}}{\displaystyle \sum _{n_{10,1}=0}^{n_{10}}{\displaystyle \sum _{n_{01,1}=0}^{n_{01}}{\displaystyle \sum _{n_{00,1}=0}^{n_{00}}I(z){\displaystyle \prod _{s=0}^1{\displaystyle \prod _{t=0}^1\left(\begin{array}{c}{n}_{st}\\ n_{st,1}\end{array}\right)}}}}}}/\left(\begin{array}{c}n\\ a+b\end{array}\right)$

is applied rather than (1), and Σ_sΣ_tn_st,1 = a + b is added to the conditions in (2). As with Fisher’s exact test, the calculation of this p-value is based on the probability that n_st,1 subjects of n_st subjects are selected in the treatment group when (a + b) subjects of the total n subjects are selected in the treatment group. Note that the special case of n₁₀ = n₀₁ = 0, for which subjects are limited to those with (Y(1), Y(0)) = (0, 0) or (1, 1), corresponds to Fisher’s exact test. For the hypothetical data listed in Table 2, we have

$p=p_{4,0,0,6}={\displaystyle \sum _{n_{11,1}=3}^4\left(\begin{array}{c}4\\ n_{11,1}\end{array}\right)\left(\begin{array}{c}6\\ 5-n_{11,1}\end{array}\right)}/\left(\begin{array}{c}10\\ 5\end{array}\right)=0.2619$