Statistical Inference: Hypothesis Testing

Hypothesis testing is a formal statistical process used to make decisions about a population parameter (such as the mean $\mu$ or proportion $p$) based on sample data. It involves weighing evidence to decide whether to reject a null hypothesis in favour of an alternative hypothesis.

1. The Logic of Hypothesis Testing

Hypothesis testing is analogous to a court trial:
* The Null Hypothesis ($H_0$): The “status quo” or the assumption of “no effect.” In a trial, this is the “presumption of innocence.” We assume $H_0$ is true unless evidence suggests otherwise.
* The Alternative Hypothesis ($H_1$): What the researcher is trying to prove (the “guilty” verdict).
* The Test Statistic: A single value calculated from sample data (e.g., the sample mean $\bar{x}$ or sample proportion $\hat{p}$) used to determine how far the sample result deviates from the null hypothesis.

KEY TAKEAWAY: In VCE Specialist Mathematics, the null hypothesis $H_0$ always involves an equality (e.g., $\mu = \mu_0$ or $p = p_0$), whereas the alternative hypothesis $H_1$ involves an inequality ($<$, $>$, or $\neq$).

2. Setting Up Hypotheses

Hypotheses must be defined before collecting data. They can be one-tailed (directional) or two-tailed (non-directional).

For Population Means ($\mu$)

For Population Proportions ($p$)

EXAM TIP: When writing hypotheses, always define the parameter in words. For example: “where $\mu$ is the mean heart rate of participants in the dark.”

3. The Test Statistic ($z$)

To determine the likelihood of our sample result, we calculate a z-score, which measures how many standard deviations the sample statistic is from the hypothesised population parameter.

For Means (Normal distribution or large $n$)

If the population standard deviation $\sigma$ is known:
$$Z = \frac{\bar{x} - \mu_0}{\sigma / \sqrt{n}}$$

For Proportions (Large $n$)

$$Z = \frac{\hat{p} - p_0}{\sqrt{\frac{p_0(1-p_0)}{n}}}$$

REMEMBER: The denominator represents the standard error of the sampling distribution. For proportions, we use the value of $p$ from the null hypothesis ($p_0$) to calculate the standard error.

4. The $p$-value

The $p$-value is the probability of obtaining a sample statistic as extreme as, or more extreme than, the one observed, assuming the null hypothesis is true.

Calculating $p$-values

Let $Z_{obs}$ be the calculated test statistic from the sample data.

• For $H_1: \mu > \mu_0$ (Upper tail): $p\text{-value} = \Pr(Z > Z_{obs})$
• For $H_1: \mu < \mu_0$ (Lower tail): $p\text{-value} = \Pr(Z < Z_{obs})$
• For $H_1: \mu \neq \mu_0$ (Two-tail): $p\text{-value} = 2 \times \Pr(Z > |Z_{obs}|)$

Interpreting $p$-values

• A small $p$-value (typically $< 0.05$) indicates that the observed data is unlikely to have occurred by chance under $H_0$, providing evidence against $H_0$.
• A large $p$-value indicates that the observed data is consistent with $H_0$.

COMMON MISTAKE: Students often forget to double the $p$-value for a two-tailed test. If $H_1$ uses $\neq$, you must account for extremes in both directions.

5. Significance Levels ($\alpha$) and Decision Rules

The significance level ($\alpha$) is a pre-determined threshold used to decide whether the $p$-value is small enough to reject $H_0$. Common levels are $0.05$ (5%) and $0.01$ (1%).

The Decision Rule

1. If $p\text{-value} < \alpha$: Reject $H_0$. There is statistically significant evidence to support $H_1$.
2. If $p\text{-value} \ge \alpha$: Do not reject $H_0$ (Fail to reject $H_0$). There is insufficient evidence to support $H_1$.

Factors Affecting the $p$-value

The $p$-value will decrease (making it more likely to reject $H_0$) if:
* The sample size $n$ increases.
* The difference between the sample mean $\bar{x}$ and the hypothesised mean $\mu_0$ increases.
* The population standard deviation $\sigma$ (or variance $\sigma^2$) decreases.

VCAA FOCUS: You must be able to state the conclusion in the context of the original problem. Avoid saying “H0 is true”; instead, say “There is insufficient evidence at the $\alpha$ level of significance to suggest that [contextual claim]…”

6. Type I and Type II Errors

Errors can occur because we are making a decision about a population based only on a sample.

• Type I Error: A “false positive” (e.g., convicting an innocent person).
• Type II Error: A “false negative” (e.g., setting a guilty person free).

APPLICATION: In medical testing, a Type II error might mean failing to detect a disease in a sick patient, while a Type I error might mean telling a healthy patient they are sick. The choice of $\alpha$ often depends on which error is more dangerous.