Support Vector Machines have an elegant geometric interpretation that is central to VCAA questions on this topic.
In 2D, the decision boundary is a straight line separating two classes:
$$w_1 x_1 + w_2 x_2 + b = 0$$
The two margin boundaries are parallel lines:
$$w_1 x_1 + w_2 x_2 + b = +1 \quad \text{(positive margin)}$$
$$w_1 x_1 + w_2 x_2 + b = -1 \quad \text{(negative margin)}$$
The margin width is $\frac{2}{\sqrt{w_1^2 + w_2^2}} = \frac{2}{|\mathbf{w}|}$.
The decision boundary is the line equidistant between the two margin boundaries.
KEY TAKEAWAY: In 2D, the SVM decision boundary is the line with maximum perpendicular distance to the nearest points of each class. Support vectors lie on the margin boundaries.
In 1D, the decision boundary is a single threshold point on the number line.
Example:
- Class $-1$: ${-3, -1, 0}$
- Class $+1$: ${2, 4, 5}$
Closest pair across classes: $0$ (class $-1$) and $2$ (class $+1$).
Classification rule: predict $+1$ if $x > 1$; predict $-1$ if $x \leq 1$.
| Element | Geometric meaning |
|---|---|
| Decision boundary | Line/point equidistant from margin boundaries |
| Margin boundaries | Lines/points through support vectors |
| Support vectors | Training points closest to the decision boundary |
| Margin width | Perpendicular distance between the two margin boundaries |
Some 1D data is not linearly separable:
No single threshold separates these classes. This motivates feature creation (mapping to 2D using $x_2 = x_1^2$).
| Dimension | Decision boundary type |
|---|---|
| 1D | Threshold point |
| 2D | Line |
| 3D | Plane |
| $n$D | Hyperplane |
EXAM TIP: For VCAA exams, you will typically be given a small 1D or 2D dataset. To find the SVM decision boundary: (1) identify the two support vectors (one from each class, closest together), (2) draw margin boundaries through them, (3) place the decision boundary midway.
COMMON MISTAKE: The decision boundary does NOT pass through the support vectors. It runs parallel to and midway between the two margin boundaries.
VCAA FOCUS: Be able to geometrically determine the SVM decision boundary and margin for 1D and simple 2D datasets. Identify which points are support vectors.
