How is the optimal hyperplane computed?

Let’s introduce the notation used to define formally a hyperplane:

where is known as the weight vector and as the bias.

The optimal hyperplane can be represented in an infinite number of different ways by scaling of and . As a matter of convention, among all the possible representations of the hyperplane, the one chosen is

where symbolizes the training examples closest to the hyperplane. In general, the training examples that are closest to the hyperplane are called support vectors. This representation is known as the canonical hyperplane.

Now, we use the result of geometry that gives the distance between a point and a hyperplane :

In particular, for the canonical hyperplane, the numerator is equal to one and the distance to the support vectors is

Recall that the margin introduced in the previous section, here denoted as , is twice the distance to the closest examples:

Finally, the problem of maximizing is equivalent to the problem of minimizing a function subject to some constraints. The constraints model the requirement for the hyperplane to classify correctly all the training examples . Formally,

where represents each of the labels of the training examples.

This is a problem of Lagrangian optimization that can be solved using Lagrange multipliers to obtain the weight vector and the bias of the optimal hyperplane.