In the context of machine learning, particularly in Support Vector Machines (SVM), support vectors are the data points that lie closest to the decision boundary or hyperplane that separates different classes. These points are crucial because they directly influence the position and orientation of the hyperplane. If these support vectors were removed, the optimal hyperplane could change, affecting the classification of other data points.
Support vectors can be thought of as the "critical" elements of the training dataset; they are the only points that matter for defining the margin, which is the distance between the hyperplane and the nearest data points from either class. Mathematically, an SVM aims to maximize this margin, which can be expressed as:
where is the weight vector orthogonal to the hyperplane. Thus, support vectors play a vital role in ensuring the robustness and accuracy of the classifier.
Start your personalized study experience with acemate today. Sign up for free and find summaries and mock exams for your university.