{"id":29475,"date":"2022-06-30T08:56:36","date_gmt":"2022-06-30T08:56:36","guid":{"rendered":"https:\/\/www.askpython.com\/?p=29475"},"modified":"2023-02-16T19:56:42","modified_gmt":"2023-02-16T19:56:42","slug":"support-vector-machines-in-python","status":"publish","type":"post","link":"https:\/\/www.askpython.com\/python\/examples\/support-vector-machines-in-python","title":{"rendered":"Support Vector Machines in Python"},"content":{"rendered":"\n

When it comes to the implementation of Machine Learning algorithms, the list starts from linear regression to decision trees. They are of various types. Mainly when it comes to separations of data points along a linear axis the recommendations from the experts are: <\/p>\n\n\n\n

  1. K-means clustering.<\/strong><\/a><\/li>
  2. Support Vector Machines.<\/strong><\/strong><\/li><\/ol>\n\n\n\n

    As we all know that an ML model is of two types: <\/p>\n\n\n\n

    1. Supervised Learning<\/a>: Needs a guide to select the input data from the programmer.<\/li>
    2. Unsupervised Learning<\/a>: Needs no guide to select the input data. It’s a learn itself model.<\/li><\/ol>\n\n\n\n

      General Theory<\/h2>\n\n\n\n

      The main aim of this article is to make the reader aware of how the technique of SVM works. On the internet, data is available in raw. So, when we structure the data and visualize it, the results are either a discrete or continuous distribution<\/strong>. According to this, SVMs are used for two purposes:<\/p>\n\n\n\n

      1. Classification: For discrete data parameters.<\/em><\/li>
      2. Regression: For Continuous data parameters.<\/em><\/li><\/ol>\n\n\n\n

        This is one of the main reasons why Support Vector Machines are highly used for classification and regression purposes. The definition says that: Support Vector Machines are a set of learning algorithms that help us classify and analyze the nature of data.<\/strong> <\/p>\n\n\n\n

        Components of SVM<\/h4>\n\n\n\n
        1. Support vectors: These are the main components. They are simple data points that lie on both sides of the Maximum margin.<\/em><\/li>
        2. Maximum margin: The maximum limit till the data classification takes place.<\/em><\/li>
        3. Maximum margin hyperplane: The maximum mid-limit margin that lies between the positive and negative hyperplanes.<\/em><\/li>
        4. Positive hyperplane: Right side of the margin plane.<\/em><\/li>
        5. Negative hyperplane: Left side of the margin plane.<\/em><\/li><\/ol>\n\n\n\n

          Diagram<\/h4>\n\n\n\n
          \"SVM
          SVM Diagram Simplified <\/figcaption><\/figure>\n\n\n\n

          In this diagram, we can clearly see that the main margin is separating all the different data points according to the color. We have used the color for showing their nature of difference. The main aim of SVM is to show the distinction and classify each point with the best possible marginal line.<\/p>\n\n\n\n

          Example and applications<\/h4>\n\n\n\n

          Suppose we have a class: Vehicle<\/strong>. Our task is to fetch the Sports Utility Vehicle (SUV) from that class. <\/strong>Now there are various other types. Now, when we try to arrange them manually in order then it may take a lot of time. This also creates some errors. So, to make the classification more stable we can create a Support Vector Machine that will classify all the models of cars from the parent vehicle class. It shall work on the following steps: <\/p>\n\n\n\n

          1. The model will take a sample image.<\/li>
          2. Then it compares it with the test data of vehicle types provided already. <\/li>
          3. After that, it tells us which type of model of the car is there in that input image.<\/li>
          4. No other algorithm can make things simpler than an SVM.<\/li><\/ol>\n\n\n\n

            Implementing Support Vector Machines<\/h2>\n\n\n\n

            In this section, we shall implement all the necessary implementation for the Support Vector Machine. So, let’s get started!<\/p>\n\n\n\n

            Environment details:<\/h4>\n\n\n\n
            1. Python 3.9.7<\/em><\/li>
            2. IDE: Jupyter Notebooks<\/em><\/li>
            3. Environment: Anaconda 3<\/em><\/li>
            4. Dataset: Cancer dataset (cell_samples.csv)<\/em><\/li><\/ol>\n\n\n\n

              Importing the necessary libraries for data reading and preprocessing<\/p>\n\n\n

              \nimport pandas as pd\nimport numpy as np\nimport matplotlib.pyplot as plt\nfrom warnings import filterwarnings\nfilterwarnings("ignore")\n<\/pre><\/div>\n\n\n
              Reading the dataset<\/p>\n\n\n
              \ncancer_data = pd.read_csv("Datasets\/cell_samples.csv", sep = ",")\ncancer_data.head()\n<\/pre><\/div>\n\n\n
              Output:<\/strong><\/p>\n\n\n\n
              \"Viewing
              Viewing The Dataset<\/figcaption><\/figure>\n\n\n\n
              Checking for null values<\/p>\n\n\n
              \ncancer_Data.isna().sum()\n<\/pre><\/div>\n\n\n
              \"No
              No Null Values<\/figcaption><\/figure>\n\n\n\n
              Getting the general info about the dataset<\/p>\n\n\n
              \nprint("The shape of the dataset is: ", cancer_data.shape)\nprint("The size of the dataset is: ", cancer_data.size, " bytes\\n")\nprint("The count of each attribute of the dataset is: \\n")\nprint(cancer_data.count())\nprint("\\nThe datatype of each attribute is: \\n")\nprint(cancer_data.dtypes)\n<\/pre><\/div>\n\n\n
              Output:<\/strong><\/p>\n\n\n
              \nThe shape of the dataset is:  (699, 11)\nThe size of the dataset is:  7689  bytes\n\nThe count of each attribute of the dataset is: \n\nID             699\nClump          699\nUnifSize       699\nUnifShape      699\nMargAdh        699\nSingEpiSize    699\nBareNuc        699\nBlandChrom     699\nNormNucl       699\nMit            699\nClass          699\ndtype: int64\n\nThe datatype of each attribute is: \n\nID              int64\nClump           int64\nUnifSize        int64\nUnifShape       int64\nMargAdh         int64\nSingEpiSize     int64\nBareNuc        object\nBlandChrom      int64\nNormNucl        int64\nMit             int64\nClass           int64\ndtype: object\n<\/pre><\/div>\n\n\n
              Converting the BareNuc column into integer type<\/strong><\/p>\n\n\n
              \ncancer_data = cancer_data[pd.to_numeric(cancer_data["BareNuc"], errors = "coerce").notnull()]\ncancer_data["BareNuc"] = cancer_data["BareNuc"].astype("int")\ncancer_data.dtypes\n<\/pre><\/div>\n\n
              \nID             int64\nClump          int64\nUnifSize       int64\nUnifShape      int64\nMargAdh        int64\nSingEpiSize    int64\nBareNuc        int32\nBlandChrom     int64\nNormNucl       int64\nMit            int64\nClass          int64\ndtype: object\n<\/pre><\/div>\n\n\n
              Separating the two classes from the data frame<\/h4>\n\n\n\n
              For cancer cells type classification we have two types of cells for classification:<\/strong><\/p>\n\n\n\n
              1. Malignant: value = 4 in our dataset<\/li>
              2. Benign: value = 2 in our dataset<\/li><\/ol>\n\n\n\n
                We create two separate data frames of the same names. Then, try to classify them using data visualization techniques. Taking only the first fifty value from the core dataset. This makes plotting easier.<\/p>\n\n\n
                \nmalignant = cancer_data[cancer_data["Class"] == 4][0:50]\nbenign = cancer_data[cancer_data["Class"] == 2][0:50]\n<\/pre><\/div>\n\n
                \nplt.figure(figsize = (10, 5))\nax = plt.axes()\nax.set_facecolor("white")\nplt.title("Separating the data points - Clump and UniformShape")\nplt.scatter(malignant["Clump"], malignant["UnifShape"] , color = "red", marker = "*")\nplt.scatter(benign["Clump"], benign["UnifShape"], color = "green", marker = "+")\nplt.legend(["Malignant cell class", "Benign cell class"])\nplt.show()\n<\/pre><\/div>\n\n\n
                \"Scatter
                Scatter Plots<\/figcaption><\/figure>\n\n\n\n
                Creating independent and dependent data column lists with their numpy arrays:<\/p>\n\n\n
                \ndependent_data = cancer_data[["ID", "Class"]]\nindependent_data = cancer_data[['Clump', 'UnifSize', 'UnifShape', 'MargAdh', 'SingEpiSize',\n       'BareNuc', 'BlandChrom', 'NormNucl', 'Mit']]\n\nX_data = np.array(independent_data)\nX_data[0:5]\n\nY_data = np.array(dependent_data["Class"])\nY_data[0:5]\n<\/pre><\/div>\n\n\n
                Output:<\/strong><\/p>\n\n\n
                \narray([[ 5,  1,  1,  1,  2,  1,  3,  1,  1],\n       [ 5,  4,  4,  5,  7, 10,  3,  2,  1],\n       [ 3,  1,  1,  1,  2,  2,  3,  1,  1],\n       [ 6,  8,  8,  1,  3,  4,  3,  7,  1],\n       [ 4,  1,  1,  3,  2,  1,  3,  1,  1]], dtype=int64)\n\narray([2, 2, 2, 2, 2], dtype=int64)\n<\/pre><\/div>\n\n\n
                Splitting the data into train and test variables<\/h4>\n\n\n\n
                From the sklearn.model_selection import the train_test_split function<\/a>. This splits the data into four arrays:<\/p>\n\n\n\n
                1. X_train<\/li>
                2. X_test<\/li>
                3. y_train<\/li>
                4. y_test<\/li><\/ol>\n\n\n\n
                  Out of these the training arrays are tow dimensional and the testing arrays are one dimensional. Just remember to take the test_size = 0.2 as we need only 20 percent of the total dataset to test our model accuracy.<\/p>\n\n\n
                  \nfrom sklearn.model_selection import train_test_split\nX_train, X_test, y_train, y_test = train_test_split(X_data, Y_data, test_size = 0.2, random_state = 4)\nprint(X_train.shape)\nprint(y_train.shape)\nprint(X_test.shape)\nprint(y_test.shape)\n<\/pre><\/div>\n\n\n
                  Output:<\/strong><\/p>\n\n\n
                  \n(546, 9)\n(546,)\n(137, 9)\n(137,)\n<\/pre><\/div>\n\n\n
                  Importing the SVM from sklearn and creating a classifier instance<\/h4>\n\n\n\n
                  First we import the model and then we import SVC. It is the classifier class for separating the support vectors. Create an instance “Classify”. Give the kernel value as “linear” it will linearly separate the support vectors. Then we fit the X_train data and Y_train data inside the model using the fit() function. After that create an instance “y_predict”, which holds all the predictions in a one-dimensional array.<\/p>\n\n\n
                  \nfrom sklearn import svm\nclassify = svm.SVC(kernel = "linear")\nClassify.fit(X_train, y_train)\ny_predict = Classify.predict(X_test)\nprint(y_predict)\n<\/pre><\/div>\n\n\n
                  Output:<\/p>\n\n\n
                  \narray([2, 4, 2, 4, 2, 2, 2, 2, 4, 2, 2, 4, 4, 4, 4, 2, 2, 2, 2, 2, 4, 2,\n       4, 4, 4, 4, 2, 2, 4, 4, 4, 2, 4, 2, 2, 2, 4, 2, 2, 2, 2, 2, 2, 4,\n       4, 2, 2, 2, 2, 4, 2, 2, 2, 2, 2, 2, 4, 2, 2, 2, 2, 4, 4, 2, 4, 4,\n       4, 2, 2, 2, 4, 4, 2, 2, 2, 4, 2, 2, 4, 4, 2, 2, 2, 2, 4, 4, 2, 4,\n       2, 2, 4, 4, 2, 2, 2, 4, 2, 2, 2, 4, 2, 4, 2, 2, 4, 2, 4, 2, 2, 4,\n       2, 2, 4, 2, 2, 2, 2, 2, 4, 4, 4, 4, 4, 2, 2, 4, 2, 2, 4, 2, 4, 2,\n       2, 2, 2, 2, 4], dtype=int64)\n<\/pre><\/div>\n\n\n
                  So, we have successfully separated all the cancerous patients with the noncancerous ones. The cells having 4 as value are cancerous and with that 2 are noncancerous. Now, that we have got the predictions we can run them against our Y_test array to check how accurate the model is. For that we can prepare a classification report.<\/p>\n\n\n\n
                  Preparing the classification report<\/h4>\n\n\n\n
                  For this, we need to import the classification_report function from the sklearn.metrics module. Then call it inside the print() function. we test it with our Y_test array and the results are as follows:<\/p>\n\n\n
                  \nfrom sklearn.metrics import classification_report\nprint(classification_report(y_test, y_predict))\n<\/pre><\/div>\n\n\n
                  Output:<\/strong><\/p>\n\n\n
                  \n                   precision    recall  f1-score   support\n\n           2       1.00           0.94      0.97        90\n           4       0.90           1.00      0.95        47\n\naccuracy                           0.96       137\nmacro avg       0.95      0.97      0.96       137\nweighted avg       0.97      0.96      0.96       137\n\n<\/pre><\/div>\n\n\n
                  As the result says the precision of the model is very good. For malignant class (value  = 4)<\/strong>: The precision score is: 100%. For the benign class (value = 2)<\/strong> the precision score is: 90%<\/p>\n\n\n\n
                  Conclusion<\/h2>\n\n\n\n
                  So, in this way we have successfully implemented the Support Vector Machines using Python and built a predictive model from the given input data.<\/p>\n","protected":false},"excerpt":{"rendered":"
                  When it comes to the implementation of Machine Learning algorithms, the list starts from linear regression to decision trees. They are of various types. Mainly when it comes to separations of data points along a linear axis the recommendations from the experts are: K-means clustering. Support Vector Machines. As we all know that an ML […]<\/p>\n","protected":false},"author":36,"featured_media":29801,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[9],"tags":[],"class_list":["post-29475","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-examples"],"blocksy_meta":[],"_links":{"self":[{"href":"https:\/\/www.askpython.com\/wp-json\/wp\/v2\/posts\/29475","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.askpython.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.askpython.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.askpython.com\/wp-json\/wp\/v2\/users\/36"}],"replies":[{"embeddable":true,"href":"https:\/\/www.askpython.com\/wp-json\/wp\/v2\/comments?post=29475"}],"version-history":[{"count":0,"href":"https:\/\/www.askpython.com\/wp-json\/wp\/v2\/posts\/29475\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.askpython.com\/wp-json\/wp\/v2\/media\/29801"}],"wp:attachment":[{"href":"https:\/\/www.askpython.com\/wp-json\/wp\/v2\/media?parent=29475"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.askpython.com\/wp-json\/wp\/v2\/categories?post=29475"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.askpython.com\/wp-json\/wp\/v2\/tags?post=29475"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}