{"id":8788,"date":"2020-09-21T21:19:19","date_gmt":"2020-09-21T21:19:19","guid":{"rendered":"https:\/\/www.askpython.com\/?p=8788"},"modified":"2020-09-21T21:19:22","modified_gmt":"2020-09-21T21:19:22","slug":"random-forest-regression","status":"publish","type":"post","link":"https:\/\/www.askpython.com\/python\/examples\/random-forest-regression","title":{"rendered":"Random Forest Regression: A Complete Reference"},"content":{"rendered":"\n

Welcome to this article on Random Forest Regression. Let me quickly walk you through the meaning of regression first.<\/p>\n\n\n\n

What is Regression in Machine Learning?<\/h2>\n\n\n\n

Regression is a machine learning technique that is used to predict values across a certain range. Let us see understand this concept with an example, consider the salaries of employees and their experience in years. <\/p>\n\n\n\n

A regression model on this data can help in predicting the salary of an employee even if that year is not having a corresponding salary in the dataset.<\/p>\n\n\n\n

What is Random Forest Regression?<\/h2>\n\n\n\n

Random forest regression is an ensemble learning technique. But what is ensemble learning?<\/p>\n\n\n\n

In ensemble learning, you take multiple algorithms or same algorithm multiple times and put together a model that\u2019s more powerful than the original.<\/p>\n\n\n\n

Prediction based on the trees is more accurate because it takes into account many predictions. This is because of the average value used. These algorithms are more stable because any changes in dataset can impact one tree but not the forest of trees. <\/p>\n\n\n\n

Steps to perform the random forest regression<\/h2>\n\n\n\n

This is a four step process and our steps are as follows:<\/p>\n\n\n\n

  1. Pick a random K data points from the training set.<\/li>
  2. Build the decision tree associated to these K data points.<\/li>
  3. Choose the number N tree of trees you want to build and repeat steps 1 and 2.<\/li>
  4. For a new data point, make each one of your Ntree trees predict the value of Y for the data point in the question, and assign the new data point the average across all of the predicted Y values.<\/li><\/ol>\n\n\n\n

    Implementing Random Forest Regression in Python<\/h2>\n\n\n\n

    Our goal here is to build a team of decision trees, each making a prediction about the dependent variable and the ultimate prediction of random forest is average of predictions of all trees. <\/p>\n\n\n\n

    For our example, we will be using the Salary – positions dataset which will predict the salary based on prediction. <\/p>\n\n\n\n

    The dataset used can be found at https:\/\/github.com\/content-anu\/dataset-polynomial-regression<\/a><\/p>\n\n\n\n

    1. Importing the dataset<\/h3>\n\n\n\n

    We’ll use the numpy<\/a>, pandas<\/a>, and matplotlib<\/a> libraries to implement our model.<\/p>\n\n\n

    \nimport pandas as pd\nimport numpy as np\nimport matplotlib.pyplot as plt\n\ndataset = pd.read_csv('Position_Salaries.csv')\ndataset.head()\n<\/pre><\/div>\n\n\n
    The dataset snapshot is as follows: <\/p>\n\n\n\n
    \"Image
    Output snapshot of dataset<\/figcaption><\/figure><\/div>\n\n\n\n
    2. Data preprocessing<\/h3>\n\n\n\n
    We will not have much data preprocessing. We will just have to identify the matrix of features and the vectorized array.<\/p>\n\n\n
    \nX = dataset.iloc[:,1:2].values\ny = dataset.iloc[:,2].values\n<\/pre><\/div>\n\n\n
    3. Fitting the Random forest regression to dataset<\/h3>\n\n\n\n
    We will import the RandomForestRegressor from the ensemble library of sklearn. We create a regressor object using the RFR class constructor. The parameters include:<\/p>\n\n\n\n
    1. n_estimators : number of trees in the forest. (default = 10)<\/li>
    2. criterion : Default is mse ie mean squared error. This was also a part of decision tree.<\/li>
    3. random_state<\/li><\/ol>\n\n\n
      \nfrom sklearn.ensemble import RandomForestRegressor\nregressor = RandomForestRegressor(n_estimators = 10, random_state = 0)\nregressor.fit(X,y)\n<\/pre><\/div>\n\n\n
      The regressor line is as follows:<\/strong><\/p>\n\n\n\n
      \"Image
      Regressor line<\/figcaption><\/figure><\/div>\n\n\n\n
      We will just make a test prediction as follows:<\/p>\n\n\n
      \ny_pred=regressor.predict([[6.5]])\ny_pred\n<\/pre><\/div>\n\n\n
      \"Image
      Output of the prediction<\/figcaption><\/figure><\/div>\n\n\n\n
      4. Visualizing the result<\/h3>\n\n\n
      \n#higher resolution graph\nX_grid = np.arange(min(X),max(X),0.01)\nX_grid = X_grid.reshape(len(X_grid),1) \n\nplt.scatter(X,y, color='red') #plotting real points\nplt.plot(X_grid, regressor.predict(X_grid),color='blue') #plotting for predict points\n\nplt.title("Truth or Bluff(Random Forest - Smooth)")\nplt.xlabel('Position level')\nplt.ylabel('Salary')\nplt.show()\n<\/pre><\/div>\n\n\n
      The graph produced is as shown below:<\/p>\n\n\n\n
      \"Image
      Graph for 10 trees<\/figcaption><\/figure><\/div>\n\n\n\n
      5. Interpretation of the above graph<\/h3>\n\n\n\n
      We get many steps in this graph than with one decision tree. We have a lot more of intervals and splits. We get more steps in our stairs. <\/p>\n\n\n\n
      Every prediction is based on 10 votes (we have taken 10 decision trees). Random forest calculates many averages for each of these intervals. <\/p>\n\n\n\n
      The more number of trees we include, more is the accuracy because many trees converge to the same ultimate average.<\/p>\n\n\n\n
      6. Rebuilding the model for 100 trees<\/h3>\n\n\n
      \nfrom sklearn.ensemble import RandomForestRegressor\nregressor = RandomForestRegressor(n_estimators = 100, random_state = 0)\nregressor.fit(X,y)\n<\/pre><\/div>\n\n\n
      The regressor equation formed for the above 100 trees is as follows:<\/p>\n\n\n\n
      \"Image
      Regressor equation<\/figcaption><\/figure>\n\n\n\n
      7. Creating the graph for 100 trees<\/h3>\n\n\n
      \n#higher resolution graph\nX_grid = np.arange(min(X),max(X),0.01)\nX_grid = X_grid.reshape(len(X_grid),1) \nplt.scatter(X,y, color='red') \n\nplt.plot(X_grid, regressor.predict(X_grid),color='blue') \nplt.title("Truth or Bluff(Random Forest - Smooth)")\nplt.xlabel('Position level')\nplt.ylabel('Salary')\nplt.show()\n<\/pre><\/div>\n\n\n
      \"Image
      Graph with 100 trees<\/figcaption><\/figure><\/div>\n\n\n\n
      The steps of the graph don\u2019t increase 10 times as the number of trees in the forest. But the prediction will be better. Let\u2019s predict the result of the same variable.<\/p>\n\n\n
      \ny_pred=regressor.predict([[6.5]])\ny_pred\n<\/pre><\/div>\n\n\n
      \"Image
      Output prediction<\/figcaption><\/figure><\/div>\n\n\n\n
      8. Rebuilding the model for 300 trees<\/h3>\n\n\n
      \nfrom sklearn.ensemble import RandomForestRegressor\nregressor = RandomForestRegressor(n_estimators = 300, random_state = 0)\nregressor.fit(X,y)\n<\/pre><\/div>\n\n\n
      The output for the above code snippet produces the following regressor:<\/p>\n\n\n\n
      \"Image
      Regressor for 300 trees<\/figcaption><\/figure>\n\n\n\n
      9. Graph for 300 trees<\/h3>\n\n\n
      \n#higher resolution graph\nX_grid = np.arange(min(X),max(X),0.01)\nX_grid = X_grid.reshape(len(X_grid),1) \n\nplt.scatter(X,y, color='red') #plotting real points\nplt.plot(X_grid, regressor.predict(X_grid),color='blue') #plotting for predict points\n\nplt.title("Truth or Bluff(Random Forest - Smooth)")\nplt.xlabel('Position level')\nplt.ylabel('Salary')\nplt.show()\n<\/pre><\/div>\n\n\n
      The above code produces the following graph:<\/p>\n\n\n\n
      \"Image
      Graph for 300 trees<\/figcaption><\/figure><\/div>\n\n\n\n
      Now, let us make a prediction. <\/p>\n\n\n
      \ny_pred=regressor.predict([[6.5]])\ny_pred\n<\/pre><\/div>\n\n\n
      The output for the above code is as follows:<\/p>\n\n\n\n
      \"Image
      Prediction using 300 trees<\/figcaption><\/figure><\/div>\n\n\n\n
      Complete Python Code for Implementing Random Forest Regression<\/h2>\n\n\n
      \nimport pandas as pd\nimport numpy as np\nimport matplotlib.pyplot as plt\n \ndataset = pd.read_csv('Position_Salaries.csv')\ndataset.head()\n\nX = dataset.iloc[:,1:2].values\ny = dataset.iloc[:,2].values\n\n# for 10 trees\nfrom sklearn.ensemble import RandomForestRegressor\nregressor = RandomForestRegressor(n_estimators = 10, random_state = 0)\nregressor.fit(X,y)\n\ny_pred=regressor.predict([[6.5]])\ny_pred\n\n#higher resolution graph\nX_grid = np.arange(min(X),max(X),0.01)\nX_grid = X_grid.reshape(len(X_grid),1) \n \nplt.scatter(X,y, color='red') #plotting real points\nplt.plot(X_grid, regressor.predict(X_grid),color='blue') #plotting for predict points\n \nplt.title("Truth or Bluff(Random Forest - Smooth)")\nplt.xlabel('Position level')\nplt.ylabel('Salary')\nplt.show()\n\n\n# for 100 trees\nfrom sklearn.ensemble import RandomForestRegressor\nregressor = RandomForestRegressor(n_estimators = 100, random_state = 0)\nregressor.fit(X,y)\n\n#higher resolution graph\nX_grid = np.arange(min(X),max(X),0.01)\nX_grid = X_grid.reshape(len(X_grid),1) \nplt.scatter(X,y, color='red') \n \nplt.plot(X_grid, regressor.predict(X_grid),color='blue') \nplt.title("Truth or Bluff(Random Forest - Smooth)")\nplt.xlabel('Position level')\nplt.ylabel('Salary')\nplt.show()\n\ny_pred=regressor.predict([[6.5]])\ny_pred\n\n# for 300 trees\nfrom sklearn.ensemble import RandomForestRegressor\nregressor = RandomForestRegressor(n_estimators = 300, random_state = 0)\nregressor.fit(X,y)\n\n#higher resolution graph\nX_grid = np.arange(min(X),max(X),0.01)\nX_grid = X_grid.reshape(len(X_grid),1) \n \nplt.scatter(X,y, color='red') #plotting real points\nplt.plot(X_grid, regressor.predict(X_grid),color='blue') #plotting for predict points\n \nplt.title("Truth or Bluff(Random Forest - Smooth)")\nplt.xlabel('Position level')\nplt.ylabel('Salary')\nplt.show()\n\ny_pred=regressor.predict([[6.5]])\ny_pred\n\n<\/pre><\/div>\n\n\n
      The output of the above code will be graphs and prediction values. Below are the graphs: <\/p>\n\n\n\n
      \"Image
      Output graphs<\/figcaption><\/figure><\/div>\n\n\n\n
      Conclusion<\/h2>\n\n\n\n
      As you have observed, the 10 trees model predicted the salary for 6.5 years of experience to be 167,000. The 100 trees model predicted 158,300 and the 300 trees model predicted 160,333.33. Hence more the number of trees, the more accurate is our result. <\/p>\n","protected":false},"excerpt":{"rendered":"
      Welcome to this article on Random Forest Regression. Let me quickly walk you through the meaning of regression first. What is Regression in Machine Learning? Regression is a machine learning technique that is used to predict values across a certain range. Let us see understand this concept with an example, consider the salaries of employees […]<\/p>\n","protected":false},"author":12,"featured_media":8803,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[9],"tags":[],"class_list":["post-8788","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-examples"],"blocksy_meta":[],"_links":{"self":[{"href":"https:\/\/www.askpython.com\/wp-json\/wp\/v2\/posts\/8788","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.askpython.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.askpython.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.askpython.com\/wp-json\/wp\/v2\/users\/12"}],"replies":[{"embeddable":true,"href":"https:\/\/www.askpython.com\/wp-json\/wp\/v2\/comments?post=8788"}],"version-history":[{"count":0,"href":"https:\/\/www.askpython.com\/wp-json\/wp\/v2\/posts\/8788\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.askpython.com\/wp-json\/wp\/v2\/media\/8803"}],"wp:attachment":[{"href":"https:\/\/www.askpython.com\/wp-json\/wp\/v2\/media?parent=8788"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.askpython.com\/wp-json\/wp\/v2\/categories?post=8788"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.askpython.com\/wp-json\/wp\/v2\/tags?post=8788"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}