Replacing data
Replacing values in rows or columns is a common practice when working with tabular data. There are many reasons why we may need to replace specific values within a dataset. Python provides the flexibility to replace single values or multiple values within our dataset. We can use the replace method to achieve this.
Getting ready
We will work with the Marketing Campaign data again for this recipe.
How to do it…
We will remove duplicate data using the pandas library:
- Import the
pandaslibrary:import pandas as pd
- Load the
.csvfile into a dataframe usingread_csv. Then, subset the dataframe to include only relevant columns:marketing_data = pd.read_csv("data/marketing_campaign.csv")marketing_data = marketing_data[['ID', 'Year_Birth', 'Kidhome', 'Teenhome']]
- Inspect the data. Check the first few rows, and check the number of columns and rows:
ID Year_Birth Kidhome Teenhome
0 5524 1957 0 0
1 2174 1954 1 1
2 4141 1965 0 0
3 6182 1984 1 0
4 5324 1981 1 0
marketing_data.shape
(2240, 4)
- Replace the values in
Teenhomewithhas teenandhasno teen:marketing_data['Teenhome_replaced'] = marketing_data['Teenhome'].replace([0,1,2],['has no teen','has teen','has teen'])
- Inspect the output:
marketing_data[['Teenhome','Teenhome_replaced']].head()
Teenhome Teenhome_replaced
0 0 has no teen
1 1 has teen
2 0 has no teen
3 0 has no teen
4 0 has no teen
Great! We just replaced values in our dataset.
How it works...
We refer to pandas as pd in step 1. In step 2, we use read_csv to load the .csv file into a pandas dataframe and call it marketing_data. We also subset the dataframe to include only four relevant columns. In step 3, we inspect the dataset using head() to see the first five rows in the dataset. Using the shape method, we get a sense of the number of rows and columns.
In step 4, we use the replace method to replace values within the Teenhome column. The first argument of the method is a list of the existing values that we want to replace, while the second argument contains a list of the values we want to replace it with. It is important to note that the lists for both arguments must be the same length.
In step 5, we inspect the result.
There’s more...
In some cases, we may need to replace a group of values that have complex patterns that cannot be explicitly stated. An example could be certain phone numbers or email addresses. In such cases, the replace method gives us the ability to use regex for pattern matching and replacement. Regex is short for regular expressions, and it is used for pattern matching.
See also
- You can check out this great resource by Data to Fish on replacing data in
pandas: https://datatofish.com/replace-values-pandas-dataframe/ - Here is an insightful resource by GeeksforGeeks on regex in the
replacemethod inpandas: https://www.geeksforgeeks.org/replace-values-in-pandas-dataframe-using-regex/ - Here is another article by W3Schools that highlights common regex patterns in Python: https://www.w3schools.com/python/python_regex.asp
