How To Get Unique Values From A List In Python?

As a data scientist working with large datasets in the USA, I often encounter situations where I need to identify and work with distinct values within a list. Python provides several efficient methods to accomplish this task. In this tutorial, I will explain how to get unique values from a list in Python. Let us get into the details and explore some examples.

Table of Contents

Get Unique Values from a List in Python

Imagine you have a list of customer names from various states in the USA, but the list contains duplicates. For instance:

customer_names = ['John', 'Emma', 'Oliver', 'John', 'Ava', 'Emma', 'Liam']

To perform certain analyses or generate reports, you need to obtain a list of unique customer names.

Read How to Replace Values in a List Using Python?

Method 1: Use the set() Function

One of the simplest and most efficient ways to get unique values from a list is by using the built-in set() function in Python. A set is an unordered collection of unique elements. By converting a list to a set, duplicates are automatically removed. Here’s an example:

customer_names = ['John', 'Emma', 'Oliver', 'John', 'Ava', 'Emma', 'Liam']
unique_names = list(set(customer_names))
print(unique_names)

Output:

['Oliver', 'Liam', 'Ava', 'John', 'Emma']

I executed the above example code and added the screenshot below.

In this code, we convert the customer_names list to a set using set() which eliminates duplicates. Then, we convert the set back to a list using list() to obtain a list of unique names. The set() function provides a concise and efficient solution to remove duplicates from a list.

Preserve the Original Order

One drawback of using set() is that it does not preserve the original order of elements in the list. If maintaining the order is important, you can use a list comprehension along with the set() function:

customer_names = ['John', 'Emma', 'Oliver', 'John', 'Ava', 'Emma', 'Liam']
unique_names = [name for name in set(customer_names)]
print(unique_names)

Output:

['Oliver', 'John', 'Emma', 'Ava', 'Liam']

This approach creates a new list by iterating over the unique elements obtained from the set. The resulting list will have unique values while preserving the order of their first occurrence in the original list.

Check out How to Shuffle a List in Python?

Method 2: Use List Comprehension and Membership Testing

Another approach to get unique values from a Python list is by using list comprehension and membership testing. This method allows you to iterate over the list and add elements to a new list only if they haven’t been encountered before. Here’s an example:

customer_names = ['John', 'Emma', 'Oliver', 'John', 'Ava', 'Emma', 'Liam']
unique_names = []
[unique_names.append(name) for name in customer_names if name not in unique_names]
print(unique_names)

Output:

['John', 'Emma', 'Oliver', 'Ava', 'Liam']

I executed the above example code and added the screenshot below.

How to Get Unique Values from a List in Python

In this code, we initialize an empty list called unique_names. We then use a list comprehension to iterate over each name in the customer_names list. For each name, we check if it is not already present in the unique_names list using the not in operator. If the name is not found, it is appended to the unique_names list using the append() method.

This approach preserves the original order of elements and provides more control over the logic for determining uniqueness.

Read How to Convert a Dictionary to a List in Python?

Method 3: Use the pandas Library

If you are working with large datasets and using the Python pandas library, you can leverage its built-in functions to obtain unique values from a list. The pandas library provides powerful data manipulation and analysis tools. Here’s an example:

import pandas as pd

customer_names = ['John', 'Emma', 'Oliver', 'John', 'Ava', 'Emma', 'Liam']
df = pd.DataFrame({'Name': customer_names})
unique_names = df['Name'].unique().tolist()
print(unique_names)

Output:

['John', 'Emma', 'Oliver', 'Ava', 'Liam']

I executed the above example code and added the screenshot below.

Get Unique Values from a List in Python using pandas

In this code, we create a pandas DataFrame called df with a single column ‘Name’ containing the customer_names list. We then use the unique() function provided by pandas to get the unique values from the ‘Name’ column. Finally, we convert the resulting array back to a list using tolist().

Check out How to Randomly Select from a List in Python?

Conclusion

In this tutorial, I explained how to get unique values from a list in Python. I discussed using the set() function , using list comprehension, and using the pandas library.