How to Find Duplicates in a List - Python

Finding duplicates in a list is a common task in programming. In Python, there are several ways to do this. Let’s explore the efficient methods to find duplicates.

Using a Set (Most Efficient for Large Lists)

Set() method is used to set a track seen elements and helps to identify duplicates.

a = [1, 2, 3, 4, 5, 2, 6, 3]

# A set to keep track of elements that have been seen
seen = set()
# A list to store duplicates found in the input list
duplicates = []

# Iterate over each element in the list
for i in a:
    if i in seen:
        duplicates.append(i)
    else:
        seen.add(i)

print(duplicates)

[2, 3]

There are more approaches to Find Duplicates in a List :

Using collection.counter

Using collection.counter is efficient as in this Counter class from the collections module is greater for counting occurrences of each element .

from collections import Counter

a = [1, 2, 3, 4, 5, 2, 6, 3]

# Use Counter to count 
#the occurrences of each element in the list
counts = Counter(a)
duplicates = [item for item, count in counts.items() if count > 1]
print(duplicates)

[2, 3]

Using List Comprehension

List comprehension allows us to write more compact code. We then use a list comprehension to find items that appear more than once and store them in the duplicates list. Here’s how we can find duplicates using list comprehension.

a = [1, 2, 3, 4, 5, 2, 6, 3]

# Create a list of duplicates using list comprehension
duplicates = [i for i in set(a) if a.count(i) > 1]
print(duplicates)

[2, 3]