Find Unique Strings Or Objects Using Java Stream API

Unique Strings/Objects

In this tutorial you will see an example on how to find unique strings or objects using Java stream API.

Java stream API has a method distinct() that removes duplicates and returns unique strings from a list of strings. But for objects you cannot directly apply this distinct() method directly to find out unique elements from a list of objects. Therefore I will use Predicate to remove duplicates and find the unique objects by a particular attribute of the object.

Prerequisites

Java 8+ (19)

Remove Duplicates from Strings

Here is the below example, I will show you how to remove duplicate strings from a list of strings and return the unique strings.

Let’s say you have the below list of strings:

List<String> strings = Arrays.asList("Ani", "Sou", "Lit", "Sum", "Ani", "Sou", "Pab");

Obviously you see that the above list has few duplicate strings, such as Ani and Sou. Now you need to remove these duplicate strings from the resultant strings. The final strings become:

Ani, Sou, Lit, Sum, Pab

So the below code returns the unique strings:

List<String> uniqueStrings = strings.stream().distinct().collect(Collectors.toList());

Remove Duplicates from Objects

Now you will see how to remove duplicates from objects. You need to make the object unique by a particular attribute of the object. So let’s create a class User that will have below attributes or properties:

protected static class User {

	private String name;
	private String email;
	private int age;
	
	//...
	
}

First I will create few dummy User objects:

private static List<User> createUserList() {
	List<User> userList = new ArrayList<>();

	User e1 = new User("John", "[email protected]", 21);
	User e2 = new User("Martin", "[email protected]", 19);
	User e3 = new User("Marry", "[email protected]", 31);
	User e4 = new User("Stephan", "[email protected]", 18);
	User e5 = new User("Garry", "[email protected]", 26);
	User e6 = new User("Marry", "[email protected]", 30);
	User e7 = new User("Garry", "[email protected]", 25);





	userList.add(e1);
	userList.add(e2);
	userList.add(e3);
	userList.add(e4);
	userList.add(e5);
	userList.add(e6);
	userList.add(e7);

	return userList;
}

Here I will use two approaches to find out the unique users from a list of users – first approach is to put all the user objects into a map by one of its attributes and second approach is to use Predicate to find out the unique user objects.

Below code put users into map by its name attribute:

List<User> userList = createUserList();
Collection<User> uniqueUsers = userList.stream().collect(Collectors.toMap(User::getName, p -> p, (p, q) -> p)).values();

Below code use Predicate to remove duplicates:

private static <T> Predicate<T> distinctByKey(Function<? super T, ?> keyExtractor) {
	Set<Object> seen = ConcurrentHashMap.newKeySet();
	return t -> seen.add(keyExtractor.apply(t));
}

Finally call distintByKey() method to remove duplicates:

List<User> distinctUsers = userList.stream().filter(distinctByKey(User::getName)).collect(Collectors.toList());

The whole source code can be written as:

public final class JavaStreamUniqueStringsObjects {

	public static void main(String[] args) {
		List<String> strings = Arrays.asList("Ani", "Sou", "Lit", "Sum", "Ani", "Sou", "Pab");
		System.out.println("Unique strings");
		List<String> uniqueStrings = strings.stream().distinct().collect(Collectors.toList());
		uniqueStrings.forEach(s -> System.out.println(s));





		System.out.println();

		List<User> userList = createUserList();

		System.out.println("Unique users by name");
		Collection<User> uniqueUsers = userList.stream().collect(Collectors.toMap(User::getName, p -> p, (p, q) -> p))
				.values();
		uniqueUsers.forEach(usr -> System.out.println(usr));

		System.out.println();

		List<User> distinctUsers = userList.stream().filter(distinctByKey(User::getName)).collect(Collectors.toList());
		distinctUsers.forEach(usr -> System.out.println(usr));
	}

	private static <T> Predicate<T> distinctByKey(Function<? super T, ?> keyExtractor) {
		Set<Object> seen = ConcurrentHashMap.newKeySet();
		return t -> seen.add(keyExtractor.apply(t));
	}

	private static List<User> createUserList() {
		List<User> userList = new ArrayList<>();

		User e1 = new User("John", "[email protected]", 21);
		User e2 = new User("Martin", "[email protected]", 19);
		User e3 = new User("Marry", "[email protected]", 31);
		User e4 = new User("Stephan", "[email protected]", 18);
		User e5 = new User("Garry", "[email protected]", 26);
		User e6 = new User("Marry", "[email protected]", 30);
		User e7 = new User("Garry", "[email protected]", 25);

		userList.add(e1);
		userList.add(e2);
		userList.add(e3);
		userList.add(e4);
		userList.add(e5);
		userList.add(e6);
		userList.add(e7);

		return userList;
	}

	protected static class User {

		private String name;
		private String email;
		private int age;

		public User() {
		}

		public User(String name, String email, int age) {
			this.name = name;
			this.email = email;
			this.age = age;
		}

		public String getName() {
			return name;
		}

		public void setName(String name) {
			this.name = name;
		}

		public String getEmail() {
			return email;
		}

		public void setEmail(String email) {
			this.email = email;
		}

		public int getAge() {
			return age;
		}

		public void setAge(int age) {
			this.age = age;
		}

		@Override
		public String toString() {
			return "User [name=" + name + ", email=" + email + ", age=" + age + "]";
		}

	}
}

Testing the Program

Executing the above class will give you below output:

Unique strings
Ani
Sou
Lit
Sum
Pab

Unique users by name
User [name=Stephan, [email protected], age=18]
User [name=John, [email protected], age=21]
User [name=Garry, [email protected], age=26]
User [name=Martin, [email protected], age=19]
User [name=Marry, [email protected], age=31]

User [name=John, [email protected], age=21]
User [name=Martin, [email protected], age=19]
User [name=Marry, [email protected], age=31]
User [name=Stephan, [email protected], age=18]
User [name=Garry, [email protected], age=26]

Hope you got an idea how to remove duplicate items using Java stream.

Source Code

Download