Python - Concatenate Values with Same Keys in a List of Dictionaries

We are given a list of dictionaries where the values are lists and our task is to concatenate the values of the same keys across these dictionaries. For example, if we have a list of dictionaries like this: [{'gfg': [1, 5, 6, 7], 'good': [9, 6, 2, 10], 'CS': [4, 5, 6]}, {'gfg': [5, 6, 7, 8], 'CS': [5, 7, 10]}, {'gfg': [7, 5], 'best': [5, 7]}] then the output will be: {'gfg': [1, 5, 6, 7, 5, 6, 7, 8, 7, 5], 'good': [9, 6, 2, 10], 'CS': [4, 5, 6, 5, 7, 10], 'best': [5, 7]}

Using defaultdict

This method uses defaultdict from the collections module which automatically creates an empty list for any key that doesn't exist yet. This eliminates the need to check for keys before adding values.

from collections import defaultdict

li = [{'gfg' : [1, 5, 6, 7], 'good' : [9, 6, 2, 10],'CS' : [4, 5, 6]}, {'gfg' : [5, 6, 7, 8], 'CS' : [5, 7, 10]},{'gfg' : [7, 5], 'best' : [5, 7]}]

res = defaultdict(list)
for d in li:
  for k, v in d.items():
    res[k].extend(v)

print(dict(res))

{'gfg': [1, 5, 6, 7, 5, 6, 7, 8, 7, 5], 'good': [9, 6, 2, 10], 'CS': [4, 5, 6, 5, 7, 10], 'best': [5, 7]}

Explanation:

• defaultdict ensures that any key accessed in res will automatically have an empty list associated with it.
• extend() method appends all elements of v to the existing list associated with the key k in the res dictionary.

Using the itertools module

This method uses itertools.chain.from_iterable() to concatenate values of the same keys across all dictionaries in the list, it first extracts all unique keys and then combines the values associated with each key from all dictionaries.

import itertools

li = [{'gfg': [1, 5, 6, 7], 'good': [9, 6, 2, 10], 'CS': [4, 5, 6]}, {'gfg': [5, 6, 7, 8], 'CS': [5, 7, 10]}, {'gfg': [7, 5], 'best': [5, 7]}]

# Using chain.from_iterable() to create a single list of all values for each key
li = {key: list(itertools.chain.from_iterable([d.get(key, []) for d in li])) for key in set().union(*li)}

print(str(li))

{'good': [9, 6, 2, 10], 'CS': [4, 5, 6, 5, 7, 10], 'best': [5, 7], 'gfg': [1, 5, 6, 7, 5, 6, 7, 8, 7, 5]}

Explanation:

• Extract all unique keys from the list of dictionaries using set().union(*li).
• For each key, concatenate the values from all dictionaries using itertools.chain.from_iterable() and create a dictionary with the concatenated values.

Using reduce() from the functools module

In this method we use reduce() function from the functools module to iteratively combine dictionaries in the list by merging them and appending values for the same keys.

from functools import reduce

li = [{'gfg': [1, 5, 6, 7], 'good': [9, 6, 2, 10], 'CS': [4, 5, 6]}, {'gfg': [5, 6, 7, 8], 'CS': [5, 7, 10]}, {'gfg': [7, 5], 'best': [5, 7]}]

res = reduce(lambda d1, d2: {**d1, **{k: d1.get(k, []) + v for k, v in d2.items()}}, li)

print(str(res))

{'gfg': [1, 5, 6, 7, 5, 6, 7, 8, 7, 5], 'good': [9, 6, 2, 10], 'CS': [4, 5, 6, 5, 7, 10], 'best': [5, 7]}

Explanation:

• lambda d1, d2 is a function that takes two dictionaries.
• {k: d1.get(k, []) + v for k, v in d2.items()} in this expression, for each key in d2 we append its values to the corresponding key in d1. If the key isn't in d1, it starts with an empty list.
• {**d1, **new_dict} merges the updated dictionary into d1.
• reduce() is used to apply this to all dictionaries in the list thus resulting in a single merged dictionary with concatenated values.

Using a loop

This method iterates through the list of dictionaries using a loop and concatenates the values of each key into a single list.

li = [{'gfg': [1, 5, 6, 7], 'good': [9, 6, 2, 10], 'CS': [4, 5, 6]}, {'gfg': [5, 6, 7, 8], 'CS': [5, 7, 10]}, {'gfg': [7, 5], 'best': [5, 7]}]
res = {}

# Iterate through each dictionary in the list
for d in li:
   for k, v in d.items():
       res[k] = res.get(k, []) + v 

print(res)

{'gfg': [1, 5, 6, 7, 5, 6, 7, 8, 7, 5], 'good': [9, 6, 2, 10], 'CS': [4, 5, 6, 5, 7, 10], 'best': [5, 7]}