PEP 584 - Union of dictionaries in Python

Writing user facing applications and frameworks in any language is challenging. One of the most common scenarios encountered while writing frameworks is to accept user configurations as dictionaries or a JSON and over-riding the default options with the user options. Combining two dictionaries is necessary in such situations.

Let us say we have two dictionaries

dict_a = {'apples': 2, 'oranges': 3 }
dict_b = {'peaches': 4, 'roses': 1}

A combined dictionary should result in a new dictionary with the keys merged.

dict_combined = {'apples': 2, 'oranges': 3, 'peaches': 4, 'roses': 1}

Dan Bader’s Python Tricks book highlights a syntactic sugar to combine two dictionaries available from Python 3.5+. This seemed like magic, when I first saw it.

dict_combined = {**dict_a, **dict_b}

It is not very obvious from the expression that the above expression combines two dictionaries. Not many people know about this syntax. Imagine using this expression in your code and coming back after 6 months to wonder what this actually does.

Let us see an older method that clearly spells out that a dictionary is getting updated by another dictionary.

dict_a.update(**dict_b)

The update method updates dict_a in place. This is one of the caveats and one has to be careful. The syntax is clear and there is little to guess its purpose. I would use something like this, compared to something that looks like magic. One reason is, it is simple and I will be able to understand what is happening even after a year or two. When I have to share my code with someone, I do not want my collaborators to be perplexed. Clarity is more important.

Duplicate keys in dictionaries

What happens when there are duplicate keys in the dictionaries ? Say, you are building a framework and you are allowing your users to pass options as a dictionary. The framework itself provides some default options for convenince. Now you want to over-ride the default options with the user provided ones. You can do this using both of the above methods. The value of the key in the second dictionary will be preferred.

default_dict.update(**user_dict)
dict_combined = {**default_dict, **user_dict}

The New | and |= Operator

The syntactic sugar is not intuitive and the update method provides just an inplace alternative. PEP 584 provides the best of both worlds and introduces two new operators | and |=. The first one is a | - the pipe operator to combine two dictionaries.

dict_merged = dict_a | dict_b

It provides all the niceties. Unlike the update method, it does not do merge in place and a new dictionary is returned. It does not seem like magic. Looking at the pipe | operator gives you a hint that it is some kind of a merge operation. If a key appears in both dict_a and dict_b then the key in dict_b will be preferred.

The second operator |= is an inplace version of the first one. Here dict_a will be updated in place.

dict_a |= dict_b

Over all this method looks pretty clean and simple. You can even pipe multiple dictionaries to merge all of them, although it is very rare to do so and may be a little inefficient.

dict_d = dict_a | dict_b | dict_c 

Give the pipe operator a try in your next project. Until the next PEP talk, cheers!