Nik Kantar

Thursday, March 30, 2023

Getting Nothing Out of a Python Dictionary

A bit of nuance about dealing with null and missing values in Python dictionaries.

Let’s say you have a garage, like so:

my_garage = {
    "cars": ["Mazderp3", "Hyundai Veloster"],
    "motorcycles": ["Yamaha FJ-09", "Yamaha XSR700"],
    "bicycles": ["Cannondale Synapse", "Schwinn Herald"],
}

and you also have some new bicycles:

new_bicycles = ["Trek Checkpoint", "Electra Loft"]

You need to do something with both old and new bikes, but you’re me, and thus have a propensity for changing things up on a whim, which makes it hard to be sure if there are any bicycles already in the garage, or if the corner for them is even available. In other words, your garage dictionary might instead look like:

my_garage_empty = {
    "cars": ["Mazderp3", "Hyundai Veloster"],
    "motorcycles": ["Yamaha FJ-09", "Yamaha XSR700"],
    "bicycles": [],
}

or:

my_garage_none = {
    "cars": ["Mazderp3", "Hyundai Veloster"],
    "motorcycles": ["Yamaha FJ-09", "Yamaha XSR700"],
    "bicycles": None,
}

or even:

my_garage_missing = {
    "cars": ["Mazderp3", "Hyundai Veloster"],
    "motorcycles": ["Yamaha FJ-09", "Yamaha XSR700"],
}

OK, well, fine, you still need to do stuff with all the bikes, and to do that you need to collect them in a single list, and list unpacking is pretty great, so you try the obvious:

>>> [*my_garage["bicycles"], *new_bicycles]
['Cannondale Synapse', 'Schwinn Herald', 'Trek Checkpoint', 'Electra Loft']

>>> [*my_garage_empty["bicycles"], *new_bicycles]
['Trek Checkpoint', 'Electra Loft']

>>> [*my_garage_none["bicycles"], *new_bicycles]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: Value after * must be an iterable, not NoneType

>>> [*my_garage_missing["bicycles"], *new_bicycles]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
KeyError: 'bicycles'

Huh. Seems like everything was fine with the first two cases, but less so with the latter two, but it all makes sense: None is not a list and thus cannot be extended, and a missing value is also unusable in this way. The last error is actually quite easy to solve, so you leverage the dict.get method, and isolate your problem until you figure out how to get a list back:

>>> my_garage.get("bicycles")
['Cannondale Synapse', 'Schwinn Herald']

>>> my_garage_empty.get("bicycles")
[]

>>> my_garage_none.get("bicycles")

>>> my_garage_missing.get("bicycles")

Well, at least now you have only one error to fix—the None returned in the last two cases—which seems like an improvement. Since dict.get also accepts a default fallback, you give that a whirl:

>>> my_garage.get("bicycles", [])
['Cannondale Synapse', 'Schwinn Herald']

>>> my_garage_empty.get("bicycles", [])
[]

>>> my_garage_none.get("bicycles", [])

>>> my_garage_missing.get("bicycles", [])
[]

Interestingly enough, this solves the very last case, but not the penultimate one. That’s because dict.get falls back to the supplied default only when the key is completely missing from the dictionary, and None is a perfectly valid value to have there instead. To deal with this, or to the rescue:

>>> my_garage.get("bicycles", []) or []
['Cannondale Synapse', 'Schwinn Herald']

>>> my_garage_empty.get("bicycles", []) or []
[]

>>> my_garage_none.get("bicycles", []) or []
[]

>>> my_garage_missing.get("bicycles", []) or []
[]

Since or is triggered by any falsy value, and dict.get uses None as the default fallback, the explicitly specified empty list is redundant, and you can clean this up a little bit:

>>> my_garage.get("bicycles") or []
['Cannondale Synapse', 'Schwinn Herald']

>>> my_garage_empty.get("bicycles") or []
[]

>>> my_garage_none.get("bicycles") or []
[]

>>> my_garage_missing.get("bicycles") or []
[]

Trying list unpacking again (with parenthesis for required disambiguation) yields the desired results:

>>> [*(my_garage.get("bicycles") or []), *new_bicycles]
['Cannondale Synapse', 'Schwinn Herald', 'Trek Checkpoint', 'Electra Loft']

>>> [*(my_garage_empty.get("bicycles") or []), *new_bicycles]
['Trek Checkpoint', 'Electra Loft']

>>> [*(my_garage_none.get("bicycles") or []), *new_bicycles]
['Trek Checkpoint', 'Electra Loft']

>>> [*(my_garage_missing.get("bicycles") or []), *new_bicycles]
['Trek Checkpoint', 'Electra Loft']

To recap, if you absolutely, positively need to get something back when looking for a value in a dictionary, a combination of dict.get and or can get you there:

As always, please use responsibly. :)

P.S.: Someone on the Internet [reminded me][1] about collections.defaultdict, which solves a fair bit of this, though not the explicit None case. I [wrote a bit][2] about the related dict.setdefault and it a little while ago.

[1]: https://mastodon.social/@qwwerty@hachyderm.io/110116663171795694 "qwwerty: "@nkantar collections.defaultdi…" - Mastodon" [2]: /blog/2020/12/dict-setdefault-rocks/ "Python’s dict.setdefault Rocks | Blog | Nik Kantar"


Tags: programming, python

Thanks for reading! You can keep up with my writing via the feed or newsletter, or you can get in touch via email or Mastodon.


Older:
Dedupe a List in Python, Slowly
Newer:
Garage Detector