Part 2 · Intermediate Python · Module 11
The Python Data Model
Why does len() work on lists, strings, and dicts alike? Why can you write a + b for numbers and for strings? Because Python's syntax is a set of hooks called special methods. Implement them and your own classes behave exactly like built-ins.
- Explain how syntax such as
len(x),a + b, andx[i]maps to special ("dunder") methods - Write useful
__repr__and__str__methods and know when each is used - Implement
__eq__and__hash__while honoring the hash contract - Add ordering with
functools.total_orderingand returnNotImplementedcorrectly - Overload arithmetic operators, including reflected and in-place forms
- Make container-like classes with
__len__,__getitem__,__contains__,__iter__, and__bool__ - Make instances callable with
__call__and preview the context-manager protocol
Everything is an object
In Python, every value is an object: a chunk of memory with three properties. Its identity (id(x)) never changes while the object exists. Its type (type(x)) decides which operations it supports. Its value is the data it holds. That includes numbers and strings, but also functions, classes, modules, and even types themselves.
The data model is the part of the language reference that describes how objects plug into Python's syntax. The central idea: syntax calls special methods on the object's type. Special methods have names surrounded by double underscores, like __len__. People call them dunder methods ("double underscore").
| You write | Python calls (roughly) |
|---|---|
len(x) | type(x).__len__(x) |
a + b | type(a).__add__(a, b), then maybe type(b).__radd__(b, a) |
x[k] | type(x).__getitem__(x, k) |
k in x | type(x).__contains__(x, k) |
for item in x | iter(x) → type(x).__iter__(x) |
if x: | type(x).__bool__(x) (or __len__) |
repr(x), str(x) | __repr__, __str__ |
x(args) | type(x).__call__(x, args) |
with x: | __enter__ and __exit__ |
Notice "type(x)": special methods are looked up on the class, not the instance. Assigning obj.__len__ = ... on one instance has no effect on len(obj). And you almost never call dunders directly; write len(x), not x.__len__(). The built-in functions do extra work (for example, len checks the result is a non-negative int).
That last line prints NotImplemented: a special singleton value that a binary method returns to say "I don't know how to handle this operand." You will learn exactly what Python does with it in the operator sections.
Remember: every value is an object, and syntax is a call to a special method on the object's type.
__repr__ and __str__
Without help, your objects print as something like <__main__.Point object at 0x10a3f2d50>. That tells you nothing while debugging. Two methods fix this:
__repr__: the developer representation, returned byrepr(x). It should be unambiguous, and ideally look like a Python expression that recreates the object:Point(x=1, y=2). The REPL, debuggers, logs, and containers (a list prints thereprof each element) all use it.__str__: the user-facing representation, returned bystr(x)and used byprint()and f-strings. If you don't define it, Python falls back to__repr__.
Remember: __repr__ is for developers (unambiguous, recreatable); __str__ is for end users and falls back to __repr__.
Equality and hashing: the contract
By default, a == b for your objects is the same as a is b: two separately created points are never equal, even with identical coordinates. Define __eq__ to compare by value:
class Point:
def __init__(self, x, y):
self.x, self.y = x, y
def __eq__(self, other):
if not isinstance(other, Point):
return NotImplemented # let Python try other.__eq__, then fall back to identity
return (self.x, self.y) == (other.x, other.y)
Returning NotImplemented (not False, and not raising) for unknown types is important: it gives the other operand a chance to answer, and if neither knows, Python falls back to an identity check, so Point(1, 2) == "hello" is simply False. You also get != for free: the default __ne__ inverts __eq__.
Why __hash__ must follow
Dictionaries and sets find keys by their hash: an integer computed by hash(x) that decides which bucket to look in (you will build a hash table in Part 3). The lookup first jumps to the bucket by hash, then confirms with ==. This only works if Python can rely on the hash contract:
To protect you, Python sets __hash__ = None automatically when a class defines __eq__ without __hash__. Instances become unhashable: putting them in a set raises TypeError. That is a deliberate safety net, not a bug. Step through the trace to see a set lookup succeed only after both methods are defined consistently:
The idiomatic hash combines exactly the fields used in __eq__ into a tuple and hashes that. The fields are stored as _x/_y (with no setters) to signal that they must not change.
Remember: define __eq__ and __hash__ together, hash the same fields you compare, keep those fields immutable, and return NotImplemented for unknown types.
Ordering and functools.total_ordering
sorted(), min(), max(), and comparisons like a < b use the rich comparison methods: __lt__ (<), __le__ (<=), __gt__ (>), and __ge__ (>=). Writing all four is repetitive and error-prone. The functools.total_ordering class decorator fills in the rest if you supply __eq__ and one ordering method.
Two lessons hide here. First, comparing as tuples of ints gives the right answer (3.9 < 3.13), while comparing the strings would not ("3.9" > "3.13"). Second, returning NotImplemented for an unknown type lets Python try the reflected comparison ("2.0".__gt__(v)); when that is also NotImplemented, Python raises a clear TypeError.
Remember: supply __eq__ plus __lt__ and decorate with @total_ordering; compare tuples of fields; return NotImplemented for foreign types.
Arithmetic operators and reflected operations
Operator overloading means defining what +, -, *, and friends do for your type. Each binary operator has three methods:
| Operator | Normal | Reflected (right side) | In-place |
|---|---|---|---|
+ | __add__ | __radd__ | __iadd__ (+=) |
- | __sub__ | __rsub__ | __isub__ |
* | __mul__ | __rmul__ | __imul__ |
/ | __truediv__ | __rtruediv__ | __itruediv__ |
@ | __matmul__ | __rmatmul__ | __imatmul__ |
Unary operators have single methods: __neg__ (-v), __pos__, __abs__ (abs(v)).
How Python evaluates a + b
- Call
type(a).__add__(a, b). If it returns a value other thanNotImplemented, that is the result. - Otherwise call
type(b).__radd__(b, a), the reflected method. If it returns a real value, done. - If both return
NotImplemented(or don't exist), raiseTypeError: unsupported operand type(s).
(One refinement: if b's type is a subclass of a's type and overrides the reflected method, Python tries b's reflected method first, so subclasses can take control.)
This is why 3 * v can work for your vector: int.__mul__(3, v) has no idea what a vector is and returns NotImplemented, so Python asks v.__rmul__(3).
Here is a complete, well-behaved Vector2. Experiment with it: try v - "x", abs(v), and v += w.
Look at the += line. Without __iadd__, Python computes v + w and rebinds v to the new object, leaving alias pointing at the old one. Mutable built-ins such as list define __iadd__ to mutate in place (and return self), which is why += on a list affects every alias. For value-like types, not defining __iadd__ is usually the safer choice.
Remember: a + b tries a.__add__(b), then b.__radd__(a); return NotImplemented for types you don't support; += falls back to + and rebinding.
The sequence and container protocols
A protocol in Python is an informal set of methods that, when present, let an object work in a certain context. You don't inherit from anything; you just implement the methods. This is duck typing: "if it walks like a duck and quacks like a duck, it's a duck." (M12 shows how to formalize protocols with typing.Protocol.)
| Method | Enables | Fallback when missing |
|---|---|---|
__len__ | len(x), truthiness | TypeError |
__getitem__ | x[i], x[a:b], x["key"] | TypeError |
__iter__ | for, list(x), unpacking | calls __getitem__(0), (1), … until IndexError |
__contains__ | item in x | iterates and compares with == |
__reversed__ | reversed(x) | uses __len__ + __getitem__ |
The fallbacks are generous: a class with just __len__ and __getitem__ is already iterable, supports in, and works with reversed(). You add __iter__ and __contains__ when you can do them more efficiently or more correctly.
__getitem__ receives whatever is inside the brackets. For deck[1:5] that is a slice object, slice(1, 5, None); because we delegate to a list, slicing works automatically. If you want slices to return your own type rather than a list, check isinstance(position, slice) and wrap the result.
For mappings, __getitem__ takes keys instead of positions, and should raise KeyError for a missing key (sequences raise IndexError). Those exception types matter: the iteration fallback stops on IndexError, and dict.get-style code catches KeyError.
Remember: __len__ + __getitem__ gets you a lot; add __iter__ and __contains__ for efficiency; raise IndexError for sequences and KeyError for mappings.
Truthiness with __bool__
Every object can be tested in if, while, and, or, and not. Python decides its truth value in this order:
- If the type defines
__bool__, call it. It must returnTrueorFalse. - Otherwise, if it defines
__len__, the object is false when the length is 0. - Otherwise the object is always true.
That's why empty lists, strings, and dicts are falsy: they define __len__. Your Deck above is falsy once empty, with no extra work.
Remember: __bool__ first, then __len__, then always true.
Callable objects and context managers (preview)
__call__
Functions are objects whose type defines __call__. Your own objects can be callable too. This is useful when you need a function that remembers state or is configured once and used many times:
Anything that accepts a function (like key= for sorted, or callbacks) accepts a callable object. In M15 you will see closures, which are often a lighter-weight alternative.
__enter__ and __exit__
You have used with open(...) as f: since M09. The with statement is another protocol: it calls __enter__ at the start (its return value is bound by as), and __exit__(exc_type, exc, tb) at the end, even if an exception was raised. If __exit__ returns a true value, the exception is suppressed; normally it returns None/False so errors propagate.
M15 goes deeper, including contextlib.contextmanager, which builds context managers from generators.
Remember: __call__ makes instances callable; __enter__/__exit__ power with, and __exit__ always runs.
Practice
Give Book(title, year) a __repr__ returning exactly Book('Dune', 1965) (use !r so the title is quoted, and type(self).__name__ for the class name) and a __str__ returning Dune (1965).
Hint
f"{type(self).__name__}({self.title!r}, {self.year!r})"
Show solution
class Book:
def __init__(self, title, year):
self.title = title
self.year = year
def __repr__(self):
return f"{type(self).__name__}({self.title!r}, {self.year!r})"
def __str__(self):
return f"{self.title} ({self.year})"
Write Version(text) that parses "1.10.2" into a tuple of ints stored as parts. Implement __eq__, __hash__, and __lt__ (returning NotImplemented for non-Versions) and use functools.total_ordering so all comparisons work. Also add __repr__ returning Version('1.10.2').
Hint
Compare and hash self.parts. For the repr, rebuild the text with ".".join(map(str, self.parts)).
Show solution
from functools import total_ordering
@total_ordering
class Version:
def __init__(self, text):
self.parts = tuple(int(p) for p in text.split("."))
def __repr__(self):
return f"Version({'.'.join(map(str, self.parts))!r})"
def __eq__(self, other):
if not isinstance(other, Version):
return NotImplemented
return self.parts == other.parts
def __hash__(self):
return hash(self.parts)
def __lt__(self, other):
if not isinstance(other, Version):
return NotImplemented
return self.parts < other.parts
Implement Vector2(x, y) with: __eq__; + and - between vectors; * with an int or float on either side; unary -; abs() (length); and __repr__ returning Vector2(3, 4). Every binary method must return NotImplemented for unsupported operands, so Vector2(1, 2) + 5 raises TypeError.
Hint
Check isinstance(k, (int, float)) in __mul__, and set __rmul__ = __mul__. Beware: bool is a subclass of int, which is fine here.
Show solution
import math
class Vector2:
def __init__(self, x, y):
self.x = x
self.y = y
def __repr__(self):
return f"Vector2({self.x!r}, {self.y!r})"
def __eq__(self, other):
if not isinstance(other, Vector2):
return NotImplemented
return (self.x, self.y) == (other.x, other.y)
__hash__ = None
def __add__(self, other):
if not isinstance(other, Vector2):
return NotImplemented
return Vector2(self.x + other.x, self.y + other.y)
def __sub__(self, other):
if not isinstance(other, Vector2):
return NotImplemented
return Vector2(self.x - other.x, self.y - other.y)
def __mul__(self, k):
if not isinstance(k, (int, float)):
return NotImplemented
return Vector2(self.x * k, self.y * k)
__rmul__ = __mul__
def __neg__(self):
return Vector2(-self.x, -self.y)
def __abs__(self):
return math.hypot(self.x, self.y)
Write Sentence(text) that splits the text into words (text.split()). Implement __len__ (number of words), __getitem__ (indexing and slicing: a slice returns a new Sentence of those words), __iter__ (yields words), __contains__ (case-insensitive word membership), __bool__ (false when there are no words), and __repr__ returning Sentence('the words').
Hint
In __getitem__, check isinstance(index, slice); build the new Sentence from " ".join(self.words[index]). For __iter__, return iter(self.words).
Show solution
class Sentence:
def __init__(self, text):
self.words = text.split()
def __repr__(self):
return f"Sentence({' '.join(self.words)!r})"
def __len__(self):
return len(self.words)
def __getitem__(self, index):
if isinstance(index, slice):
return Sentence(" ".join(self.words[index]))
return self.words[index]
def __iter__(self):
return iter(self.words)
def __contains__(self, word):
target = word.lower()
return any(w.lower() == target for w in self.words)
def __bool__(self):
return bool(self.words)
Write Polynomial(*coeffs) where coefficients are given from the constant term upward: Polynomial(1, 0, 2) means 1 + 0x + 2x². Implement __call__(x) to evaluate it, __add__ for two polynomials (pad the shorter one with zeros), __eq__ (compare coefficients after dropping trailing zeros), and __bool__ (false for the zero polynomial).
Hint
Normalize in __init__: convert to a list and pop trailing zeros. Then equality is plain list comparison and __bool__ is bool(self.coeffs). Evaluate with sum(c * x ** i for i, c in enumerate(self.coeffs)).
Show solution
from itertools import zip_longest
class Polynomial:
def __init__(self, *coeffs):
self.coeffs = list(coeffs)
while self.coeffs and self.coeffs[-1] == 0:
self.coeffs.pop()
def __repr__(self):
return f"Polynomial{tuple(self.coeffs)!r}"
def __call__(self, x):
return sum(c * x ** i for i, c in enumerate(self.coeffs))
def __add__(self, other):
if not isinstance(other, Polynomial):
return NotImplemented
return Polynomial(*(a + b for a, b in zip_longest(self.coeffs, other.coeffs, fillvalue=0)))
def __eq__(self, other):
if not isinstance(other, Polynomial):
return NotImplemented
return self.coeffs == other.coeffs
__hash__ = None
def __bool__(self):
return bool(self.coeffs)
Mini-project: a currency-safe Money type
Quiz
Recap
- Every value is an object with identity, type, and value; syntax calls special methods on the type.
- Write
__repr__always (unambiguous, for developers); add__str__for user-friendly output. __eq__and__hash__go together; equal objects must hash equally; keep hashed fields immutable.@total_orderingplus__eq__and__lt__gives all comparisons.a + btries__add__, then the reflected__radd__; returnNotImplementedfor unsupported types.+=uses__iadd__if present, otherwise+and rebinding.- Container protocol:
__len__,__getitem__,__iter__,__contains__, with generous fallbacks. - Truthiness:
__bool__, else__len__, else true. __call__makes objects callable;__enter__/__exit__drive thewithstatement.
Going further
- Language reference: Data model (the definitive list of special methods)
functools.total_ordering- Implementing the arithmetic operations (the
numbersmodule guide) collections.abcand its table of mixin methodsdecimalfor the Money mini-project- Next: M12 Inheritance, Composition & Protocols