Metaprogramming

Classes are objects too

Metaprogramming means writing code that inspects or modifies the structure of a program (its classes, functions, and attributes) rather than just its data. In many languages that requires macros or code generation. In Python it falls out of one design decision: everything, including a class, is an ordinary runtime object. A class statement is executed like any other statement, and its result is an object you can pass around, inspect, and modify.

That object has a type too. The type of most classes is type, which is why type is called the default metaclass: the class of classes. And because type is itself a class, type(type) is type closes the loop.

There are five hooks, in increasing order of power and decreasing order of how often you should reach for them:

The attribute lookup algorithm

Everything in this module hangs on one algorithm: what Python does when you write obj.name. That expression calls type(obj).__getattribute__(obj, "name"). For ordinary objects that is object.__getattribute__, which implements the following precedence. First, two definitions:

• A descriptor is any object whose class defines __get__, __set__, or __delete__, and which lives as an attribute on a class.
• A data descriptor defines __set__ or __delete__ (usually both __get__ and __set__). A non-data descriptor defines only __get__. Plain functions are non-data descriptors; property is a data descriptor.

Why this specific order? It lets property (a data descriptor) protect an attribute even if someone stuffs a same-named key into obj.__dict__, while letting an instance shadow a method (a non-data descriptor) by assigning to it. It is also why functools.cached_property works: it is a non-data descriptor that computes once and writes the result into the instance dictionary, so the next lookup stops at step 2 and never calls the descriptor again.

Assignment follows a shorter path: obj.name = v calls type(obj).__setattr__, which looks for a data descriptor on the type and calls its __set__; otherwise it writes to obj.__dict__.

__getattr__ vs __getattribute__

These two names are easy to confuse and behave very differently:

Remember: data descriptor, then instance dict, then non-data descriptor or class attribute, then __getattr__. Special methods are looked up on the type.

Descriptors: reusable attribute logic

A property customizes one attribute of one class. When you need the same logic (say, "must be a positive int") on many attributes across many classes, you write a descriptor class once and instantiate it as a class attribute wherever you need it. The protocol has four methods:

The key design question is where to store the per-instance value. The descriptor object is shared by every instance (it lives on the class), so storing self.value = value on the descriptor would make every instance share one value, a classic bug. The standard answers:

• Store it in the instance's __dict__ under the same name. This works because a data descriptor takes precedence over the instance dict on lookup, so the descriptor still intercepts reads and writes.
• Store it under a private name ("_" + name) with setattr.
• Store it in a weakref.WeakKeyDictionary keyed by instance (for classes that use __slots__ and have no __dict__; the instances must be hashable and weak-referenceable).

__set_name__ (added in 3.6) solved the old annoyance of repeating the attribute name: age = Positive("age") becomes simply age = Positive().

Step through a smaller version to watch __set_name__ run at class-creation time (before any instance exists) and __set__ run during __init__:

Remember: a descriptor is shared by all instances, so per-instance state goes on the instance. __set_name__ tells the descriptor its name. Data descriptors are what make validation unavoidable.

Methods, properties, and classmethods are descriptors

Descriptors are not an exotic add-on; they are how Python's object model works. A function defined in a class body is stored in the class __dict__ as a plain function. Functions have a __get__ method, so they are non-data descriptors. When you access obj.method, step 3 of the lookup calls function.__get__(obj, type(obj)), which returns a bound method: a small object holding the function (__func__) and the instance (__self__). Calling it calls func(obj, *args). That is the entire mechanism behind self.

The other built-in method decorators are descriptors that bind differently:

To demystify property, here is a working pure-Python version. CPython's is written in C, but it behaves the same way:

Remember: self is not a keyword; it is the instance a function was bound to by function.__get__. property is a data descriptor; methods and cached_property are non-data descriptors.

__slots__: fixed attribute layouts

By default every instance can carry a __dict__, a hash table that lets you add any attribute at any time, plus a __weakref__ slot. That flexibility costs memory. Recent CPython versions soften it (ordinary instances store attributes "inline" with a key table shared across the class, and a real dict is only materialized if something asks for __dict__), but the overhead is still measurable, and when you create millions of small objects (points, graph nodes, parsed records) it adds up.

Declaring __slots__ = ("x", "y") tells type to allocate a fixed array of references in each instance instead of a dict, and to create a member descriptor on the class for each name (a data descriptor that reads and writes a fixed offset in the object). Consequences:

• Memory: instances are much smaller; there is no per-instance dict.
• Speed: attribute access is slightly faster (a fixed offset, no hashing), although CPython's specializing interpreter (3.11+) and inline attribute storage have narrowed the gap considerably. Measure before claiming a speedup.
• Safety: assigning a misspelled attribute (p.lenght = 3) raises AttributeError instead of silently creating a new attribute.

The trade-offs

Remember: __slots__ trades flexibility for memory. It replaces the per-instance dict with member descriptors, and every class in the hierarchy must cooperate.

__init_subclass__: hooking subclass creation

A very common reason people historically wrote metaclasses was "do something whenever a subclass is defined": register it, validate it, or configure it. PEP 487 (3.6) added a much simpler hook: __init_subclass__(cls, **kwargs), an implicit classmethod on the parent that runs after each subclass is created. Keyword arguments in the class statement (class Csv(Exporter, fmt="csv")) are passed to it.

Notice that the subclass registers itself just by being defined. That's why plugin systems often say "import your module to activate it", and why such registration is a form of import-time side effect: test isolation and import order now matter. Validation at class-definition time is a big win, though: the error appears when the module is imported, not hours later when a rarely used code path runs.

Remember: __init_subclass__ lives on the parent and runs once per subclass. It covers most registration and validation use cases that used to need a metaclass.

Class decorators

A class decorator is a callable that takes a fully built class and returns a class (usually the same one, modified). It runs once, after the class body and all class-creation hooks. Unlike __init_subclass__, it is opt-in per class and is not inherited: a subclass of a decorated class is not automatically decorated. @dataclass, @functools.total_ordering, and @enum.unique are all class decorators.

This is the same strategy dataclasses uses, except that it reads the class's annotations and generates source code for __init__ that it compiles with exec, so the generated methods run as fast as hand-written ones.

Metaclasses: customizing class creation

Since a class is an object, something must construct it. When Python executes a class statement, it gathers the name, the bases, and a namespace dictionary filled by running the body, then calls the metaclass with them. The default metaclass is type, and you can call it directly with the three-argument form type(name, bases, namespace):

The full class-creation sequence

For class C(Base, metaclass=M, flag=True): body, Python does the following (simplified from the data model reference):

1. Resolve bases (including __mro_entries__, which is how list[int] and Generic[T] can appear as bases).
2. Determine the metaclass: the explicit metaclass=, otherwise the most derived metaclass among the bases' types. If two bases have unrelated metaclasses, you get a metaclass conflict TypeError.
3. Prepare the namespace: ns = M.__prepare__("C", bases, flag=True). The default returns an empty dict.
4. Execute the body in ns. Each def and assignment becomes a key.
5. Create the class: C = M("C", bases, ns, flag=True), which calls M.__new__ then M.__init__. Inside type.__new__, Python calls __set_name__ on every descriptor in the namespace and then the parent's __init_subclass__.
6. Apply class decorators, innermost first, and bind the result to the name C.

What only a metaclass can do

Most class customization no longer needs a metaclass. What remains:

• Control the namespace with __prepare__: record definition order (historically; dicts are ordered now), reject duplicate definitions, or auto-create entries. enum uses a special namespace so RED = auto() works and duplicate member names raise.
• Customize behavior of the class object itself: methods defined on the metaclass are available on the class but not on instances. Color["RED"], len(Color), and iter(Color) on an Enum work because EnumType defines __getitem__, __len__, and __iter__.
• Customize isinstance/issubclass via __instancecheck__/__subclasscheck__, which is how ABCMeta.register creates virtual subclasses.

Remember: class creation is __prepare__ → body → metaclass.__new__ (which triggers __set_name__ and __init_subclass__) → __init__ → decorators.

Dynamic access, inspect, and annotationlib

Introspection is examining objects at runtime. The basic tools are built-ins: getattr(obj, name[, default]), setattr, hasattr, delattr, vars(obj) (the __dict__), and dir(obj) (all reachable names). They let you choose an attribute by a string computed at runtime, which is how command dispatchers, serializers, and test runners work.

The inspect module goes deeper: inspect.signature returns a Signature with typed Parameter objects (kind, default, annotation), Signature.bind validates arguments exactly like a real call would, getmembers lists attributes with a predicate, and isfunction/ismethod/ isclass/iscoroutinefunction classify objects. Frameworks like pytest (fixture injection by parameter name) and FastAPI (request parsing from type hints) are built on these.

Annotations in 3.14: deferred evaluation and annotationlib

For metaprogramming this matters in two ways. Libraries that read annotations at class-creation time (dataclasses, ORMs, validators) should use annotationlib.get_annotations(cls, format=Format.FORWARDREF) so that a not-yet-defined name doesn't crash class creation. And reading cls.__annotations__ directly now runs code (the __annotate__ function), so it can raise. Prefer annotationlib.get_annotations (or inspect.get_annotations) over poking at dunder attributes.

Practice

class Positive:
    def __set_name__(self, owner, name):
        self.name = name

    def __get__(self, obj, objtype=None):
        if obj is None:
            return self
        return obj.__dict__[self.name]

    def __set__(self, obj, value):
        if isinstance(value, bool) or not isinstance(value, (int, float)):
            raise TypeError(f"{self.name} must be a number")
        if value <= 0:
            raise ValueError(f"{self.name} must be > 0")
        obj.__dict__[self.name] = value


class Order:
    qty = Positive()
    price = Positive()

    def __init__(self, qty, price):
        self.qty = qty
        self.price = price

class Command:
    registry = {}

    def __init_subclass__(cls, name=None, **kwargs):
        super().__init_subclass__(**kwargs)
        if name is None:
            return
        if name in Command.registry:
            raise ValueError(f"command {name!r} already registered")
        Command.registry[name] = cls

    def execute(self, *args):
        raise NotImplementedError


def run(name, *args):
    return Command.registry[name]().execute(*args)


class Greet(Command, name="greet"):
    def execute(self, who):
        return f"hello {who}"

class Config:
    def __init__(self, data):
        object.__setattr__(self, "_data", data)

    def __getattr__(self, name):
        try:
            value = self._data[name]
        except KeyError:
            raise AttributeError(name) from None
        return Config(value) if isinstance(value, dict) else value

    def __setattr__(self, name, value):
        raise AttributeError("Config is read-only")

import annotationlib

MISSING = object()

def auto_init(cls):
    fields = tuple(annotationlib.get_annotations(cls))
    defaults = {f: cls.__dict__.get(f, MISSING) for f in fields}

    def __init__(self, *args, **kwargs):
        if args:
            raise TypeError(f"{cls.__name__}() takes keyword arguments only")
        unknown = set(kwargs) - set(fields)
        if unknown:
            raise TypeError(f"unexpected arguments: {sorted(unknown)}")
        for f in fields:
            if f in kwargs:
                value = kwargs[f]
            elif defaults[f] is not MISSING:
                value = defaults[f]
            else:
                raise TypeError(f"missing required argument {f!r}")
            setattr(self, f, value)

    cls.__init__ = __init__
    cls.__fields__ = fields
    return cls

@auto_init
class User:
    name: str
    email: str
    active: bool = True

class Field:
    def __init__(self, kind):
        self.kind = kind

class ModelMeta(type):
    def __new__(mcls, name, bases, ns):
        cls = super().__new__(mcls, name, bases, ns)
        fields = {}
        for base in reversed(cls.__mro__[1:]):
            fields.update(base.__dict__.get("_fields", {}))
        fields.update({k: v for k, v in ns.items() if isinstance(v, Field)})
        cls._fields = fields
        return cls

    def __len__(cls):
        return len(cls._fields)

class Model(metaclass=ModelMeta):
    pass

class Person(Model):
    name = Field(str)
    age = Field(int)

class Employee(Person):
    salary = Field(float)
    age = Field(float)

Mini-project: a tiny ORM-style model layer

pixi init orm-lite
cd orm-lite
pixi add python=3.14 pytest
mkdir -p src tests
pixi run python -m pytest -q

MISSING = object()

class Field:
    def __init__(self, default=MISSING, *, required=True):
        self.default, self.required = default, required

    def __set_name__(self, owner, name):
        self.name = name

    def __get__(self, obj, objtype=None):
        if obj is None:
            return self
        return obj.__dict__.get(self.name, self.default)

    def __set__(self, obj, value):
        obj.__dict__[self.name] = self.validate(self.to_python(value))

    def to_python(self, value):
        return value

    def validate(self, value):
        return value

class Model:
    _fields: dict[str, Field] = {}

    def __init_subclass__(cls, **kwargs):
        super().__init_subclass__(**kwargs)
        ...  # collect Field instances from the MRO into cls._fields

Quiz

Recap

• Classes are objects created by a metaclass (by default type).
• Lookup order: data descriptor → instance __dict__ → non-data descriptor/class attribute → __getattr__.
• Descriptors (__get__, __set__, __set_name__) package attribute logic; store per-instance data on the instance.
• Functions bind via __get__; property is a data descriptor; classmethod, staticmethod, and cached_property are non-data descriptors.
• __slots__ saves memory by replacing the instance dict with member descriptors, at the cost of flexibility.
• __init_subclass__ and class decorators cover most needs; metaclasses (__prepare__, methods on the class object) are a last resort.
• Use getattr/setattr, inspect.signature, and 3.14's annotationlib.get_annotations for introspection.

Going further

• Descriptor Guide (docs.python.org): the authoritative walkthrough, with pure-Python equivalents of built-ins
• Data model: customizing class creation
• annotationlib and Annotations best practices
• inspect module
• PEP 487 (__init_subclass__, __set_name__) and PEP 649
• Next: M37 Concurrency: Threads, Processes & Interpreters