Advanced Topics
Memory Management
Understanding Python memory, GC, and optimization
Interview: Performance interviews — very common at FAANG
Python uses a combination of reference counting and a generational garbage collector to manage memory automatically. Understanding these mechanisms helps you write efficient code and debug memory leaks.
Reference Counting
Every Python object maintains a reference count. When the count drops to zero, the object is immediately deallocated. This is the primary memory management mechanism in CPython.
Garbage Collector (GC)
Reference counting alone cannot handle cyclic references (A references B, B references A). Python's cyclic GC runs periodically and uses three generations. Objects surviving collection are promoted to older generations, which are scanned less frequently.
Memory Pools
CPython uses a private heap with a pymalloc allocator for objects under 512 bytes. This reduces overhead from calling the OS allocator for every small object. Larger allocations go directly to the system malloc.
Common Pitfalls
- Mutable default arguments: Default arguments are created once and shared across calls.
- Forgotten references in caches: Storing objects in a global dict prevents deallocation forever.
- Circular references with __del__: Prior to Python 3.4, objects in reference cycles with
__del__methods were never collected.
Use Cases
Optimizing memory usage in data pipelines processing millions of records
Debugging memory leaks in long-running services
Using __slots__ to reduce per-object overhead in high-cardinality systems
WeakValueDictionary for caches that should not prevent garbage collection
Common Mistakes
Mutable default arguments sharing state across function calls
Storing strong references in global caches that prevent GC
Not using weakref for observer patterns and event callbacks
Ignoring tracemalloc when debugging memory growth in production