Advanced Topics
Performance Optimization
Profiling, benchmarking, and making Python code faster
Interview: Production systems — critical for senior roles
Python optimization follows a strict hierarchy: algorithm improvement > data structure choice > built-in functions > third-party libraries > C extensions. Never optimize without profiling first.
Profiling Tools
- cProfile: Function-level timing (built-in)
- line_profiler: Line-by-line execution time
- memory_profiler: Line-by-line memory usage
- py-spy: Sampling profiler — no code changes needed
Key Optimization Strategies
- Use built-ins:
sum(),map(),any()are implemented in C - List comprehensions over loops: Faster because the loop runs in C
- Generator expressions: Save memory for large datasets
- __slots__: Reduce memory per object by 40-60%
- functools.lru_cache: Memoize expensive function calls
- NumPy vectorization: Replace loops with array operations
Use Cases
Profiling production APIs to find slow endpoints
Optimizing data pipelines processing millions of records
Reducing memory footprint with __slots__ and generators
Caching expensive computations with lru_cache
Common Mistakes
Optimizing before profiling — premature optimization is the root of all evil
Using list comprehension when a generator expression suffices for large data
Not caching with lru_cache when the same inputs are computed repeatedly
Ignoring algorithm complexity — O(n²) in Python will always be slow