Advanced Topics
C Extensions
Writing Python extensions in C for performance
Interview: Performance optimization — advanced interviews
Python's CPython implementation allows you to write performance-critical code in C and expose it as a Python module. This is how libraries like NumPy, lxml, and Pillow achieve C-level speed while providing a Pythonic interface.
Approaches
- CPython C API: Direct interaction with Python's internals using
Python.h. Maximum control, most complexity. - ctypes: Call existing C libraries from Python without compilation. Good for wrapping system libraries.
- cffi: A more modern alternative to ctypes with better type safety.
When to Use C Extensions
Use C extensions only after profiling confirms a bottleneck that pure Python cannot solve. The development cost is high (debugging segfaults, managing reference counts manually, platform-specific builds).
Use Cases
Wrapping existing C/C++ libraries for Python use (OpenCV, BLAS)
Performance-critical inner loops in scientific computing
Building Python bindings for system-level libraries
Creating high-performance data processing pipelines
Common Mistakes
Jumping to C extensions before profiling to confirm the bottleneck
Forgetting to manage reference counts manually in the CPython C API
Not handling platform differences (Windows .dll vs Linux .so vs macOS .dylib)
Ignoring buffer protocol for array data — use NumPy arrays instead