Debugging & Profiling
Profiling
Performance profiling with cProfile, pstats, and line_profiler — finding and fixing bottlenecks in Python code.
Interview: Performance optimization — profiling is essential for data-driven optimization decisions.
Profiling measures where your program spends time, identifying bottlenecks that need optimization. Python provides cProfile (C-level, low overhead) for function-level profiling and pstats for analyzing results. Third-party tools like line_profiler provide line-by-line timing.
cProfile
python -m cProfile script.py— profile an entire scriptpython -m cProfile -s cumtime script.py— sort by cumulative timecProfile.run("expression")— profile a code expression- Output shows: ncalls, tottime (self), cumtime (including calls), percall
pstats Analysis
- Save profile:
python -m cProfile -o output.prof script.py - Analyze:
python -m pstats output.prof - Sort by:
sort tottime,sort cumtime,sort calls - Filter:
stats.print_stats("module_name")— filter by module - Callers/callees:
print_callers(),print_callees()
Line Profiler
pip install line_profiler- Add
@profiledecorator to functions to measure kernprof -l -v script.py— line-by-line timing- Shows: Hits, Time, Per Hit, % Time for each line
- Pinpoints exactly which line is slow within a function
Interview Insight
Always profile before optimizing — intuition about bottlenecks is often wrong. cProfile shows function-level hotspots; line_profiler shows which exact lines are slow. Focus on cumulative time (cumtime) to find the real bottlenecks, not just frequently called functions.
Use Cases
Performance optimization — identifying slow functions before rewriting
Algorithm comparison — measuring which approach is faster
Regression detection — profiling in CI to catch performance regressions
Database optimization — finding N+1 queries and slow query patterns
Memory-intensive operations — profiling data processing pipelines
Common Mistakes
Optimizing without profiling — intuition about bottlenecks is often wrong
Looking at tottime only — cumtime includes called functions and shows real impact
Profiling with unrealistic data — profile with production-scale datasets
Not using line_profiler — cProfile shows function-level; line_profiler shows the exact slow line
Profiling in development mode — debug mode and assertions skew timing results