Python profiling with cProfile
14 May 2010
Compared to Perl's Devel::NYTProf, Python profilers are a bit rough around the edges. The recommended profiler as of 2.6 is part of the standard library and is called cProfile. It works only at the function (as opposed to statement) level and fancy graphical output is non existent. Here's a brief overview of what it offers.
Profiling output
For each function, cProfile makes the following data available:
- ncalls
- how many times the function is called
- tottime
- time spent on the function, excluding time spent on calling other functions
- percall
- tottime divided by ncalls
- cumtime
- time spent on the fucntion, incuding calls to other functions
- percall
- cumtime divided by tottime
Two different datasets are headed percall. The data is printed by default exactly in the order given above; except in columns from left to right. Keep that in mind for deciphering the ouput.
There are two ways of obtaining this display. If you run the cProfile on a script without specifying an output file, this data will just be printed to the screen. If you specify an output file, you need to use the pstats module to load the data (the actual output file is in some binary format). pstats also offers some methods to sort and filter the data. So if your program is not very small, you are better off using pstats.
python -m cProfile -o outputfile.profile name_of_your_program
then
import pstats
stats = pstats.Stats('outputfile.profile')
and now you can display the information in a manageble way by calling the appropriate methods on the stats object.