I still can’t figure out exactly which operations are expensive in Python. My C/C++ can’t help me much because it seems that things aren’t implemented like I’d've expected—like lists that aren’t lists, but array lists, leading to for operations you would otherwise expect to be .
But a friend of mine—Olivier—showed me a simple, basic, yet rather effective tool to profile Python programs (I’m not sure if I should say script or not).
The tool, RunSnakeRun, inserts hooks in the Python interpreter to build a report of where time is spent, a bit à la Valgrind/Kcachegrind but much simpler, and to display the results using both tables and a rectangular tree-structured inclusion graph (a “SquareMap”, see here).
To install RunSnakeRun, simply type:
sudo apt-get install python-profiler python-wxgtk2.8 python-setuptools sudo easy_install SquareMap RunSnakeRun
in a shell. The package python-setuptools seems to provide easy_install.
To profile your application, invoke Python as
python -m cProfile -o outputfilename pythonscript script_arguments
python -m cProfile -o profiledata uniform-spherical-points.py 1000
This will profile the application uniform-spherical-points.py, called with an argument of 1000 (whatever it may mean for that program) and will create the profiledata file containing all the information necessary to inspect the performance of the profiled program.
To view the collected data, invoke:
And it will yield a window such as:
Examining the display, you see three panes. One is the complete list of functions called in your program, with different statistics such as the number of calls, etc. You can click on the header of the column to sort by this column.
The second pane is the SquareMap that lets you see where the time is spent, with filename, function, and line number.
The third pane is a simple call-graph table.
While this is a great tool, it does suffer from its minimalism. For example, it keeps maximizing on open (which I find supremely annoying) and does not remember window position nor size. It also conspicuously lacks a preference menu.
The official web page of RunSnakeRun can be found here.