Graham King

Solvitas perambulum

Profiling Django for CPU bound apps

software
Summary
For most Django web apps, the database is often the performance bottleneck, so reducing queries and adding indexes can vastly improve efficiency. Tools like django-debug-toolbar help with this. If the app is CPU bound, use `runprofileserver` from django-extensions to identify where time is spent. Install django-extensions, add it to `INSTALLED_APPS`, and run the profiling server with `django-admin.py runprofileserver`. After accessing the desired URL and interrupting the server, analyze the generated hotshot files using a stats object in Python. Sort the stats either by cumulative or individual function call time to pinpoint performance issues.

For most Django apps, indeed most webapps, the bottleneck is the database. The biggest gains usually come from reducing the number of queries used, and adding database indexes. django-debug-toolbar helps a lot here. After that, caching and de-normalization also help reduce database queries.

But what if your app is CPU bound? How do you find out where it’s spending it’s time? It’s easy with the runprofileserver from django-extensions – here’s how:

pip install django-extensions

Add django_extensions to your INSTALLED_APPS, then run the profiling server.

django-admin.py runprofileserver

In your browser, hit the url you’re interested in. runprofileserver will create hotshot files in /tmp/. Ctrl-C out of your server, and fire up ipython.

from hotshot import stats
p = stats.load('/tmp/<your filename here>')

# In order that functions got called
p.sort_stats('cumulative').print_stats(20)

# Slowest method first
p.sort_stats('time').print_stats(20)

Here p is a pstats object. I prefer the cumulative view. Look down it until you see a big drop in the cumulative time – that’s where the time was spent.