Finding memory leaks in Python with objgraph
Summary
After a frustrating time trying to find a memory leak in my Python code with guppy / heappy hpy, I tried objgraph and, wow, it makes it so easy! Here’s what you do:
pip install objgraph
At the relevant point in your code, add an import pdb; pdb.set_trace()
to drop into the debugger. Then just follow the docs on finding memory leaks with objgraph. In short you do:
>> import objgraph
>> objgraph.show_growth(limit=10) # Start counting
Ignore that output. Call the function that leaks memory, iterate once through you loop, whatever you need to do to make your program consume more memory. Now call show_growth again:
>> my_leaky_func()
>> objgraph.show_growth(limit=10) # Stop and show change
This time it shows the difference between now and the last time you called it. Those extra objects are the problem.
Finally you need to find where the reference to those leaky objects is being held:
>> import inspect, random
>> objgraph.show_chain(
... objgraph.find_backref_chain(
... random.choice(objgraph.by_type('MyBigFatObject')),
... inspect.ismodule),
... filename='chain.png')
That generates a lovely graph in your current directory, showing the ownership chain. Now wasn’t that easy? Thanks Marius Gedminas!