Perl – Memory Leaks – Test::LeakTrace at the rescue

Hi,
I have recently face memory leaks with object usage. After a day of work to find out the solution, I want to purpose here the issue and the way I have solved this.

Memory leaks often happen with a process that run for hours, like a FastCGI. The memory grow up (more or less quickly), and never free up.

All files can be found here :

https://gitorious.celogeek.com/talks/perl-memoryleaks

Here and example with objects :

MyData will storage a hash of data :

MyStorage will used by default MyData :

And MyTest will use it’s own data structure and tell storage to use it :

Everything seems normal, but with have a memory leaks here. MyTest use MyStorage that use MyTest. Both can’t be free from memory because they need each other.

A good way to see this (except running million of new and check memory) is to add a ‘DESTROY’ sub and see when it is call.

In MyTest.pm I add (before the ‘1;’) :

The result expected is :

But we got this :

In my case, I have added this and check a request thought apache. I was expected a “destroy !” after each request, but never happen. So I reload apache and all my object was released !

Now let see how to test it with Test::LeakTrace :

The first 2 tests pass, no circular references :

But the third one leaks :

I know, it is hard to read this. But we can see that the issue is around “storage” in MyTest. And the reason is I pass “$self” to MyStorage.

So MyStorage has a strong ref to MyTest bless object, and MyTest has a strong reference to MyStorage bless object. They can not free them each other because they depends each other.

The solution is to create a weak ref in MyTest of the “storage” object.

With ‘Moo’ it’s seems not possible to have a “ro” on an attribute with a weak_ref. So I use a lazy mode.

Now let’s play again test.t :

It seems the result we expected !

Let’s run the test again :

We solve the memory leaks !

But how can I find this in a ton of code ? actually I do it raw by raw. I go to my FastCGI, stop the request at the beginning of the response, play a big log, check the memory, and move the stop raw by raw until I get the module I use who consume a lot of memory.

So I test this module.

Now for all my futur module, I will test it, each time I create a new object, I test the “new”, and the usage of the module with Test::LeakTrace. It will prevent lot’s of issue in the futur.

Enjoy !

Celogeek

Short URL: http://sck.pm/Sk