Hi,
I have recently face memory leaks with object usage. After a day of work to find out the solution, I want to purpose here the issue and the way I have solved this.
Memory leaks often happen with a process that run for hours, like a FastCGI. The memory grow up (more or less quickly), and never free up.
All files can be found here :
https://gitorious.celogeek.com/talks/perl-memoryleaks
Here and example with objects :
MyData will storage a hash of data :
|
1 2 3 4 5 6 7 |
package MyData; use Moo; has 'data' => ( is => 'rw', default => sub {{}}, ); 1; |
MyStorage will used by default MyData :
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 |
package MyStorage; use Moo; use Data::Dumper; use feature 'say'; use MyData; has 'storage' => ( is => 'ro', default => sub { MyData->new } ); sub display_storage { my ($self) = @_; say Dumper $self->storage->data; } 1; |
And MyTest will use it’s own data structure and tell storage to use it :
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 |
package MyTest; use Moo; use MyStorage; use Carp; has 'data' => ( is => 'ro', default => sub { {} }, ); has 'storage' => ( is => 'ro', default => sub { my ($self) = @_; MyStorage->new(storage => $self); }, ); 1; |
Everything seems normal, but with have a memory leaks here. MyTest use MyStorage that use MyTest. Both can’t be free from memory because they need each other.
A good way to see this (except running million of new and check memory) is to add a ‘DESTROY’ sub and see when it is call.
In MyTest.pm I add (before the ’1;’) :
|
1 2 3 |
sub DESTROY { carp "MyTest destroy !"; } |
|
1 2 3 4 5 6 7 8 9 10 11 |
#!/usr/bin/env perl use Carp; use feature 'say'; use MyTest; for (1..3) { my $t = MyTest->new; $t->data->{test} = 1; $t->storage->display_storage; } say "Finish ! "; |
The result expected is :
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
$VAR1 = { 'test' => 1 }; MyTest destroy ! at test.pl line 11. $VAR1 = { 'test' => 1 }; MyTest destroy ! at test.pl line 11. $VAR1 = { 'test' => 1 }; MyTest destroy ! at test.pl line 11. Finish! |
But we got this :
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
$VAR1 = { 'test' => 1 }; $VAR1 = { 'test' => 1 }; $VAR1 = { 'test' => 1 }; Finish ! MyTest destroy ! at test.pl line 0. MyTest destroy ! at test.pl line 0. MyTest destroy ! at test.pl line 0. |
In my case, I have added this and check a request thought apache. I was expected a “destroy !” after each request, but never happen. So I reload apache and all my object was released !
Now let see how to test it with Test::LeakTrace :
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 |
#!perl use strict; use warnings; use Test::More tests => 3; # last test to print use Test::LeakTrace; use MyData; use MyStorage; use MyTest; use Carp; use Data::Dumper; no_leaks_ok { my $r = MyData->new; $r->data->{test} = 1; } 'my data is ok'; no_leaks_ok { my $r = MyStorage->new; $r->storage->data->{test} = 1; $r->display_storage; } 'my storage is ok'; no_leaks_ok { my $r = MyTest->new; $r->data->{test} = 1; $r->storage->display_storage; } 'my test is ok'; |
The first 2 tests pass, no circular references :
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
1..1 ok 1 - my data is ok (leaks 0 <= 0) $VAR1 = { 'test' => 1 }; $VAR1 = { 'test' => 1 }; ok 2 - my storage is ok (leaks 0 <= 0) $VAR1 = { 'test' => 1 }; $VAR1 = { 'test' => 1 }; |
But the third one leaks :
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 |
not ok 3 - my test is ok (leaks 7 <= 0) # Failed test 'my test is ok (leaks 7 <= 0)' # at Test.t line 29. # '7' # <= # '0' $VAR1 = { 'test' => 1 }; $VAR1 = { 'test' => 1 }; # leaked REF(0xc32d38) from (eval 15) line 6. # SV = IV(0xc32d30) at 0xc32d38 # REFCNT = 1 # FLAGS = (ROK) # RV = 0x10c0950 # SV = PVHV(0x109ebb0) at 0x10c0950 # REFCNT = 1 # FLAGS = (OBJECT,SHAREKEYS) # STASH = 0xdbdf48 "MyStorage" # ARRAY = 0x10c42b0 (0:7, 1:1) # hash quality = 100.0% # KEYS = 1 # FILL = 1 # MAX = 7 # RITER = -1 # EITER = 0x0 # Elt "storage" HASH = 0x531f78be # SV = IV(0xf82b08) at 0xf82b10 # REFCNT = 1 # FLAGS = (ROK) # RV = 0xd81ad0 # SV = PVHV(0x109eac0) at 0xd81ad0 # REFCNT = 1 # FLAGS = (OBJECT,SHAREKEYS) # STASH = 0x10a2568 "MyTest" # ARRAY = 0xdc3ca0 (0:6, 1:2) # hash quality = 125.0% # KEYS = 2 # FILL = 2 # MAX = 7 # RITER = -1 # EITER = 0x0 # Elt "data" HASH = 0x5eb5e655 # SV = IV(0xf47568) at 0xf47570 # REFCNT = 1 # FLAGS = (ROK) # RV = 0x10c08d8 # leaked HASH(0xd81ad0) from (eval 14) line 30. # SV = PVHV(0x109eac0) at 0xd81ad0 # REFCNT = 1 # FLAGS = (OBJECT,OOK,SHAREKEYS) # STASH = 0x10a2568 "MyTest" # ARRAY = 0x10c4080 (0:6, 1:2) # hash quality = 125.0% # KEYS = 2 # FILL = 2 # MAX = 7 # RITER = -1 # EITER = 0x0 # Elt "data" HASH = 0x5eb5e655 # SV = IV(0xf47568) at 0xf47570 # REFCNT = 1 # FLAGS = (ROK) # RV = 0x10c08d8 # SV = PVHV(0x109eb88) at 0x10c08d8 # REFCNT = 1 # FLAGS = (OOK,SHAREKEYS) # ARRAY = 0x10c8c80 (0:7, 1:1) # hash quality = 100.0% # KEYS = 1 # FILL = 1 # MAX = 7 # RITER = -1 # EITER = 0x0 # Elt "test" HASH = 0x3f75ccc1 # SV = IV(0xdcaed0) at 0xdcaed8 # REFCNT = 1 # FLAGS = (IOK,pIOK) # IV = 1 # Elt "storage" HASH = 0x531f78be # SV = IV(0xc32d30) at 0xc32d38 # REFCNT = 1 # FLAGS = (ROK) # RV = 0x10c0950 # SV = PVHV(0x109ebb0) at 0x10c0950 # REFCNT = 1 # FLAGS = (OBJECT,OOK,SHAREKEYS) # STASH = 0xdbdf48 "MyStorage" # ARRAY = 0xc20f50 (0:7, 1:1) # hash quality = 100.0% # KEYS = 1 # FILL = 1 # MAX = 7 # RITER = -1 # EITER = 0x0 # Elt "storage" HASH = 0x531f78be # SV = IV(0xf82b08) at 0xf82b10 # REFCNT = 1 # FLAGS = (ROK) # RV = 0xd81ad0 # SV = PVHV(0x109eac0) at 0xd81ad0 # REFCNT = 1 # FLAGS = (OBJECT,OOK,SHAREKEYS) # STASH = 0x10a2568 "MyTest" # ARRAY = 0x10c4080 (0:6, 1:2) # hash quality = 125.0% # KEYS = 2 # FILL = 2 # MAX = 7 # RITER = 6 # EITER = 0xfca9c8 # leaked SCALAR(0xdcaed8) from Test.t line 27. # 26: my $r = MyTest->new; # 27: $r->data->{test} = 1; # 28: $r->storage->display_storage; # SV = IV(0xdcaed0) at 0xdcaed8 # REFCNT = 1 # FLAGS = (IOK,pIOK) # IV = 1 # leaked REF(0xf47570) from (eval 14) line 31. # SV = IV(0xf47568) at 0xf47570 # REFCNT = 1 # FLAGS = (ROK) # RV = 0x10c08d8 # SV = PVHV(0x109eb88) at 0x10c08d8 # REFCNT = 1 # FLAGS = (OOK,SHAREKEYS) # ARRAY = 0x10c8c80 (0:7, 1:1) # hash quality = 100.0% # KEYS = 1 # FILL = 1 # MAX = 7 # RITER = -1 # EITER = 0x0 # Elt "test" HASH = 0x3f75ccc1 # SV = IV(0xdcaed0) at 0xdcaed8 # REFCNT = 1 # FLAGS = (IOK,pIOK) # IV = 1 # leaked REF(0xf82b10) from (eval 12) line 31. # SV = IV(0xf82b08) at 0xf82b10 # REFCNT = 1 # FLAGS = (ROK) # RV = 0xd81ad0 # SV = PVHV(0x109eac0) at 0xd81ad0 # REFCNT = 1 # FLAGS = (OBJECT,OOK,SHAREKEYS) # STASH = 0x10a2568 "MyTest" # ARRAY = 0x10c4080 (0:6, 1:2) # hash quality = 125.0% # KEYS = 2 # FILL = 2 # MAX = 7 # RITER = -1 # EITER = 0x0 # Elt "data" HASH = 0x5eb5e655 # SV = IV(0xf47568) at 0xf47570 # REFCNT = 1 # FLAGS = (ROK) # RV = 0x10c08d8 # SV = PVHV(0x109eb88) at 0x10c08d8 # REFCNT = 1 # FLAGS = (OOK,SHAREKEYS) # ARRAY = 0x10c8c80 (0:7, 1:1) # hash quality = 100.0% # KEYS = 1 # FILL = 1 # MAX = 7 # RITER = -1 # EITER = 0x0 # Elt "test" HASH = 0x3f75ccc1 # SV = IV(0xdcaed0) at 0xdcaed8 # REFCNT = 1 # FLAGS = (IOK,pIOK) # IV = 1 # Elt "storage" HASH = 0x531f78be # SV = IV(0xc32d30) at 0xc32d38 # REFCNT = 1 # FLAGS = (ROK) # RV = 0x10c0950 # SV = PVHV(0x109ebb0) at 0x10c0950 # REFCNT = 1 # FLAGS = (OBJECT,OOK,SHAREKEYS) # STASH = 0xdbdf48 "MyStorage" # ARRAY = 0xc20f50 (0:7, 1:1) # hash quality = 100.0% # KEYS = 1 # FILL = 1 # MAX = 7 # RITER = -1 # EITER = 0x0 # Elt "storage" HASH = 0x531f78be # SV = IV(0xf82b08) at 0xf82b10 # REFCNT = 1 # FLAGS = (ROK) # RV = 0xd81ad0 # leaked HASH(0x10c0950) from (eval 12) line 30. # SV = PVHV(0x109ebb0) at 0x10c0950 # REFCNT = 1 # FLAGS = (OBJECT,OOK,SHAREKEYS) # STASH = 0xdbdf48 "MyStorage" # ARRAY = 0xc20f50 (0:7, 1:1) # hash quality = 100.0% # KEYS = 1 # FILL = 1 # MAX = 7 # RITER = -1 # EITER = 0x0 # Elt "storage" HASH = 0x531f78be # SV = IV(0xf82b08) at 0xf82b10 # REFCNT = 1 # FLAGS = (ROK) # RV = 0xd81ad0 # SV = PVHV(0x109eac0) at 0xd81ad0 # REFCNT = 1 # FLAGS = (OBJECT,OOK,SHAREKEYS) # STASH = 0x10a2568 "MyTest" # ARRAY = 0x10c4080 (0:6, 1:2) # hash quality = 125.0% # KEYS = 2 # FILL = 2 # MAX = 7 # RITER = -1 # EITER = 0x0 # Elt "data" HASH = 0x5eb5e655 # SV = IV(0xf47568) at 0xf47570 # REFCNT = 1 # FLAGS = (ROK) # RV = 0x10c08d8 # SV = PVHV(0x109eb88) at 0x10c08d8 # REFCNT = 1 # FLAGS = (OOK,SHAREKEYS) # ARRAY = 0x10c8c80 (0:7, 1:1) # hash quality = 100.0% # KEYS = 1 # FILL = 1 # MAX = 7 # RITER = -1 # EITER = 0x0 # Elt "storage" HASH = 0x531f78be # SV = IV(0xc32d30) at 0xc32d38 # REFCNT = 1 # FLAGS = (ROK) # RV = 0x10c0950 # SV = PVHV(0x109ebb0) at 0x10c0950 # REFCNT = 1 # FLAGS = (OBJECT,OOK,SHAREKEYS) # STASH = 0xdbdf48 "MyStorage" # ARRAY = 0xc20f50 (0:7, 1:1) # hash quality = 100.0% # KEYS = 1 # FILL = 1 # MAX = 7 # RITER = 6 # EITER = 0x102ebd0 # leaked HASH(0x10c08d8) from MyTest.pm line 10. # 9: is => 'ro', # 10: default => sub { {} }, # 11:); # SV = PVHV(0x109eb88) at 0x10c08d8 # REFCNT = 1 # FLAGS = (OOK,SHAREKEYS) # ARRAY = 0x10c8c80 (0:7, 1:1) # hash quality = 100.0% # KEYS = 1 # FILL = 1 # MAX = 7 # RITER = -1 # EITER = 0x0 # Elt "test" HASH = 0x3f75ccc1 # SV = IV(0xdcaed0) at 0xdcaed8 # REFCNT = 1 # FLAGS = (IOK,pIOK) # IV = 1 |
I know, it is hard to read this. But we can see that the issue is around “storage” in MyTest. And the reason is I pass “$self” to MyStorage.
So MyStorage has a strong ref to MyTest bless object, and MyTest has a strong reference to MyStorage bless object. They can not free them each other because they depends each other.
The solution is to create a weak ref in MyTest of the “storage” object.
With ‘Moo’ it’s seems not possible to have a “ro” on an attribute with a weak_ref. So I use a lazy mode.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 |
package MyTest; use Moo; use MyStorage; use Carp; has 'data' => ( is => 'ro', default => sub { {} }, ); has 'storage' => ( is => 'lazy', default => sub { my ($self) = @_; MyStorage->new(storage => $self); }, weak_ref => 1, ); sub DESTROY { carp "MyTest destroy !"; } 1; |
Now let’s play again test.t :
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
$VAR1 = { 'test' => 1 }; MyTest destroy ! at test.pl line 11. $VAR1 = { 'test' => 1 }; MyTest destroy ! at test.pl line 11. $VAR1 = { 'test' => 1 }; MyTest destroy ! at test.pl line 11. Finish ! |
It seems the result we expected !
Let’s run the test again :
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 |
1..1 ok 1 - my data is ok (leaks 0 <= 0) $VAR1 = { 'test' => 1 }; $VAR1 = { 'test' => 1 }; ok 2 - my storage is ok (leaks 0 <= 0) $VAR1 = { 'test' => 1 }; MyTest destroy ! at /home/vincent/perl5/perlbrew/perls/perl-5.12.4/lib/site_perl/5.12.4/x86_64-linux-thread-multi/Test/LeakTrace.pm line 92. $VAR1 = { 'test' => 1 }; MyTest destroy ! at /home/vincent/perl5/perlbrew/perls/perl-5.12.4/lib/site_perl/5.12.4/x86_64-linux-thread-multi/Test/LeakTrace.pm line 40. ok 3 - my test is ok (leaks 0 <= 0) |
We solve the memory leaks !
But how can I find this in a ton of code ? actually I do it raw by raw. I go to my FastCGI, stop the request at the beginning of the response, play a big log, check the memory, and move the stop raw by raw until I get the module I use who consume a lot of memory.
So I test this module.
Now for all my futur module, I will test it, each time I create a new object, I test the “new”, and the usage of the module with Test::LeakTrace. It will prevent lot’s of issue in the futur.
Enjoy !
Celogeek
