Track additional allocation statistics

[iceblue25]

For

a=rand(6,1);

@time command shows different behavior when

a[1:3] = a[4:6]

vs

for i=1:3
a[i]=a[i+3];
end

with respect to memory allocation. The first copy allocates fresh memory each time the command is called (not even reusing the previous temporary allocation). Is this behavior expected?

[johnmyleswhite]

Yup, the first version allocates a temporary.

[JeffBezanson]

Correct, but we are planning to change that (#3701). Also, allocation reported by @time is a running counter, not taking reuse into account. If we allocate 100 bytes, then free it, then allocate another 100 bytes, we report 200 bytes of allocation.

In the future, please put backticks around code; otherwise writing @time will ping a github user with the name "time".

[iceblue25]

Thanks for the reply. So how do I know if the memory is reused later on? The @allocated macro reports growing memory as well.

Thanks and keep up the good work.

[StefanKarpinski]

So how do I know if the memory is reused later on?

How do you mean?

[iceblue25]

Stefan, here is my issue.
I run a function foo once and @allocated reports I consumed 100 bytes. (I time a begin end block). I rerun the same but the function foo is run twice and @allocated reports 200 bytes. It means julia didnt reuse my previously allocated 100 bytes and created a new one. (By the way the memory grew because of the issue I reported above and it is fixed by using a for loop).

My question is regarding Jeff's comment: "If we allocate 100 bytes, then free it, then allocate another 100 bytes, we report 200 bytes of allocation."

How do I separate cases when 200 bytes of allocation is because I didnt free the memory or julia didnt reuse the memory, allocated a new chunk and grew the memory footprint?

Do I need to add clear_malloc_data()? If yes is that needed only during memory profiling and then removed from the code?

I find the summary reporting of valgrind in C programs easier to spot memory leaks and observe total memory consumption.

[JeffBezanson]

We could just collect and report more statistics, in particular the number of bytes freed.

[iceblue25]

That would be a nice feature for memory profiling!

[ScottPJones]

@timev now reports the number of bytes freed, and a lot of other stuff, and the change (now merged) tracks realloc correctly (i.e. if it grows the memory, it adds to allocd, if it shrinks it, it adds to freed).
@timed also now returns as an extra return value a structure that has all the information.

[iceblue25]

Thanks, I will give it a try!