Profiling a looped process

Use this forum for questions on how to use .NET Memory Profiler and how to analyse memory usage.
Post Reply
zeidman
Posts: 2
Joined: Mon Apr 20, 2009 12:32 pm

Profiling a looped process

Post by zeidman » Mon Apr 20, 2009 12:37 pm

I am very new to this so please bear with me.

I have a batch process that appears to be leaking memory. However it leaks memory on each additional loop of the batch. When it is stopped the memory is released (I think).

I followed the second tutorial and while it gives a real insight into how you go about finding memory leaks I am not sure how I should proceed.

With my process I cannot simply take a base snapshot as I cannot stop the process. It has to continue for it to leak memory. It is more of a timing issue than anything else.

Any help would be really great.

Thanks

David

Andreas Suurkuusk
Posts: 1029
Joined: Wed Mar 02, 2005 7:53 pm

Post by Andreas Suurkuusk » Tue Apr 21, 2009 8:36 pm

If you suspect that a memory leak is caused by the batch, it would be good if you could collect one snapshots before the batch processing starts and one after. As I understand it, this is not possible through the user interface in your case. Have you tried to use the profiler API to trigger the snapshot? This will give you much better control of when the snapshots are created. Maybe you can try something like:

Code: Select all

using SciTech.NetMemProfiler;
...
void RunBatchLoop()
{
  int nProcessedBatches = 0;
  while( HasPendingBatch )
  {
    MemProfiler.FullSnapShot( "Begin batch " + ( nProcessedBatches + 1 ));

    ProcessNextBatch();

    MemProfiler.FullSnapshot( "End batch " + ( nProcessedBatches + 1 ));
    ++nProcessedBatches;
  }
} 
This code assumes that just a few batches needs to be run before a memory leak can be detected (a full snapshot cost a lot of time and memory). Another option would be to use the "fast snapshots" and the assertion API to programatically detect memory leaks, but it's a bit more complitcated. For more information about the profiler API, see http://memprofiler.com/OnlineDocs/netme ... lerapi.htm.

Does this help, or did I misunderstand what you are trying to do?
Best regards,

Andreas Suurkuusk
SciTech Software AB

zeidman
Posts: 2
Joined: Mon Apr 20, 2009 12:32 pm

Post by zeidman » Tue Apr 21, 2009 8:49 pm

That is more or less it. The process takes a row from a database table and processes it. It then loops the process and takes the next row. After each row there is some kind of memory leak so that is what I need to track.

David

Andreas Suurkuusk
Posts: 1029
Joined: Wed Mar 02, 2005 7:53 pm

Post by Andreas Suurkuusk » Wed Apr 22, 2009 6:20 pm

If there are a lot rows being processed, then my previous suggestion might not work so well (too many snapshots).

You might still extract some information by collecting snapshots manually while batches are running. The problem is of course that you will get a lot of information about instances that are only temporarily used while processing a batch, but you might be able to see trends of increased memory usage of some classes. These classes can then be candidates for further investigation.

Otherwise, my previous suggestion might still be used, but try to not collect a snapshot after processing each row. Maybe take a snapshot after 10 rows and one after 20 rows. Using this approach, you will get "clean" snapshots without any temporary instances.
Best regards,

Andreas Suurkuusk
SciTech Software AB

Post Reply

Who is online

Users browsing this forum: No registered users and 14 guests