seeing contents deltas or telling what the deltas really are

Use this forum for questions on how to use .NET Memory Profiler and how to analyse memory usage.
Post Reply
nirva
Posts: 5
Joined: Wed Sep 06, 2006 3:43 am

seeing contents deltas or telling what the deltas really are

Post by nirva » Wed Sep 06, 2006 4:08 am

i have a situation where i have N new strings, and M removed strings, and N > M.

now, in many of the cases, the strings added and removed are the same string content, but in a few cases, (the N-M strings for example), they are different. is there any way I can see which of the strings are those?

ideally, there would be a option to just dump the contents of the instances (and their instance numbers, etc..) to a txt file, so i could do some analysis on them. another great option would be just to have the API give me access to all of them.

any ideas on how to deal with this? this tool has been great, but its been really hard to deal with when i have N added and N removed, or N added and M removed.

Andreas Suurkuusk
Posts: 1029
Joined: Wed Mar 02, 2005 7:53 pm

Post by Andreas Suurkuusk » Wed Sep 06, 2006 8:50 pm

In the current version of the profiler it is not possible to identify new instances that are identical to a previous instance, and I don't think it is very common that you replace a set of instances with a new identical set of instances. Anyway, I will add this to our feature wish-list, and we will consider whether this is a feature that we want to include in a future version.

When .NET Memory Profiler 3.0 Professional is released, you will get access to an extended API that will give you access to all data collected by a heap snapshot. Using this API, you will be able to perform your own analysis and thus be able to identify identical instances yourself (as long as instance field data has been collected).
Best regards,

Andreas Suurkuusk
SciTech Software AB

nirva
Posts: 5
Joined: Wed Sep 06, 2006 3:43 am

Post by nirva » Wed Sep 06, 2006 9:08 pm

does the beta 3.0 do it now? i can try that

nirva
Posts: 5
Joined: Wed Sep 06, 2006 3:43 am

Post by nirva » Wed Sep 06, 2006 10:05 pm

maybe you can suggest to me how i can find my issues then:

i have a app that connects to a server, gets a ton of data, builds lots of complex state, and then when it disconnects, it throws away all that complex state. then later it reconnects, and repeats. i think i have a leak every time this happens (taskmgr sizes go up). there were some easy wins i found because i saw stuff that was unique object types, so they were easy to track. another set was easy because it was just allocated N times and it was multiplying by N every time i disconnect/connected.

now, the memory snapshots between 2 connected situations should be identical (minus things like caches), but thats not the case. sometimes i see N adds and a few smaller than N removes. the causes are really really hard to determine because the data is 'string', and there are over 400 places that strings are allocated from, and the # of strings is huge (tens to hundreds of thousands of strings total). also the 'few smaller' changes from run to run, sometimes being zero, sometimes being large (few hundred). the data set sent is identical, so this points to potential state leakage and/or bugs.

any ideas or strategies on how to find out what is being 'leaked' between cycles?

Andreas Suurkuusk
Posts: 1029
Joined: Wed Mar 02, 2005 7:53 pm

Post by Andreas Suurkuusk » Thu Sep 07, 2006 8:46 pm

Hi,

No, the current preview of .NET Memory Profiler 3.0 does not include the extended API, but it will be included when we release the beta.

If your memory usage grows each time you update your cache, it seems possible that either the cache is not completely cleared between the updates, or you get more data to cache each time. Since you say that the DataSet you send is the same each time, I'd recommend that you check whether the cache is completely emptied before being updated. Do you think that this is possible for you? You can do this by collecting a heap snapshot after the cache is cleared but before it is updated. E.g.

Code: Select all

FillCache();
// Use cache
ClearCache();

CollectSnapshot();

FillCache();
// Use cache
ClearCache();

CollectSnapshot();

etc.
Best regards,

Andreas Suurkuusk
SciTech Software AB

nirva
Posts: 5
Joined: Wed Sep 06, 2006 3:43 am

Post by nirva » Thu Sep 07, 2006 8:57 pm

I'm aware of the cache issues.. I have verified there is no caching going on by turning off the caches completely.

It is the leaks that im looking for! Given that you said
I don't think it is very common that you replace a set of instances with a new identical set of instances
I assumed you'd have a reasonable stragegy to debug the situation I mentioned, because that is a common situation case.

So, how would you reccomend I debug this using the scitech profiler?

Also, when is the beta planned to be released?

Andreas Suurkuusk
Posts: 1029
Joined: Wed Mar 02, 2005 7:53 pm

Post by Andreas Suurkuusk » Fri Sep 08, 2006 7:36 pm

Hi,

I'm sorry, I didn't mean "cache" in my example. I was referring to the state you are building.

In your case you have a "state" which seems to include a lot of string instances, but in other cases the state might contain instances of other types than strings, and these types will probably not be identical between snapshots (e.g. they might refer to different instances within the state).

Anyway, our recommended strategies for finding memory leaks are:

Collect a snapshot before building a state, and a snapshot after clearing the state (as I suggested in my previous post). You say that you throw away the whole state when you disconnect. Is it not possible to collect a snapshot at this time?

If this is not possible and you have to collect snapshots when the state is fully built, then you should try to locate types which acts as roots of the state and hopefully only contains a few new instances. I guess this is what you did when identifying "unique object types".

As I understand it, none of these options are usable in your case. Are you updating the state "in-place"? E.g. are you doing something similar to:

Code: Select all

void AddNewState( object key, object value )
{
  aHashTable[key] = value;
}
where the hashtable is not cleared between updates. If this is the case, maybe you should look for old instance instead of new instances. I.e. instance that are "left-over" from the previous state, and not updated by the new state. If you are using .NET Memory Profiler 3.0 you will be able to add an instance filter which only show old instances in the "Type details" view. Of course, this may not work well if you have a lot of old instances.

Your suggestion about comparing instances will probably work well in your situation, and as I said previously, we might add a feature for this. But it will not be included in the 3.0 release, so you will have to implement it yourself in that version.

.NET Memory Profiler 3.0 will probably be released somewhat later than we originally planned, but the beta will probably be released before the end of October.
Best regards,

Andreas Suurkuusk
SciTech Software AB

nirva
Posts: 5
Joined: Wed Sep 06, 2006 3:43 am

Post by nirva » Sat Sep 09, 2006 6:07 pm

So now I'm doing this:

1) snapshot
2) connect to server, which fills up state
3) disconnect from server, which removes state
4) snapshot

The comparison shows +48,234 strings, and -48,225 strings

Looks like I'm leaking 9 strings.

The problem is when I look at class details page, I see allocations from many stack frames (over 100). Each stack looks something like this:

Live instances: 11; New live instances: 3; Allocs/sec: 0.04; Bytes/sec: 1.9
Live bytes: 432; New live bytes: 76

But it doesnt show the # of removed instances from that stack frame.

If I had that information, this would be much easier to figure out.

Andreas Suurkuusk
Posts: 1029
Joined: Wed Mar 02, 2005 7:53 pm

Post by Andreas Suurkuusk » Tue Sep 12, 2006 2:53 am

Hi,

Yes, you're correct that the number of removed instances for a call stack might help you locate your problem. As you have noticed, this information is not presented in the current version, but we will add as a high priority request in our new features list. Hopefully it will be included when we release the beta of .NET Memory Profiler 3.0.
Best regards,

Andreas Suurkuusk
SciTech Software AB

Post Reply

Who is online

Users browsing this forum: No registered users and 26 guests