Some pointers would be useful !
The problem occurs in a process that collects data from a DBMS. There are hundreds of queries being issued and the process runs for about 1 minute. There are no errors and all of the data is returned as expected. For testing the results are being discarded. None of the result sets are large.
During the routine, memory in use climbs from about 740Mb to 1440Mb so it is holding 700Mb. This is not expected as all of the objects and connections are I think properly disposed of after use. Worse still the memory is not freed at the end of the routine.
Sounds like a perfect application for MemProfiler.
I start the application under MemProfiler. When it is ready to go I take Snapshot 1 and everthing is fine. I cause the routine in question to execute. Memory in use climbs as before.
Then I take Snapshot 2. Immediately the extra memory is freed so comparing Snapshot 1 and 2 is pointless.
I understand that MemProfiler causes a GC before taking the snapshot. The weird thing is that doing System.GC.Collect() at intervals during the routine and even in an outer routine that calls it does not cause the memory to free up.
Even weirder, if I run the routine again in the same session, the memory build-up does not occur. It does everything that it should in exactly the same way with no memory utilisation peak like before.
This is perfectly repeatable.
Another observation is that when run outside of MemProfiler the application will hold the extra memory for quite a long time but then may release it with no apparent cause. Or the memory may be released when some other application is minimised. Putting in logging and stepping through the routine may also cause the problem to not occur or at least not manifest because possibly the memory is being reclaimed as it proceeds.
I don't think this is a traditional memory leak because GC can return the meory if it wants to. So what could be stopping it from doing so automatically and even when it is explicitly called in the routine in question ?
What should I look at next ? Can MemProfiler help ?
Below are some questions that might help you get closer to the solution of your problem:
Did you try to use the following sequence?
Code: Select all
GC.Collect(); GC.WaitForPendingFinalizers(); GC.Collect();
How much managed memory usage are you seeing? 700 MB of data seems very much, and I doubt that the profiler works very well if you have 700 MB of managed instances.
Did you use the dispose tracker to make sure thay you dispose your instances correctly? If you have instances that wrap unmanaged resources, then it is very important that you dispose them correctly.
Are you using large instances? Large instances use a separate heap which does not get compacted. If you have a lot of large instances then the memory may become very fragmented, and the runtime may fail to release the memory.
Have you looked at the real-time graph? Is the total instances count much higher than the live bytes count?
SciTech Software AB
Thank you for that suggestion.
As to the real cause I'll keep looking and follow-up on your other ideas.
By the way, can you get MemProfiler to collect a snapshot without perfroming GC ? That would seem to be the quickest way to find out what sort of objects are building up in such cases.
Thank you again.
You do this by enabling "Heap utilization tracking" in the Settings form. When "Heap utilization tracking" is enabled, you will have access to a new command: "Collect Gen #0 Heap Snapshot".
After collecting a heap snapshot using this command, you will also be able to see unreachable instances, which may be useful when you try to analyse your problem.
SciTech Software AB
Users browsing this forum: No registered users and 16 guests