Importing memory dump taking too long

[chai]

Using v3.5.131 of the profiler, I am trying to import a full memory dump (mini) that is 835 MB in size, of a 32 bit .NET 2.0 standalone app. The import has taken over 1 hour but is still "traversing heap instances." It appears to me that it is stuck as the progress indicator has not moved since having come halfway right at the beginning of the import, but the DbgEngProxy.exe is running full steam in the background. What can be happening here?

I was able to import a dump of size 600 MB of the same application though -- this took 11 minutes for the profiler (versus 60 minutes and still running, now). Can you help me out here? I read in previous posts that the traversal is done by Windows debug diag tools -- I presume that would work fine when run independently...is this possible?

I'm afraid I won't be able to share the dump with you, though. Can I find a profiler-generated log that I can send you?

[Andreas Suurkuusk]

As I assume you've read in the other post, the "Traversing heap instances" step is a single operation that is performed by the debugging tools. The profiler has no way of determining the estimated duration of this action, and therefore the progress bar does not correctly indicate the actual progress. Unfortunately, this operation can be very slow, especially in memory dumps with many instances.

DbgEngProxy is just a helper process that has loaded the debugging tool libraries; it's not actually executing any code itself when traversing instances. If you use the debugging tools directly you will also see this long duration. You can try by this by running the "!TraverseHeap" command in sos:

• Start WinDbg
• Load the memory dump
• Run the command ".loadby sos mscorwks"
• Run the command "!TraverseHeap c:\Scrap\traverse.dat"

We are looking into ways of improving this and have also discussed this with Microsoft, but so far we have not come up with a good solution.

Did the import finish eventually?

[chai]

I had to cancel the import due to resource requirements for other tasks. I will try !TraverseHeap and let you know if that works, and in how long a time.

[chai]

running TraverseHeap under Windbg took about 1 hour to dump the output. So, that confirms that the TraverseHeap itself has the bottleneck.

[akraus1]

I have 64 bit machines with leaking process which can become up to 25 GB of size. Currently I am analyzing only a "small" 4GB Dump file which is now loading already over 1 1/2 hour. My colleague has told me something about 20h of load time for such a dump. This is prohibitively slow. But there is no other way to analyze these issues. Are there any updates from MS to speed things up?

Yours,
Alois Kraus

[Andreas Suurkuusk]

At the moment there are no faster way of importing memory dumps that I know of. Unfortunately, this is still very slow in .NET Framework 4.0. In the next version of the profiler we will look into ways of speeding up the memory dump import, possibly by sacrificing some functionality (like identifying all GC handle roots). In the meantime, I'm afraid that I don't have any suggestions on how to speed up the memory dump import.