How To Associate MemProfiler Instance# to Instance Address

Use this forum for questions on how to use .NET Memory Profiler and how to analyse memory usage.
Post Reply
briddle
Posts: 6
Joined: Thu Mar 06, 2008 7:13 pm

How To Associate MemProfiler Instance# to Instance Address

Post by briddle » Mon Jan 23, 2012 6:34 pm

I am working on a dump with 14 million instances of a class type. I can see several instances of interest in MemProfiler v4. But I need to find these same instances in the dump via WinDbg to get further information. The Instance Number shown in MemProfiler is a straight decimal number, so I assumed that if I converted this number to hex, I should end up with the instance address and could take this into WinDbg and dump it via "!dumpobject [hex equivalent of MemProfiler Instance #"]. But this does not seem to work.

For System.Data.DataRowView instance displayed as "#28,085,692, hex equivalent is 1AC8DBC

If I take that into "!Dumpobject":

0:000> !do 1ac8dbc
<Note: this object has an invalid CLASS field>
Invalid object

How can I display the actual dump object instance address of an instance# shown in MemProfiler?

Andreas Suurkuusk
Posts: 1029
Joined: Wed Mar 02, 2005 7:53 pm

Re: How To Associate MemProfiler Instance# to Instance Addre

Post by Andreas Suurkuusk » Mon Jan 23, 2012 9:17 pm

The instance number presented by the profiler is just a sequential number assigned by the profiler for each instance, it is not related to the memory address of the instance. Instances can be moved in memory between snapshots and thus the memory address will change. If the memory address were used as the identifier, it would not be possible to identify instances between snapshots.

However, when importing memory dumps, things are a bit different. As you have noticed, the memory address would be useful to get additional info about an instance using the !DumpObject command in WinDbg. And since the profiler has no way of knowing how instances are moved in memory between two memory dumps, no instance tracking takes place, and a new instance number is assigned to all instances for each snapshot. So theoretically, the memory address could be used. But, to improve memory usage, the instance number is just a 32-bit number, which would prevent using the address in a 64-bit process.

To sum it up. No, there's currently no way of associating an instance number with the instance address. In the next version of the profiler (v4.1) we might add this information for snapshots collected using a memory dump. Unfortunately, this will add an additional 4 or 8 bytes to each instance in the snapshot.
Best regards,

Andreas Suurkuusk
SciTech Software AB

briddle
Posts: 6
Joined: Thu Mar 06, 2008 7:13 pm

Re: How To Associate MemProfiler Instance# to Instance Addre

Post by briddle » Tue Jan 24, 2012 2:28 pm

Thank you for considering adding the capability to get the instance addresses when importing a memory dump. I understand that enabling this feature would add to the memory demand. To tailor that impact, I suggest that you add a checkbox for this feature to the "Import Memory Dump" dialog. You could then either choose to use the decimal equivalent of the instance address as MemProfiler's instance identification number or perhaps to save the instance address as a separate property for the instance obtainable by right-click if not always displayed.

But it would be extremely valuable to be able to relate the information out of MemProfiler to further analysis via WinDbg.

The dump set I am currently analyzing has what looks like an event handler related leak and a data-binding link. I need to follow several pointer chains to provide enough information for the developer to identify the scenario causing the leaks. But with 14 million instances, a simple "!dumpheap -type" of one of the suspect class types literally took two days just to complete! There is no way I could ever run "gcroot" on any significant number of instances.

So getting all of the information through WinDbg (where I DO get the instance addresses) is not really an option. By importing the dump into MemProfiler and using your "roots" displays, I can narrow down the instances of interest. With this "instance address" feature, I could then follow the chains back in WinDbg without having to use WinDbg to initially identify the instances with the problem root.

In this scenario, it would also be extremely helpful if there was a way for you to add a feature to copy the entire, fully expanded "Field Values" treeview. (This is the popup from the popup dialog off of the "Type Details" tabthat has both "Field Values" and "Allocation Call Stack"). That dialog cannot be re-sized, so it is difficult to see enough of the "Field Values" tree at any one time to get a feel for the available data; particulary since I' not familiar enough with the application I'm analyzing to know what I'm looking for in terms of field names and tree node depth.

I would love to be able to right-click that "Field Values" tab and copy that whole, fully expanded treeview of Field Values data to the clipboard. I could then paste it into an EMail or Word document for the developer and I to look at the whole tree in total.

Thank you again for replying.

Andreas Suurkuusk
Posts: 1029
Joined: Wed Mar 02, 2005 7:53 pm

Re: How To Associate MemProfiler Instance# to Instance Addre

Post by Andreas Suurkuusk » Thu Jan 26, 2012 3:09 pm

Thanks for the update. We will probably add the option to include the address of an instance as an additional property when importing memory dumps. The profiler expects the instance numbers to be sequential, so it's probably not a good idea to use the address as instance identifier. Besides, it will not work well for 64-bit processes anyway.

May I ask what kind of information you want to extract using WinDbg, that is not available in the profiler?
Best regards,

Andreas Suurkuusk
SciTech Software AB

briddle
Posts: 6
Joined: Thu Mar 06, 2008 7:13 pm

Re: How To Associate MemProfiler Instance# to Instance Addre

Post by briddle » Wed Jun 20, 2012 5:01 pm

What we need that is only available back in WinDbg varies quite a bit. Sometimes we need to run a script on the object of interest; either a straight WinDbg script, a PowerDbg/PowerShell script, a custome WinDbg extension, etc. Sometimes we need printable output to share with the Dev group that owns the problem application. (MemProfiler is a great interactive tool, but has minimal ability to print data and root-reference-diagrams in a format we can easily share). And sometimes we need to just generally follow address links into and/or out of an object which MemProfiler shows as an instance of particular interest. This may be to evaluate associated locks, handles, communications sockets, thread stacks/storage, etc. to help determine why the object is in the state MemProfiler shows.

Andreas Suurkuusk
Posts: 1029
Joined: Wed Mar 02, 2005 7:53 pm

Re: How To Associate MemProfiler Instance# to Instance Addre

Post by Andreas Suurkuusk » Thu Jun 21, 2012 12:21 pm

Thanks for the information. You're correct that there is information that can be extracted using WinDbg that is not available in .NET Memory Profiler, so associating an instance with the instance address definitely makes sense. We're still planning to include the instance address when importing memory dump, but as you have noticed, this feature was unfortunately not included in version 4.5.

We have also planned to add better scripting support to the profiler. Currently scripting is possible by accessing the SciTech.NetMemProfiler.Core assembly, but no public documentation is yet available. This assembly is used by the profiler application itself and it has become pretty complex and is also being refactored occasionally, so it is not very suitable for public access. Maybe we will add a smaller API interface as a facade to SciTech.NetMemProfiler.Core.

Better reporting is another feature that has been planned for a long time, but has been postponed for several versions. Do you have any specific ideas on how you'd like the reporting to work?
Best regards,

Andreas Suurkuusk
SciTech Software AB

briddle
Posts: 6
Joined: Thu Mar 06, 2008 7:13 pm

Re: How To Associate MemProfiler Instance# to Instance Addre

Post by briddle » Mon Jul 09, 2012 5:41 pm

I'd like to be able to print the following; to help share them with the original developers, to project them at team meetings, and also to be better able to visualize them for myself (particularly in the case of the instance graph, which I can rarely see all of at once and in the case of hte "Info/Fields" tabbed display, which always seems to close on me before I can mouse down far enough to see what I need to see).

The main "Overview" grid with a header that includes the settings that tailored it (i.e.: "Show Types" settings, etc.)

The "Root Paths" display along a header that includes the settings that tailored it (i.e.: "truncate root paths" settings, etc.)

On the "Instance Details" display, the selected, or optionally all (one-at-a-time)the Root Path text description

On the "Instance Details" display, the selected, or optionally all (one-at-a-time) complete, non-truncated instance graph

Wherever it shows, the information on "Info/Fields" tabs. An option would allow you to either fully expand the "Fields" treeview or else just to print it showing the nodess expanded/collapsed just as I had currently made them via clicking or not clicking on the node plus/minus signs.

Andreas Suurkuusk
Posts: 1029
Joined: Wed Mar 02, 2005 7:53 pm

Re: How To Associate MemProfiler Instance# to Instance Addre

Post by Andreas Suurkuusk » Fri Jul 13, 2012 1:54 pm

Thank you for sharing your ideas about the reporting functionality. This is definitely something we will keep in mind when designing the reporting.

Note that you can copy information to the clipboard from the different views in the profiler. However, the way of doing it is unfortunately not the same everywhere:

The data in the Oveview grid (and all other grids) can be copied using the Edit->Copy all (Ctrl+Shift+C) command, or using the Copy all command in the context menu. To print the data you will probably need to paste it into Excel or similar. The current settings for the grid are currently not included.

Root paths and allocation stacks can be copied using the Edit->Copy (Ctrl+C) command, or using the Copy command in the context menu. Information in the info view is currently no included.

The full instance graph can be copied as a bitmap using the Edit->Copy (Ctrl+C) command (there's currently no Copy in the context menu).

The fields tree under Instance details can be copied using Edit->Copy (Ctrl+C) command, or using the Copy command in the context menu. The fields tree in popup windows can only be copied using the context menu. Only already expanded items will be copied (a fully expanded tree can be very big).

Finally you can copy the information presented in the info views by selecting the text (e.g. using Ctrl+A or the mouse) and pressing Ctrl+C (Edit->Copy cannot currently be used and there's currently no context menu).

So by copying data to the clipboard and pasting it into another program (e.g. Excel), you have the possibility to extract information for presentation. However, this can obviously made much easier by adding built-in reporting, the possibility to export to file and to print. And some of the information that might be useful, e.g. filter settings, is currently not included.
Best regards,

Andreas Suurkuusk
SciTech Software AB

briddle
Posts: 6
Joined: Thu Mar 06, 2008 7:13 pm

Re: How To Associate MemProfiler Instance# to Instance Addre

Post by briddle » Fri Feb 08, 2013 4:21 pm

Hi Andreas,

I just wanted to check the status of my request to include the address of an instance as an additional property when importing a memory dump. I've been fighting a low-memory condition for a large ASP.NET application for days. I've found a few paths of interest in MemProfiler's object graphs, but there are so many instances of the classes involved that I cannot find them in the dump. In this particular case, there are generic collections of interest. MemProfiler just shows them as List<T>. I'd need to find the actual instance to find out what class type is involved in the collection. (The mouse-over fly-out "Fields" display in MemProfiler just shows "No instance data collected). I'm having the same problem with some HashTables of interest - some of which seem to be rooting suspicious numbers of instances but there are so many in the dump I can't find the matching instances. Again, MemProfiler's Fields context fly-out shows "no instance data collected".

In our previous discussion on this feature, you were concerned about the amount of memory required to carry the instance address around; not to mention the GUI changes to access it and display it. I have a suggestion for a much lower impact way to achieve the same result. How about adding an option to simply write out a cross-reference file while you are importing a dump? As you assign each new instance your unique sequential number, just write out that index number and the underlying instance address to a disk file. The output format could be a simple flat file, a compressed binary file with matching simple search utility, or some searchable format like an .MDB. This solution would require no additional memory during any of the run-time for MemProfiler except a small amount during the "import" itself.

Yet the resulting file could still be used to find the corresponding dump address that matches a MemProfiler non-meaningful but unique sequential number for later use with WinDbg, PowerShell, PowerDbg, or Visual Studio. You could simply add the option as a checkbox and an output path text control to the "Import Memory Dump dialog. Or if you wanted the cheapest possible implementation, you could even add no GUI at all and make producing such a cross-reference file a commandline-only option.

You have such a great tool! But this would really help when bouncing back and forth with WinDbg. Please, please, please move this up higher on your "To Do" list.

Thanks!
-Bob
Last edited by briddle on Mon Feb 11, 2013 2:11 pm, edited 1 time in total.

Andreas Suurkuusk
Posts: 1029
Joined: Wed Mar 02, 2005 7:53 pm

Re: How To Associate MemProfiler Instance# to Instance Addre

Post by Andreas Suurkuusk » Mon Feb 11, 2013 1:11 pm

This feature will be included in .NET Memory Profiler 4.6. It was actually one of the first features implemented for the next version.

In version 4.6 you will get the option to include instance addresses when importing memory dumps.
IncludeAddress.png
Include instance address when importing dump
The instance address is presented as an additional column in the instances list.
AddressColumn.png
Instance address column in instance list
And in the instance details info panel.
InstanceAddress.png
Instance address in info view
The release of .NET Memory Profiler 4.6 has been somewhat delayed, but hopefully a beta will be available by the end of the month (February).
Best regards,

Andreas Suurkuusk
SciTech Software AB

briddle
Posts: 6
Joined: Thu Mar 06, 2008 7:13 pm

Re: How To Associate MemProfiler Instance# to Instance Addre

Post by briddle » Mon Feb 11, 2013 2:10 pm

Yeah! Thank you! This feature will help immensely when troubleshooting large dumps. I run 64-bit Win7 with 16GB RAM and am less concerned with memory consumption than with finding the objects of interest often flagged by MemProfiler.

Post Reply

Who is online

Users browsing this forum: Bing [Bot] and 14 guests