Investigating memory traffic

[orangy]

Not sure if this is feature request, or I just didn't find how to do this with current version...

I'm investigating high memory traffic in Gen0, when lots of shot-life objects are allocated and die almost immediately. I'm looking at Realtime Allocs/sec graph, but I can't understand where I can see what objects were allocated at the given moment in time (or range). I would like to select some time on the graph and on the Types page see what objects were allocated there, how much, etc. Then, I would like to see Allocation call stacks for those objects, so that I can reduce unnecessary allocations.

Hmm, now I found help topic on Session Time Selector, but it looks like there is bug that prevents Type data to be updated when time range changes. Or may be this is because I tried to play with Peak snapshot...

Any help will be greatly appreciated

[orangy]

I would also like to open Call stacks/Methods tab and see functions ordered by Total Memory Traffic they produced during whole session, not just what is live at the moment of snapshot capture. I.e. Total Allocations and Total Allocated Bytes.

[Andreas Suurkuusk]

The session time selector does allow you to view allocation information for any Type at any point in time. However, to prevent excessive memory usage, real-time data is reduced when a set memory threshold is reached. The data is reduced by removing older data. If the time selector is moved to a point in time where data has been reduced, no Type information will be shown. Types that are included in the Graph, i.e. types that are selected in the Types list, will be much less reduced. So if you have a set of types you want to investigate, select them in the Types list and you will get better real-time information.

You can also change the real-time data collection session settings. The default settings keeps 1 MB of real-time data for selected types and statistics, and 2 MB for all other types. Increase the value for "Maximum memory to use for types not in graph" to get more real time data for all types in the list (see screenshot below).

RealTime.png (34.7 KiB) Viewed 19984 times

The real-time view is not based on snapshots and is separate from the other snapshots views. So it's not possible to get allocation information or other memory details at a specific point in time. To get these details, a snapshot has to be collected.

If you to get information about the generations of allocations, you can try to enable the heap utilization tracker. The heap utilization tracker can be enabled by selecting the "High" profiling level (note that this will have a significant performance impact).

You can get information about all allocations performed by a method by selecting "Empty" as the comparison snapshot, and deselecting "Show allocs and bytes per second" (under the View menu). This will cause all "Allocs" information (instances and bytes) to include all allocations performed since the start of the profiling.

[orangy]

Thank you for detailed explanation! Method with "per second" switched off is my favorite view now to analyse memory traffic

However, real-time graph and type information doesn't sync with each other for me. No matter what time I select on graph and zoom into, I always see the last numbers I had when stopped profiling.

Also, I would appreciate "pause" button, so that I can pause application and examine current real-time data and decide if I need snapshot or not.

[Andreas Suurkuusk]

The real-time Types view is not affected by the selected range in the graph. To change the time of the Types data, the time selector must be used. The time selector is only available in a stopped (or loaded) session and it appears as a cyan-colored vertical line. As soon as the session is stopped, it should appear in the graph. However, I just noticed that this is not the case in the current beta. To see the time selector, the session has to be saved and re-reloaded. We will fix this in the next release.

A pause button is a good idea and is something we will most likely implement in the next version. While the graph is paused, the time selector can be available even in an active session. Currently, the time selector is only a single point in time, and not a range. Data that is presented as an average (Allocs/sec, Bytes/sec) is always the average value over the last second (before the time selector). In the next version the averaging range will also be selectable.

[orangy]

I see, waiting for bugfix. Also, after stopping (and saving) session, window caption should change from name of the process to name of snapshot. And it also would be nice to see what portion of time graph is saved for detailed type view.

In fact, I'd like something like Record button, so I can press and depress it as needed and for this period of time Type allocation statistics would be saved. I'm profiling lengthy process, and I'd like to have statistics for few time ranges for further analysis.

[Andreas Suurkuusk]

We've just released version 4.0.57. You can download it from the beta page. It includes a fix for the missing time selector.

Allowing the real-time collector to be enabled/disabled during the profiling session would be a useful feature. Actually we originally (in version 2.0) intended to include a feature like that. However, due to the complexity in how real-time data is stored, reduced, and compressed, this feature was was abandoned. Since then, the real-time data collection has been redesigned and we will look into the possibility to implementing it in a future version.

After you have stopped an active session, the session is still active, even if you have saved a session file. For instance, you can restart the session by clicking the Start button, and you might have snapshots is the active session that were not saved in the file. But, you are right, some kind of indication of the session file would be a good idea. I have added this as a feature request.

Indicating the range over which Types real-time is available is also a good idea and I have added this as a feature request.

[orangy]

Also found small bug in Methods view - if I sort by Alloced bytes (per sec is switched off), and then start expanding callers/calles they temporary exchange places while expansion is being performed and then comes back. A bit annoying.

For memory traffic, it would be really great if I could compare two snapshots from different sessions in terms of call stacks and allocation statistics, so I can see gain/loss after optimization I performed. No instance details needed here, just Overview, Methods and Real-time tabs are enough for such a comparison view.

Another thing I'm missing while analyzing allocation patterns in our application, is "thread grouping". Our application is heavily multithreaded, and I would like to see allocation stacks per thread (including its name!) . Ideally, that would be grouping on Methods view, in addition to "Show hierarchical". Something like "Group by thread". Also, often I don't need grouping by assembly, only by namespace. So, I'd like to have 3 separate groupings -- by assembly, by namespace, by thread, so that I can select whichever combination I want.

And the last one for today I'm getting familiar with Heap Utilization analysis. First, it is really slow in my case, up to 5 times slower than without it. Do not know if it is ok or not. Second, I don't quite get this "Alloced in GenX", I thought allocations are always done in Gen0 (except for LOH), and the objects gets promoted to other generations. Is it really "Promoted to GenX"? In fact, I actually need "Collected in GenX" so that I can analyse "mid-life crisis". We call it "mid-life crisis" when objects are promoted to Gen1 or Gen2 and die there soon enough (in the next few collections of that generation).

Ah, not really last On the Real-time view in the Types tab there is context menu on type featuring "Show instance details" item, which does nothing for me.

Thank you very much for all your support, by using your profiler we've reduced memory traffic and improved allocation patterns a lot!

[Andreas Suurkuusk]

Thanks for the information about the bug in the methods view. We will try to fix that for the next release.

It is possible to compare two snapshots from two different sessions using the project explorer, but the comparison data will not include any allocation information, just delta live instances information. So, this feature cannot be used for your memory traffic analysis. We have had a low prority feature planned for quite some time, were you would get better information when you compare snapshots from different sessions. To get useful "Allocs/sec" or "Allocs count" information, you would need to select a base snapshot for each session, i.e. it would require you to select two pairs of snapshots, one pair one session and one from the other. This feature is still planned, but we have not decided for which version we will implement it.

In version 3.5 you can select to track allocations by thread, and then you can use the AppDomains/Thread selector to select which thread you want to investigate. In version 4.0 we removed this feature since it could cause excessive memory usage (if you have a lot of short-lived threads, a lot of memory would be used to keep track of the thread allocation information). Actually, the feature has only been removed from the user interface, the implementation still exists internally. We might update this feature and include it again in a new version, e.g. by only tracking named threads, and merging multiple threads that have the same name.

Heap utilization tracking is very slow. It analyses all instances on the heap for each GC, which effectively prevents any optimization by the generational garbage collector. You are correct that all allocations occurs in the Gen# 0 heap (or the large heap), but the "Allocs in gen" number counts instances that ends up in a specific generation. If you allocate an instance and it gets immediately GCed (i.e. it never get promoted to gen #1), then it will be counted as a Gen #0 allocation. It the instance gets promoted to gen #1 but GCed before it gets promoted to gen #2, then it will counted as a gen #1 allocation. And if the instance is still alive when you collect the snapshot, then the generation it lives in at the time of the snapshot will be used when counting the allocation generation. This will give you some information on how well the GC generations are used, e.g. if you do a lot of short-lived allocations, you only want to see a few gen #1 allocations and no gen #2 allocations, but it is not perfect for analysing mid-life crisis. One of the major features planned for version 4.1 is better presentation of mid-life crisis allotions.

The "Show instance detail" menu choice should not exist in the Real-time types view, we will remove it in the next release.

[KerryNorman]

thank you for sharing this information it was quite helful.