Value types memory summary
Value types memory summary
Hi, is there any way to see the total memory consumed by value types?
i.e. I'd like to see that my snapshot had x decimal values consuming x*16 bytes
i.e. I'd like to see that my snapshot had x decimal values consuming x*16 bytes
-
- Posts: 1029
- Joined: Wed Mar 02, 2005 7:53 pm
Re: Value types memory summary
No, there's no functionality to present the number of bytes used by value types within other classes. The profiler will only present information about boxed value type instances.
It would be fairly easy to retrieve this information from a snapshot, but I don't know how it would be presented in a good way, and I'm not sure how valuable this information is.
If this information is important to you, I can help write a small helper console application that retrieves this information using the external profiler API. Is this something you would be interested in?
It would be fairly easy to retrieve this information from a snapshot, but I don't know how it would be presented in a good way, and I'm not sure how valuable this information is.
If this information is important to you, I can help write a small helper console application that retrieves this information using the external profiler API. Is this something you would be interested in?
Best regards,
Andreas Suurkuusk
SciTech Software AB
Andreas Suurkuusk
SciTech Software AB
Re: Value types memory summary
Hi Andreas, yes this would be helpful to me.
FYI I'll outline the reasons I am after this information.
I have a fairly large application where the memory foot print is often ~2GB. A significant proportion of this memory is used by types containing many BCL value types that are significantly more precise than required. e.g. DateTimeOffset and decimal.
For the vast majority of decimal values, I could substitute a custom value type that was 4 bytes instead of 16 that would hold the full range of required values, reducing the memory requirement to 25% of the original. Similarly for DateTimeOffset.
I am trying to understand the impact of the overhead from the use of these types where a smaller custom value type would suffice. I know that I can calculate it manually per class, but I have many classes overall and in this case I know that the vast majority of DateTimeOffset and decimal values will be candidates for smaller types.
All I really need at this stage is a total count of the instances and bytes consumed for all value types (or at least in this case DateTimeOffset and decimal).
regards,
Eamon
FYI I'll outline the reasons I am after this information.
I have a fairly large application where the memory foot print is often ~2GB. A significant proportion of this memory is used by types containing many BCL value types that are significantly more precise than required. e.g. DateTimeOffset and decimal.
For the vast majority of decimal values, I could substitute a custom value type that was 4 bytes instead of 16 that would hold the full range of required values, reducing the memory requirement to 25% of the original. Similarly for DateTimeOffset.
I am trying to understand the impact of the overhead from the use of these types where a smaller custom value type would suffice. I know that I can calculate it manually per class, but I have many classes overall and in this case I know that the vast majority of DateTimeOffset and decimal values will be candidates for smaller types.
All I really need at this stage is a total count of the instances and bytes consumed for all value types (or at least in this case DateTimeOffset and decimal).
regards,
Eamon
-
- Posts: 1029
- Joined: Wed Mar 02, 2005 7:53 pm
Re: Value types memory summary
OK, now I understand this requirement better. I will write an example program that extracts this information from a session file. I will post it as a reply to this topic, but it may be a few days before I get the time to write the program.
Best regards,
Andreas Suurkuusk
SciTech Software AB
Andreas Suurkuusk
SciTech Software AB
-
- Posts: 1029
- Joined: Wed Mar 02, 2005 7:53 pm
Re: Value types memory summary
Now I have written an example that extracts value type usage information from a session. It was a little bit more complicated than I expected. You will find the sample code below. To compile the code, you need to add a reference to the "SciTech.NetMemProfiler.Core" assembly, which you will find in the directory "C:\Program Files\SciTech\NetMemProfiler5\Assemblies".
I hope you find this sample useful and if you have any questions, you can reply to this post.
I hope you find this sample useful and if you have any questions, you can reply to this post.
Code: Select all
using SciTech.Metadata;
using SciTech.Metadata.Managed;
using SciTech.Profiler;
using SciTech.Profiler.Memory;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Reflection;
internal class Program
{
private static void Main(string[] args)
{
ProfilerApplicationCore application = new ProfilerApplicationCore();
// Load the session..
using (ProfilerSession session = application.LoadSession(@"<path to session>"))
{
// Retrieve the available snapshots.
SnapshotHeader[] snapshots = session.SessionFile.GetSnapshotHeaders();
var process = session.GetProfiledProcesses().FirstOrDefault();
if (snapshots.Length > 0 && process != null)
{
// Name of the types for which field memory information is wanted.
string[] fieldTypeNames = {
"System.Decimal,mscorlib",
"System.TimeSpan,mscorlib",
"System.DateTime,mscorlib" };
using (ProfilerComparison comparison = application.CreateComparison())
{
// Select snapshots. Currently only the last one is selected, and no comparison snapshot is used.
ProcessSnapshotHeader selectedSnapshot = snapshots[snapshots.Length - 1].Processes.FirstOrDefault(s => s.ProcessId == process.Id);
ProcessSnapshotHeader comparisonSnapshot = ProcessSnapshotHeader.Empty;// snapshots[snapshots.Length - 2].Processes.FirstOrDefault(s => s.ProcessId == process.Id);
// A comparison needs to be made to allow instance information to
// be extracted from the session.
comparison.CompareSnapshots(selectedSnapshot, comparisonSnapshot);
// Create the extractor that is used to extract field information
var extractor = new FieldUsageExtractor(fieldTypeNames, comparison);
// Extract and write the field information.
var fieldsInformation = extractor.GetValueTypeFieldInformation();
FieldUsageExtractor.WriteFieldInformation(fieldsInformation);
}
}
}
}
/// <summary>
/// Helper class that extracts information about fields of selected types.
/// </summary>
public class FieldUsageExtractor
{
private readonly ProfilerComparison comparison;
private readonly ProfiledProcess process;
/// <summary>
/// Cache used when extracting information about type fields.
/// </summary>
private readonly Dictionary<RuntimeType, List<TypeFieldInformation>> typeFieldsCache = new Dictionary<RuntimeType, List<TypeFieldInformation>>();
private bool fieldInformationExtracted;
/// <summary>
/// Dictionary for holding the results of the field information extraction. The keys
/// in the dictionary are the field types for which usage information is wanted.
/// </summary>
private Dictionary<ManagedType, List<TypeFieldInformation>> valueTypes = new Dictionary<ManagedType, List<TypeFieldInformation>>();
public FieldUsageExtractor(IReadOnlyList<string> valueTypeNames, ProfilerComparison comparison)
{
this.comparison = comparison;
this.process = comparison.PrimaryProcess;
foreach (var valueTypeName in valueTypeNames)
{
ManagedType valueType = this.process.GetTypeNameCache().GetMatchingType(valueTypeName, false);
if (valueType != null)
{
this.valueTypes.Add(valueType, new List<TypeFieldInformation>());
}
}
}
public static void WriteFieldInformation(Dictionary<ManagedType, List<TypeFieldInformation>> fieldsDictionary)
{
foreach (var pair in fieldsDictionary)
{
var fieldType = pair.Key;
var fieldsList = pair.Value;
int totalCount = fieldsList.Sum(f => f.Count);
Console.WriteLine($"{GetShortTypeName(fieldType)} (total instances: {totalCount})");
foreach (var field in fieldsList)
{
string fieldName = !string.IsNullOrEmpty(field.Name) ? field.Name : " (instance)";
Console.WriteLine($" {GetShortTypeName(field.DeclaringType.ManagedType)}{fieldName}: {field.Count}");
}
}
}
/// <summary>
/// Extracts field usage information based on the types provided to the constructor.
/// </summary>
/// <returns></returns>
public Dictionary<ManagedType, List<TypeFieldInformation>> GetValueTypeFieldInformation()
{
if (!this.fieldInformationExtracted)
{
foreach (var instanceId in this.comparison.EnumInstances(null, true))
{
TypeInstance instance = this.comparison.GetTypeInstance(instanceId);
var rtType = instance.RuntimeType;
if (rtType.IsArray)
{
var valueTypeFields = this.ExtractValueTypeFields(rtType);
int nComponents = instance.GetArrayUsage()?.ComponentsCount ?? 0;
foreach (var valueTypeField in valueTypeFields)
{
valueTypeField.Count += nComponents;
}
}
else
{
var valueTypeFields = this.ExtractValueTypeFields(rtType);
foreach (var valueTypeField in valueTypeFields)
{
valueTypeField.Count++;
}
}
}
this.fieldInformationExtracted = true;
}
return this.valueTypes;
}
private static string GetShortTypeName(ManagedType managedType)
{
return managedType.ToString(SyntaxLanguage.CSharp,
TypeNameOptions.IncludeNamespaceAndName
| TypeNameOptions.UseBuiltInTypeNames
| TypeNameOptions.UseBuiltInInternalTypeNames
| TypeNameOptions.IncludeGenericArguments);
}
/// <summary>
/// Helper method for ExtractValueTypeFields. Appends relevant fields to the provided fieldsList
/// </summary>
/// <param name="rtType"></param>
/// <param name="outerType"></param>
/// <param name="baseFieldName"></param>
/// <param name="fieldsList"></param>
private void AppendValueTypeFields(RuntimeType rtType, RuntimeType outerType, string baseFieldName, List<TypeFieldInformation> fieldsList)
{
foreach (var field in rtType.EnumFields(BindingFlags.Instance | BindingFlags.Public | BindingFlags.NonPublic))
{
var fieldType = field.FieldType;
if (fieldType != null)
{
var managedFieldType = fieldType.ManagedType;
string fullFieldName;
if (!string.IsNullOrEmpty(baseFieldName))
{
fullFieldName = $"{baseFieldName}.{field.Name}";
}
else
{
fullFieldName = $".{field.Name}";
}
if (this.valueTypes.TryGetValue(managedFieldType, out var wantedTypeList))
{
var valueTypeField = new TypeFieldInformation(managedFieldType, outerType, fullFieldName);
wantedTypeList.Add(valueTypeField);
fieldsList.Add(valueTypeField);
}
else if (fieldType.IsValueType && !fieldType.IsPrimitive)
{
this.AppendValueTypeFields(fieldType, outerType, fullFieldName, fieldsList);
}
}
}
}
/// <summary>
/// Extracts fields information for the specified type. Caches the result in <see cref="typeFieldsCache"/>.
/// </summary>
/// <param name="rtType"></param>
/// <returns></returns>
private IReadOnlyList<TypeFieldInformation> ExtractValueTypeFields(RuntimeType rtType)
{
if (!this.typeFieldsCache.TryGetValue(rtType, out var fieldsList))
{
fieldsList = new List<TypeFieldInformation>();
if (rtType.IsArray)
{
// Array types are handles separately. If they
// are arrays of value types, they may contain fields that we're interested in.
if (rtType.ElementType != null && rtType.ElementType.IsValueType)
{
var elementTypeFields = this.ExtractValueTypeFields(rtType.ElementType);
foreach (var f in elementTypeFields)
{
var arrayFieldInformation = new TypeFieldInformation(f.FieldType, rtType, f.Name);
fieldsList.Add(arrayFieldInformation);
var arrayTypeList = this.valueTypes[arrayFieldInformation.FieldType];
arrayTypeList.Add(arrayFieldInformation);
}
}
}
else if (this.valueTypes.TryGetValue(rtType.ManagedType, out var directTypeList))
{
var directTypeField = new TypeFieldInformation(rtType.ManagedType, rtType, "");
fieldsList.Add(directTypeField);
directTypeList.Add(directTypeField);
}
else
{
this.AppendValueTypeFields(rtType, rtType, "", fieldsList);
}
this.typeFieldsCache.Add(rtType, fieldsList);
}
return fieldsList;
}
}
public class TypeFieldInformation
{
public TypeFieldInformation(ManagedType fieldType, RuntimeType declaringType, string fullFieldName)
{
this.FieldType = fieldType;
this.DeclaringType = declaringType;
this.Name = fullFieldName;
}
public int Count { get; set; }
public RuntimeType DeclaringType { get; }
public ManagedType FieldType { get; }
public string Name { get; }
}
}
Best regards,
Andreas Suurkuusk
SciTech Software AB
Andreas Suurkuusk
SciTech Software AB
Re: Value types memory summary
Thanks Andreas, it is very much appreciated. I will try to find time later today to test it.
regards,
Eamon
regards,
Eamon
Who is online
Users browsing this forum: No registered users and 22 guests