extracting snapshot data for trend analysis

Use this forum to discuss and/or suggest future improvements of .NET Memory Profiler and to report bugs.

Moderator: SciTech Software

extracting snapshot data for trend analysis

Postby svemo » Wed Oct 01, 2008 11:50 am

This question is related to unit testing, but I have a slightly different goal: I would like to collect memory usage data for every build (or perhaps weekly) of our application, store the information in a database, and later use the collected data to evaluate the performance of the application. That is, find out if the memory usage is increasing or decreasing over time.

I cannot find a way to do this automatically with the current API. The closest I get to exporting the data is by manually using Copy All in the Types page and the pasting into a text file.

I would be very useful if you could add API support for saving snapshot data in a well-defined simple format, such as CSV.

Sven
svemo
 
Posts: 1
Joined: Wed Oct 01, 2008 11:39 am

Postby Andreas Suurkuusk » Fri Oct 03, 2008 4:05 pm

The profiler API provided by MemProfiler2.dll is only intended to be used within the profiled process. But, ever since we released .NET Memory Profiler 3.0, we have planned to release an "external" API that provides much more control over the profiler. Due to refactorings and lacking documentation we have not yet released this API. However, we decided to make an undocumented (and unsupported) version of the API available. It has not been included in an official release yet, but an installer including the API can be downloaded from http://memprofiler.com/MemProfilerInstaller3_1_315.msi (or http://memprofiler.com/MemProfilerInsta ... -64bit.msi for the 64-bit version).

This version contains an assembly named SciTech.NetMemProfiler.Core.dll (in the "Assemblies" subfolder under the installation folder). Adding a reference to this assembly will provide access to the "external" .NET Memory Profiler API.

Below I have provided an example that will export some data from a profiler comparison to a CSV file.

Code: Select all
using System;
using SciTech.NetMemProfiler;
using System.IO;
using System.Globalization;

class Program
{
    static void Main( string[] args )
    {
      // Create a ProfilerApplicationCore instance to get
      // access to the profiler API.
      ProfilerApplicationCore application = new ProfilerApplicationCore();

      // Create a comparison for a session, using the third (2) snapshot as the selected
      // snapshot and the second (1) snapshot as comparison.
      using( ProfilerComparison comparison = application.CompareSnapshots( "<path to session>", 2, 1 ) )
      {
         // Get the compared types from the comparison (if the resource tracker was enabled,
         // GetComparedResources can also be used)
         ComparedTypeInfo[] comparedTypes = comparison.GetComparedTypes();

         // Write the compared types to a CSV-file (only
         // a subset of the ComparedTypeInfo properties are included, Intellisense
         // should provide a list of all available properties).
         using( StreamWriter writer = new StreamWriter( @"d:\Scrap\ProfilerSession.csv" ) )
         {
            writer.WriteLine(
               "\"Type name\"," +
               "\"Live instances\",\"Delta live instances\"," +
               "\"New live instances\",\"Removed live instances\"," +
               "\"Live bytes\",\"Delta live bytes\"" );
            foreach( ComparedTypeInfo comparedType in comparison.GetComparedTypes() )
            {
               writer.WriteLine( string.Format( CultureInfo.InvariantCulture,
                  "\"{0}\",{1},{2},{3},{4},{5},{6}",
                  comparedType.FullName,
                  comparedType.LiveInstancesCount, comparedType.DeltaLiveInstancesCount,
                  comparedType.NewLiveInstancesCount, comparedType.RemovedLiveInstancesCount,
                  comparedType.LiveBytesCount, comparedType.DeltaLiveBytesCount ) );
            }
         }
      }
   }
}


The example above requires knowledge about the available snapshots. It's also possible to load a session file explicitly and retrieve the available snapshots. The code below shows an example of that:

Code: Select all
class Program
{
   static void Main()
   {
      ProfilerApplicationCore application = new ProfilerApplicationCore();

      // Load the session.
      using( ProfilerSession session = application.LoadSession( @"<path to session>" ) )
      {
         // Retrieve the available snapshots.
         SnapshotHeader[] snapshots = session.SessionFile.GetSnapshotHeaders();

         // Select snapshots. Currently the last two are selected (assuming that
         // at least two snapshots exist)
         SnapshotHeader selectedSnapshot = snapshots[snapshots.Length - 1];
         SnapshotHeader comparisonSnapshot = snapshots[snapshots.Length - 2];

         using( ProfilerComparison comparison = application.CompareSnapshots(
            selectedSnapshot, comparisonSnapshot ) )
         {
            // Get the compared types from the comparison (if the resource tracker was enabled,
            // GetComparedResources can also be used)
            ComparedTypeInfo[] comparedTypes = comparison.GetComparedTypes();

            // Process the compared types
            // ...
         }
      }
   }
}


The API also allows you to start profiling a process. This provides more flexibility than can be acheived using command line arguments and profiler projects.

Code: Select all
class Program
{
   static void Main()
   {
      ProfilerApplicationCore application = new ProfilerApplicationCore();

      // Define the process that should be profiled.
      ProfileProcessStartInfo startInfo = new ProfileProcessStartInfo(
         ProfilingType.Application,
         "<path to executable>" );

      // Create a new session (ActiveSession is derived from ProfilerSession)
      using( ActiveSession session = application.CreateSession( startInfo ) )
      {
         // Start the profiling.
         session.Start( TimeSpan.FromMinutes( 1 ) );

         // Wait for a while.
         Thread.Sleep( TimeSpan.FromSeconds( 10 ) );         

         // And collect a snapshot.
         // NOTE! The "internal" API (MemProfiler.FullSnapShot) is probably more suitable
         // for collecting snapshots.
         session.CollectSnapshot( false );

         // Wait some more.
         Thread.Sleep( TimeSpan.FromSeconds( 10 ) );

         // And collect a second snapshot.
         session.CollectSnapshot( false );

         // Stop the profiling session (and terminate the profiled process).
         session.Stop( true );

         // Save the session (the second argument can be used
         // to specify which snapshots to include in the saved session
         // file).
         session.Save( @"C:\Scrap\NewSession.prfsession", null, true );
      }
    }
}


Currently, no documentation is provided about this API, apart from Visual Studio Intellisense (documentation partly exists, but it needs to be significantly improved). In the next version of the profiler we will provide some additional documentation, but we will not make the API "official", since we're planning on doing some changes to the API. The changes will mainly be name and namespace changes, not structural changes.

Hopefully you (and others) will find this API useful (even if the documentation is missing). You should at least be able to export some data to CSV.
Best regards,

Andreas Suurkuusk
SciTech Software AB
Andreas Suurkuusk
 
Posts: 959
Joined: Wed Mar 02, 2005 7:53 pm
Location: Sweden

Postby bralston7 » Tue Nov 11, 2008 7:57 pm

That last section of code seems almost like what I was going to start a new thread to request a feature for. I'd like to be able to attach to the currently running process if memprofiler is not running like this:

Code: Select all
if (!MemProfiler.IsProfiling)
{
   MemProfiler.AttachToCurrentThread(); //or something like that
}


Is there something in your undocumented API that can do that? If not, I'd like to see it.
bralston7
 
Posts: 6
Joined: Tue Sep 09, 2008 11:00 pm
Location: St. Louis, MO - USA

Postby Andreas Suurkuusk » Wed Nov 12, 2008 7:51 pm

Theoretically you could attach the profiler to your own process using the external API, but that would cause the profiler to profile itself as well, which would probably cause strange results.

In order for your proposed AttachToCurrentThread method to work, the profiled process would need to spawn a separate profiler. Additionally, no profiler data would be available in the profiled process, which would make the Fastsnapshot and Assertion methods hard to implement and probably very slow.

That said, we will improve the "attach to" feature in future versions, and we will take your suggestion in consideration. It's possible that we find a way of using the internal API in an attached process as well.
Best regards,

Andreas Suurkuusk
SciTech Software AB
Andreas Suurkuusk
 
Posts: 959
Joined: Wed Mar 02, 2005 7:53 pm
Location: Sweden


Return to Future Improvements

Who is online

Users browsing this forum: No registered users and 1 guest

SciTech Software logo

© Copyright 2001-2016. SciTech Software AB
All rights reserved.


SciTech Software AB
Kartvägen 21
SE-175 46 Järfälla
Sweden


E-mail: mail@scitech.se

Telephone: +46-706868081

cron