NmpCore fails to start

[KevinH]

I am currently evaluating the Memory Profiler while trying to find an apparent memory leak on a ASP.NET website. Because this is on our production webserver, I am trying to use NmpCore to capture a session for evaluation. However, when I try to run NmpCore, I get a failed with time out error. (I'm including the full error at the bottom). I can run this just fine on our dev system, but since I don't know what page may have the leak, I really need to run this on production.

We are running on Windows Server 2008 R2, IIS 7.5. The app pool is .Net 4.0.

I am running the command

nmpcore /a 14464 /cs2 /ac5 /sf "C:\NmpCore\Sessions\DumpSession.prfsession"

Am I doing something wrong? Am I missing something? Any help would be appreciated.

Kevin

C:\NmpCore>nmpcore /a 14464 /cs2 /ac5 /sf "C:\NmpCore\Sessions\DumpSession.prfsession"
.NET Memory Profiler NmpCore Tool - Version 5.5.66.0
Copyright (C) SciTech Software AB. All rights reserved
Extracting temporary profiler files
Starting session
Attaching using API.
Process 'w3wp.exe' connected.
The operation failed with the following error:
Timed out while attaching to process '14464'.
Failed to delete temporary NmpCore files
Failed to delete temporary files at 'C:\Users\KEVINH~1\AppData\Local\Temp\SciTec
h\NmpCore-{54C2046D-3402-47AF-9CD2-174259EDE799}'.

[Andreas Suurkuusk]

Thank you for reporting this issue. We have confirmed that there is an issue when attaching to a process when NmpCore is running as an administrator. Unfortunately we have not yet been able to reproduce this in our development environment.

We are investigating this further and will hopefully come up with a solution soon. I will get back to you as soon as we have some more information.

[Andreas Suurkuusk]

After some further testing we can no longer reproduce this error. It seems like the .NET runtime can enter a state where it is not possible to attach to the process and the attach dead-locks somehow. To investigate this further we will need some logs from NmpCore. However, the logging from NmpCore is rather limited (compared to the full profiler), so we need to add some additional logging to it. During next week I will post a link to an updated version of NmpCore that includes the additional logging. This will hopefully guide us to the cause of this issue.

[KevinH]

OK, thank you for your help.

[Andreas Suurkuusk]

I'm sorry for the delay, but now we have created a build of NmpCore that performs detailed logging. You can download it from http://memprofilerwe.blob.core.windows. ... oreLog.exe

Run this version using the command line below:

NmpCoreLog /log:c:\MemProfilerLogs /a xxx /cs2 /ac5 /sf "C:\NmpCore\Sessions\DumpSession.prfsession"

This will create a set of log files in the c:\MemProfilerLogs directoy. If you send use those files, we can hopefully find out what's causing this problem.

Note that the log files will become very large in case you actually managed to start profiling. You should only use this version for troubleshooting.

[KevinH]

How should I send you the log files? Just attach them here?

[Andreas Suurkuusk]

It's probably best if you e-mail them to support@scitech.se. If they are too big to send by e-mail you can upload them to us instead. I will send an e-mail to you with instructions on how to upload files to us.

[KevinH]

The log files have been emailed.

[Andreas Suurkuusk]

Thank you for the log-files. With the help of them I have located the area where the attach hangs (and then the attach times out). It seems to be related to symbol lookup for loaded modules (hanging within an API call). Symbol lookup is something we have planned to improve/replace in a future version, but if it causes hangs we definitely must prioritize it.

However, this improvement will take some time, so in the meantime we need to come up with a work-around. We will look into this as soon as possible and I will get back here when I have some more information.