Session files CONSTANTLY getting corrupted

Use this forum for questions on how to use .NET Memory Profiler and how to analyse memory usage.
Post Reply
endian675
Posts: 5
Joined: Wed Apr 04, 2007 11:19 am

Session files CONSTANTLY getting corrupted

Post by endian675 » Thu Apr 05, 2007 4:48 pm

Hi

I am using MP3.0 to benchmark the memory usage of a very large application (500-1000mb of RAM). Frequently, MP hangs when creating a session file. The only way to release the system is to kill MP, but this corrupts the session file. It's not possible to back up the file before killing it, as the application has an exclusive lock and doesn't permit copying.

Are the session files backed up anywhere before writing starts? If not, then PLEASE PLEASE PLEASE PLEASE do so as losing session files that have taken 3 hours to create is NOT A GREAT WAY TO END THE WEEK.

On a related note, are you seriously telling me that a 178mb session file can be corrupted so badly that it's completely unreadable by the application? I find that impossible to believe. If that is the case then batch your session writes into multiple files so that at least data up until the failed write can be retrieved.

I recommend before you add ANY more functionality, you look at this issue with the utmost urgency. It's just a shame that I've already paid for the licence, based on good experiences with 2.6. If I'd used 3.0 for the trial period there's no way I would be buying it.

Andreas Suurkuusk
Posts: 1029
Joined: Wed Mar 02, 2005 7:53 pm

Post by Andreas Suurkuusk » Fri Apr 06, 2007 9:39 am

We have not previously heard about problems with the profiler hanging while saving session files. You are profiling an application with very high memory usage, and your session files are pretty big as well, maybe that causes problems in some way.

It is our intention to make v3.0 a better product than v2.6 in all aspects, and we hope that we will be able to make you feel that it was a good idea to upgrade to this version. We have put a a lot of effort into v3.0 of the profiler and it has been more thoroughly tested than the previous version. However, we have made some major refactoring of the code and we have included several advanced features, so there is still a risk that we have introduced some new bugs. However, the actual creation of session files has not been changed much, we just include some additional information, like the resource tracker information.

If there is a serious bug in the software we always make that our highest priority. We will fix that before working on additional features on the product. During the next week we will improve the way we save the session files and add additional logging to the save process, so that we will be able to identify why the profiler hangs while creating the session file.

Since session files can be very big, we try to avoid creating to many copies of them. We only have one copy of the session file while running a session, and that is just a temporary file containing the saved snapshots. It lacks some vital information needed to recreate the full session file (which is only kept in memory). When creating the actual session file, we retrieve the information from the temporary session file, delta pack the snapshot information, and finally we add vital file structure information (which is not known until everything is written). Thus we currently never have a valid session file until the session has been fully saved. This is something that we will improve for the next release.

We will get back to you when we have updated the session creation routines, but in the meantime it would be good if you could provide us with some additional information.

Was the resource tracker enabled when your problem occured?
When the profiler hung, was there any CPU or disk activity, or was the profiler just idle?
How many snapshots had you collected?
Did you get any error messages?
Best regards,

Andreas Suurkuusk
SciTech Software AB

endian675
Posts: 5
Joined: Wed Apr 04, 2007 11:19 am

Post by endian675 » Sat Apr 07, 2007 11:52 am

Andreas Suurkuusk wrote:We have not previously heard about problems with the profiler hanging while saving session files. You are profiling an application with very high memory usage, and your session files are pretty big as well, maybe that causes problems in some way.

We will get back to you when we have updated the session creation routines, but in the meantime it would be good if you could provide us with some additional information.

Was the resource tracker enabled when your problem occured?
When the profiler hung, was there any CPU or disk activity, or was the profiler just idle?
How many snapshots had you collected?
Did you get any error messages?
Hi Andreas

1. No, the resource tracker was not enabled when the problem occurred
2. There was no disk or cpu activity when the profiler hung, and even after leaving it for 10 minutes or so nothing happened.
3. I think I was creating the 4th snapshot
4. No error message, sadly. Is there any logging I can enable to help you investigate the problem? I checked the event log but there was nothing in there.

Thanks
mike

Andreas Suurkuusk
Posts: 1029
Joined: Wed Mar 02, 2005 7:53 pm

Post by Andreas Suurkuusk » Mon Apr 09, 2007 10:23 am

We're currently working on this problem, and we will hopefully have a fix by the end of the week. I just have one more question.

When you saved the session file, did you use the "File->Save session As" command, or did you save it when the session was ended?

A log file is created in the C:\Document and settings\<User>\SciTech\MemProfiler3" folder, but it currently does not include much information about the save process. We will add some more logging in the next release.
Best regards,

Andreas Suurkuusk
SciTech Software AB

endian675
Posts: 5
Joined: Wed Apr 04, 2007 11:19 am

Post by endian675 » Wed Apr 11, 2007 2:27 pm

I used the File->Save Session As menu item.

I've looked through C:\documents and settings but can't find a Scitech folder.

Andreas Suurkuusk
Posts: 1029
Joined: Wed Mar 02, 2005 7:53 pm

Post by Andreas Suurkuusk » Wed Apr 11, 2007 8:38 pm

I forgot to include "Application data" in the path in my previous message. The correct path is "C:\Document and settings\<User>\Application data\SciTech\MemProfiler3"

We have now updated the way snapshots and session files are saved. Whenever a snapshot is saved, all data necessary to read the session file is also saved. This will increase the size of the temporary session files, but in return you can always restore your session files if the profiler is prematurely terminated.

If any "unsaved" session files exists when you restart the profiler, the dialog below will be shown, allowing you to save the temporary session files to a new location.

Image

We have not been able to fully explain why you fail to save your session file, but we have found some minor threading issues that might have caused the problem.

We will release a maintenance version of .NET Memory Profiler 3.0 within a few days. This version will include the fixes and improvements related to the session files.
Best regards,

Andreas Suurkuusk
SciTech Software AB

endian675
Posts: 5
Joined: Wed Apr 04, 2007 11:19 am

Post by endian675 » Thu Apr 12, 2007 12:59 pm

Rock on, nice one.

Andreas Suurkuusk
Posts: 1029
Joined: Wed Mar 02, 2005 7:53 pm

Post by Andreas Suurkuusk » Wed Apr 18, 2007 12:19 pm

We have now improved the way session files are saved, and added the possibility restore session files that have not been correctly saved (e.g. due to the profiler terminating unexpectedly). The improvement are available in the latest release (v3.0.103), which can be downloaded from http://memprofiler.com/download.aspx.
Best regards,

Andreas Suurkuusk
SciTech Software AB

Post Reply

Who is online

Users browsing this forum: No registered users and 25 guests