trying to decrease memory footprint

Use this forum for questions on how to use .NET Memory Profiler and how to analyse memory usage.
Post Reply
dave123

trying to decrease memory footprint

Post by dave123 » Fri Feb 03, 2006 4:07 pm

Hi,
I'm trying to understand where all the memory is going in my application. I don't believe I have any memory leaks, however, the memory footprint is much larger than I would expect. Basically, I’m parsing some large (30 meg) binary files in a tight loop. I read the files at 64K at a time, until I find a specific byte array in the file. Then I return the byte array accumulated to that point. However for a 30meg file, my memory footprint is around 200megs. Something I don’t want.

When using your profiler, everything looks pretty decent, except for the "Native Memory" tab. Here’s what I see:

Code: Select all

Private - 235756KB
	Managed heaps - 1156KB
		Normal heap - 507KB
			Generation #0 - 304KB
				Data - 0KB
				"Holes" - 304KB
			Generation #2 - 203KB
				Data - 200KB
				"Holes" - 3KB
		Large heap - 600KB
			Large heap - 18KB
			Large heap overhead - 582KB
		Overhead - 49KB
			Extra - 45KB
			Other - 4KB
	Code - 681KB
		diasymreader.dll - 4KB
		mscordbc.dll - 4KB
		MSCORJIT.DLL - 4KB
		mscorsn.dll - 4KB
		mscorwks.dll - 4KB
		SHELL32.dll - 4KB
		urlmon.dll - 4KB
		WS2_32.DLL - 4KB
		zcs.exe - 4KB
		<Other> - 645KB
	JIT - 307KB
		aspNetEmail.dll - 4KB
		aspNetIMAP.dll - 3KB
		aspNetMime.dll - 26KB
		aspNetPOP3.dll - 2KB
		CommonLanguageRuntimeLibrary - 182KB
		RootIdTrigger.dll - 1KB
		System.Data.dll - 11KB
		System.dll - 66KB
		System.Web.dll - 0KB
		System.Xml.dll - 12KB
		zcs.exe - 0KB
		<Other> - 0KB
	Win32 heaps - 3048KB
	Other data - 229816KB
		ADVAPI32.dll - 4KB
		COMCTL32.DLL - 4KB
		CRYPT32.dll - 4KB
		diasymreader.dll - 4KB
		KERNEL32.dll - 12KB
		mscordbc.dll - 4KB
		mscoree.dll - 4KB
		MSCORJIT.DLL - 8KB
		mscorsn.dll - 4KB
		mscorwks.dll - 76KB
		MSVCR70.dll - 20KB
		msvcrt.dll - 4KB
		ntdll.dll - 8KB
		ole32.dll - 16KB
		OLEAUT32.dll - 8KB
		RPCRT4.DLL - 4KB
		SHELL32.dll - 4KB
		urlmon.dll - 4KB
		VERSION.dll - 4KB
		vmsrvc.dll - 12KB
		WLDAP32.DLL - 8KB
		WS2_32.DLL - 4KB
		<Other> - 229596KB
	System - 748KB
Shared - 1668KB
	Managed heaps - 0KB
		Normal heap - 0KB
	Code - 1588KB
		ADVAPI32.dll - 16KB
		CRYPT32.dll - 8KB
		KERNEL32.dll - 156KB
		mscoree.dll - 8KB
		MSCORJIT.DLL - 260KB
		mscorlib.dll - 8KB
		mscorsn.dll - 8KB
		mscorwks.dll - 756KB
		MSVCR70.dll - 16KB
		msvcrt.dll - 8KB
		ntdll.dll - 112KB
		ole32.dll - 80KB
		OLEAUT32.dll - 24KB
		RPCRT4.DLL - 64KB
		SAMLIB.DLL - 4KB
		SHELL32.dll - 4KB
		urlmon.dll - 12KB
		USER32.DLL - 12KB
		VERSION.dll - 4KB
		vmsrvc.dll - 16KB
		WLDAP32.DLL - 4KB
		<Other> - 8KB
	Win32 heaps - 4KB
	Other data - 76KB
		ADVAPI32.dll - 4KB
		GDI32.dll - 4KB
		KERNEL32.dll - 4KB
		mscoree.dll - 4KB
		MSVCR70.dll - 4KB
		msvcrt.dll - 8KB
		ntdll.dll - 4KB
		ole32.dll - 4KB
		RPCRT4.DLL - 4KB
		sorttbls.nls - 4KB
		USER32.DLL - 4KB
		<Other> - 28KB
Potentially shared - 452KB
	Managed heaps - 0KB
		Normal heap - 0KB
	Code - 432KB
		aspnetmime.dll - 44KB
		diasymreader.dll - 60KB
		KERNEL32.dll - 4KB
		mscordbc.dll - 20KB
		MSCORJIT.DLL - 4KB
		mscorlib.dll - 36KB
		mscorwks.dll - 200KB
		MSVCR70.dll - 44KB
		ntdll.dll - 8KB
		rootidtrigger.dll - 4KB
		zcs.exe - 8KB
	Other data - 20KB
		aspnetmime.dll - 4KB
		mscorlib.dll - 4KB
		mscorwks.dll - 4KB
		rootidtrigger.dll - 4KB
		zcs.exe - 4KB
It looks like all of my memory is in
<Other> - 229596KB

How do I reclaim that memory?

My biggest objects are the byte arrays. However, when looking at the class details of the byte arrays, here’s all I see.

Code: Select all

#1,517	2	65548	433
#3,065	1	2060	430
#3,055	1	1778	430
#1,501	1	524	3
#3,093	1	268	430
#1	4	268	3
#1,495	1	268	3
#1,512	1	268	3
#3,083	1	204	430
#1,649	2	140	433
#305	1	76	434
#3,112	1	76	430
#349	1	76	434
#379	1	76	434
#290	1	76	434
#3,073	1	44	431
#3,064	1	44	431
#3,063	1	44	431
#3,091	1	44	430
#3,110	1	44	430
#3,101	1	44	430
#3,118	1	44	430
#3,111	1	44	430
#3,092	1	44	430
#738	1	28	434
#2,268	1	28	432
#3,103	1	18	430
#3,142	1	14	3
#63	3	12	434
#387	1	12	434
Where is the memory going, and how can I get it back? What can I do to help you help me with my memory footprint?

Thanks,
Dave

Andreas Suurkuusk
Posts: 1029
Joined: Wed Mar 02, 2005 7:53 pm

Post by Andreas Suurkuusk » Sat Feb 04, 2006 10:02 am

Your application obviously uses a very large amount of unmanaged memory. Currently it is not possible to use the profiler to get any additional information about this memory, and therefore it is hard to know what's causing the memory usage.

However, we are currently working on an unmanaged resources tracker, which will be released as a preview version at the end of February. This tracker should provide you with much more detailed information about the unmanaged memory usage of your application.

If possible, it would be very good if you could provide us with your application and a test binary file. This would allow us to test our resources tracker before we release, and to provide you with information about the memory usage of your application.

If you have the possibility to provide us with a runnable application, please contact us at support@scitech.se.
Best regards,

Andreas Suurkuusk
SciTech Software AB

Post Reply

Who is online

Users browsing this forum: No registered users and 22 guests