memory not recovering

Use this forum for questions on how to use .NET Memory Profiler and how to analyse memory usage.
Post Reply
shan
Posts: 5
Joined: Mon Jul 24, 2006 7:02 am

memory not recovering

Post by shan » Tue Jul 25, 2006 1:22 pm

hi
i need ur help
my application is a simple web application which fetch data on the users request .
when i run my application with the profiler , the follwoing statistics it shows
1) in task manager memory is 261 mb
2) in the profiler (physical memory) is 166 ( with profiler 261)
private 147 mb
managed heap 5 mb
code 2
win32 24
OTHER DATA 112 mb (what is this??)
3) in snapshot live instances are
byte[] 2.3 mb
string 0.9mb
hashtable.bucket[] 01mb
4) in the perfmon.exe
private bytes (aspnet_wp.exe - process) 293 mb
bytes in all heap (aspnet_wp.exe - .Net CLR memory) 5 mb

now without profiler when i run my application it shows

1)taskmanger 152mb
2)perfmon.exe
privatebytes (aspnet_wp.exe - process) 165
bytes in all heap 119mb
gen 2 86mb
large obj heap 20mb

when i used windbg and took dump of the application with the same set ofdata ,it shows almost the same amout of large obj heap and
gen2 heap as it was showing in the perfmon ( without running profiler).major chunk was the string (35 mb) ,byte(25mb) and char(15mb).Even i checked those objects having size of 15 mb but the main problem was i couldnt found their root because of the missing symbol files
(crystal reports symbols) .

now my queries are following:

1) i saw in one of ur post that ur using gc.collect() many times while taking heap snapshot. so i think ,its because of that collection ,the size
of bytes in all heap reduced to 5mb in perfmon.exe . now if the heap is already collected ,why doesnt the size of private bytes(perfmon) or
the size of physical memory (profiler) got reduced . what is this :-> OTHER DATA 112 mb shows in the profiler's .

2)in my applicaton this memory doesnt recoverd and hence after some times gives out of memory problem. if i use the gc.collect() and
gc.waitforpendingfinalizers() then it collects the memory some times but not always( mostly in the first run it doesnt but on the second run it
recovers all).but as gc.collect() is considered to be expensive what should i do to recover all my memory?

3) we are using sessions heavily ( sql server mode) .so is there any possibility that because of sessions the memory cant be recovered?
i think it shouldnt be the case cause it happens when the mode is inproc

waiting for ur reply
thanks in advance
bye

shan
Posts: 5
Joined: Mon Jul 24, 2006 7:02 am

need ur help Andreas Suurkuusk

Post by shan » Wed Jul 26, 2006 12:45 pm

hi andreas
i need ur help
please reply this problem
i am much puzzled about this problem
just give me few hints
waiting ur answer

shan
Posts: 5
Joined: Mon Jul 24, 2006 7:02 am

new behavior with application

Post by shan » Wed Jul 26, 2006 3:31 pm

i have added gc.collect() then gc.waitforpendingfinalizers() and then gc.collect() in my pageunload event .Now when i run my application,in the first go click( asp button, for fetching data from server) the memory goes up and doesnt recover on page unload ,but on the second go click (same button) the memory goes down to normal. so why this happen?.
Now the actuall problem is ,when i use it with profiler this behaviour doesnt repeat,what i mean is ,on the first run or in the second run(button click) the memory doesnt recover at all . so why this unusual behaviour with application when we run it with profiler??
waithing for ur reply
urgently
[/b]

Andreas Suurkuusk
Posts: 1029
Joined: Wed Mar 02, 2005 7:53 pm

Post by Andreas Suurkuusk » Wed Jul 26, 2006 9:01 pm

Hi,

Since it seems to help to make a call to GC.WaitForPendingFinalizers, it is possible that the finalizer thread has become stuck. If you have finalizers that make calls to COM objects (which you probably have when working with SQL connections), there will be a problem if the COM objects were created in a STA thread. If this is the case, you must make sure that there is a message pump on that thread, otherwise the finalizer thread will become stuck. Calling GC.WaitForPendingFinalizers will pump messages, and will therefore allow the finalizer thread to continue running. To get some more information about the problem with apartments and finalization you can read a long blog post by Chris Brumme at http://blogs.msdn.com/cbrumme/archive/2 ... 66219.aspx (search for finalizer).

I'm not sure what the 112MB "other data" is, but it is possible that it is memory that has previously been used by instances on the the GC heap. Under .NET Framework 1.x, the runtime doesn't provide the runtime with information about the location of the GC heap, so if a memory area is completely empty, the profiler will not be able to identify it as a managed heap.

Note that the runtime will not necessarily release memory even though it is not currently used by any instances, since it is quite likely that the memory will be needed again. I don't know the actual rules for when memory is released (and the rules can be changed in new versions of the runtime).

As you mentioned, the profiler initiates several GCs when collecting a heap snapshot, but it does not actually make a call to GC.Collect. Since GC.Collect might work a little differently than the collect initiated by the profiler, it can explain why you are seeing different behaviour when manually calling GC.Collect and GC.WaitForPendingFinalizers.
Best regards,

Andreas Suurkuusk
SciTech Software AB

shan
Posts: 5
Joined: Mon Jul 24, 2006 7:02 am

Post by shan » Thu Jul 27, 2006 12:47 pm

i am using ado.net for the connection in the application ,so i think no problem with unmanaged resources.
see even if i use gc.collect() one time, then also it recovers all the memory in first run or in second run ,not necessarily to use waitforpendingfinalizers() for collection ,what i wrote in my previous post.I think the problem with my application is generational garbage collections or large object heaps. My session objects are very big and global in scope ,so when garbage collector starts ,it goes on increasing the age of these objects because of their scope .Now ,garbage collector doesnt generally collects the gen 2 heap or the large obj heap unless forcefully we call it( tell me if i am wrong ?).so here is the problem ,this memory accumulate itself and not recoverd unless we call all generations (3) collections ,i mean gc.collect().
tell me i am going wrong??
now ,if this the problem ,what would be the solution? .reduce the scope of sessions OR CALL THE GC.COLLECT() explicitly???

Andreas Suurkuusk
Posts: 1029
Joined: Wed Mar 02, 2005 7:53 pm

Post by Andreas Suurkuusk » Mon Jul 31, 2006 2:22 pm

Hi,

The runtime tries to adapt to the way you use managed memory, but the fundamental assumption of the runtime is that young objects die young, and old objects live forever. If you break this assumption, the memory overhead of your application may become large. Note that large objects are considered to be old immediately, so you should avoid to create short-lived large objects.

Even though it is considered to be a bad practice to call GC.Collect() explicitly, there are scenarios where it might be justified. One such scenerio is when you stop using a large data structure which contains instances that might have been promoted to a higher GC generation (or that contains a lot of large objects). The runtime doesn't know that the large structure can be GCed and it may take a while before a higher generation GC is performed.
Best regards,

Andreas Suurkuusk
SciTech Software AB

Post Reply

Who is online

Users browsing this forum: No registered users and 15 guests