The service runs fine for about 7 hours and then simply stops processing (service status is that it is still running). The effect is as though the timer has been disabled and so no more processing iterations occur.
While the service executes there is a steady build up in the number of System.WeakReference instances, to the point where there are many millions of them hanging around. They never seem to get cleared out. We have manually placed DataTable dispose calls wherever appropriate to try to ensure they are GC'd as soon as possible, but we are not sure what to do about these WeakReferences. If indeed we need to. I don't know if it is this that is causing the appliaction to fail or not, but they look suspicious to me. Has anyone else experieced anything similar? Do you have any pointers?
I suggest that you collect a heap snapshot and investigate the WeakReference instances. By double-clicking the WeakReference class in the Classes view you will bring up the details view of the class. This view will show you summary information about all WeakReference instances.
The things you should look for is allocation stacks that have created a lot of new instances (by selecting "New live instances" in the "Sort stacks by" list) and root paths that are holding on to a large number of new instances (by selecting "New live instances" in the "Sort root paths by" list).
By investigating the allocation call stacks and the root paths, you should be able to get an idea on why the WeakReferences are created, and why they're not released (or at least who's responsible for hanging on to the instances).
You can also double-click one of the WeakReference instances, to get information about a specific instance. Under the "Field values" tab you can for instance see if the WeakReference instances still have a target, or if the Target is null. If the Target is null, that's an indication that the WeakReference should probably have been cleaned up.
SciTech Software AB