Comments on: Out of Memory Exception While Attempting to Do SQL CLR http://chrisbenard.net/2009/07/01/out-of-memory-exception-while-attempting-to-do-sql-clr/ Ramblings of an inquisitive software developer in the Dallas area Tue, 17 Apr 2012 16:46:27 +0000 hourly 1 http://wordpress.org/?v=3.3.2 By: CruiZen http://chrisbenard.net/2009/07/01/out-of-memory-exception-while-attempting-to-do-sql-clr/comment-page-1/#comment-117491 CruiZen Sat, 28 Nov 2009 12:09:55 +0000 http://chrisbenard.net/?p=165#comment-117491 If I have 32 GB RAM on a 32 bit system with PAE (bit not the 3 GB switch) enabled and AWE configured, what should be the value of the -g switch? Our DBA tried 2Gb! We are seeing that memory consumption goes on increasing for SQL CLR (probably memory leaks from SPs If I have 32 GB RAM on a 32 bit system with PAE (bit not the 3 GB switch) enabled and AWE configured, what should be the value of the -g switch? Our DBA tried 2Gb!
We are seeing that memory consumption goes on increasing for SQL CLR (probably memory leaks from SPs

]]> By: Chris http://chrisbenard.net/2009/07/01/out-of-memory-exception-while-attempting-to-do-sql-clr/comment-page-1/#comment-113629 Chris Mon, 19 Oct 2009 19:37:24 +0000 http://chrisbenard.net/?p=165#comment-113629 Gus, Nope! Just restarted SQL server and it was good to go. Gus,

Nope! Just restarted SQL server and it was good to go.

]]> By: Gus http://chrisbenard.net/2009/07/01/out-of-memory-exception-while-attempting-to-do-sql-clr/comment-page-1/#comment-113628 Gus Mon, 19 Oct 2009 19:17:36 +0000 http://chrisbenard.net/?p=165#comment-113628 Did you need to reboot after adding the ;-g384 to the parameters? Did you need to reboot after adding the ;-g384 to the parameters?

]]> By: Chris http://chrisbenard.net/2009/07/01/out-of-memory-exception-while-attempting-to-do-sql-clr/comment-page-1/#comment-101550 Chris Thu, 02 Jul 2009 14:41:02 +0000 http://chrisbenard.net/?p=165#comment-101550 Jonathan, Our admin increased the MemToLeave (as he had already suggested might be the case) to 448 last night, and now I am able to return 1536 rows in 1 min 21 seconds, including the conversion that is in my example code. Thank you. Jonathan,

Our admin increased the MemToLeave (as he had already suggested might be the case) to 448 last night, and now I am able to return 1536 rows in 1 min 21 seconds, including the conversion that is in my example code.

Thank you.

]]> By: Jonathan Kehayias http://chrisbenard.net/2009/07/01/out-of-memory-exception-while-attempting-to-do-sql-clr/comment-page-1/#comment-101488 Jonathan Kehayias Wed, 01 Jul 2009 22:37:17 +0000 http://chrisbenard.net/?p=165#comment-101488 Your limitation here is linked to being on a 32 bit server. Without really getting into all the details, there is only 4GB VAS on a 32 bit server split 50/50 into user and kernel modes unless you use the /USERVA or /3GB switches in the boot.ini file. Because there is only 2GB of user VAS, SQL Server creates a reservation in VAS called the MemToLeave which is generally only 384MB of memory. The MTL is used for worker threads, connections with a network packet size > 8000 bytes, multi-page allocations, linked servers, OLE Automation, and SQLCLR. Your process is likely causing VAS fragmentation and since allocations from the MTL are required to be contiguous allocations, you could have 100MB of available VAS but if the largest contiguous available block is only 4MB and you need an 8MB allocation, you will get an OOM exception. What you can try to do to get around this issue is bump up the base VAS reservation size from 256MB to a larger value with the -g startup parameter. This might work, but it also might not, it depends on how heavy a consumer your code is, and how fastly fragmented the VAS becomes. if you wanted to bump the base VAS reservation to 384MB which would equate to roughly 512MB total VAS on most servers, you would simply add ;-g384 at the end of the start parameters for the SQL Server Service. To check the size and max contiguous block size of VAS on your server you can grab the code example on Christian Bolton's blog post: http://sqlblogcasts.com/blogs/christian/archive/2008/01/07/sql-server-memtoleave-vas-and-64-bit.aspx If you bump the base reservation up, you are making a trade off with the size of your Buffer Pool, so you need to be careful with how far you increase the VAS reservation. Try bumping it to 384 and see if that solves the problem. I wouldn't go anything beyond 512MB. Since you already know that this isn't really something you should be doing in SQL, I won't go down that road. By chance have you looked at the memory usage of these methods external to SQL Server using the .NET profiler or other tool? You might find that it is just to big no matter what you do. Your limitation here is linked to being on a 32 bit server. Without really getting into all the details, there is only 4GB VAS on a 32 bit server split 50/50 into user and kernel modes unless you use the /USERVA or /3GB switches in the boot.ini file. Because there is only 2GB of user VAS, SQL Server creates a reservation in VAS called the MemToLeave which is generally only 384MB of memory. The MTL is used for worker threads, connections with a network packet size > 8000 bytes, multi-page allocations, linked servers, OLE Automation, and SQLCLR. Your process is likely causing VAS fragmentation and since allocations from the MTL are required to be contiguous allocations, you could have 100MB of available VAS but if the largest contiguous available block is only 4MB and you need an 8MB allocation, you will get an OOM exception.

What you can try to do to get around this issue is bump up the base VAS reservation size from 256MB to a larger value with the -g startup parameter. This might work, but it also might not, it depends on how heavy a consumer your code is, and how fastly fragmented the VAS becomes. if you wanted to bump the base VAS reservation to 384MB which would equate to roughly 512MB total VAS on most servers, you would simply add ;-g384 at the end of the start parameters for the SQL Server Service.

To check the size and max contiguous block size of VAS on your server you can grab the code example on Christian Bolton’s blog post:

http://sqlblogcasts.com/blogs/christian/archive/2008/01/07/sql-server-memtoleave-vas-and-64-bit.aspx

If you bump the base reservation up, you are making a trade off with the size of your Buffer Pool, so you need to be careful with how far you increase the VAS reservation. Try bumping it to 384 and see if that solves the problem. I wouldn’t go anything beyond 512MB. Since you already know that this isn’t really something you should be doing in SQL, I won’t go down that road.

By chance have you looked at the memory usage of these methods external to SQL Server using the .NET profiler or other tool? You might find that it is just to big no matter what you do.

]]> By: Chris http://chrisbenard.net/2009/07/01/out-of-memory-exception-while-attempting-to-do-sql-clr/comment-page-1/#comment-101484 Chris Wed, 01 Jul 2009 22:21:57 +0000 http://chrisbenard.net/?p=165#comment-101484 @Jonathan, thanks for the reply. We are running 32 bit SQL Server 2008. The method in question creates a Graphics object, draws a large set of lines (signature) to it, then scales the image down to a smaller image (creating 2 Bitmaps and 2 Graphics objects in the process, but I am disposing them all). If you can help, I would be most gracious. @Jonathan, thanks for the reply. We are running 32 bit SQL Server 2008. The method in question creates a Graphics object, draws a large set of lines (signature) to it, then scales the image down to a smaller image (creating 2 Bitmaps and 2 Graphics objects in the process, but I am disposing them all). If you can help, I would be most gracious.

]]> By: Jonathan Kehayias http://chrisbenard.net/2009/07/01/out-of-memory-exception-while-attempting-to-do-sql-clr/comment-page-1/#comment-101482 Jonathan Kehayias Wed, 01 Jul 2009 21:57:32 +0000 http://chrisbenard.net/?p=165#comment-101482 Chris, Is this on a 32 bit or 64 bit SQL Server? Your problem is likely that you are running out of virtual address space before the GC is triggered in SQLCLR. If you are on a 32bit server this can be more prominent because of the limited VAS which is likely getting fragmented by the operation that you are doing. Post some more details, or contact me through my blog and I'll offer some ideas about how you might get around this. Chris,

Is this on a 32 bit or 64 bit SQL Server? Your problem is likely that you are running out of virtual address space before the GC is triggered in SQLCLR. If you are on a 32bit server this can be more prominent because of the limited VAS which is likely getting fragmented by the operation that you are doing. Post some more details, or contact me through my blog and I’ll offer some ideas about how you might get around this.

]]>