Update: We figured out how to make it work with the help of our DBA and Jonathan Kehayias (see comments). We increased SQL Server’s MEM_TO_LEAVE property, by adjusting the –g command line switch for the service, to 448MB. This increase of the shared memory pool gave SQL Server enough breathing room for its worker threads, and now we are able to return 1536 records in 1 minute 21 seconds, including the Bitmap conversion.
Furthermore, we found a workaround using our reporting engine, which we will implement when we have some breathing room ourselves in our rollout timeline. This will “get it working” for now, which is the requirement handed down from above. Soon though, we will have this working in our application tier, where it belongs.
SQL Server’s CLR abilities are really cool. I have done some benchmarking, performing the same computations in both T-SQL and CLR and I have found CLR to outperform T-SQL by factors of greater than 10 to 1. It’s fantastic for this use.
Unfortunately, it has won a battle against me today. I’ll provide a bit of background first. I am currently working on the Point of Sale component of my company’s pharmacy system, and in particular, I am writing all of the code associated with interacting with the signature pad. I’ve abstracted everything nicely, such that we can support multiple pad’s, and I just have to write a .Net class that implements the ISignaturePad interface.
Obviously, other than the navigational aspects of buttons and listboxes and sale line items being displayed on the pad, capturing signatures themselves is paramount. Because each signature pad can spew out the signature data in a different way, and we want to store the “perfect” vector information, I’ve abstracted the signature data into two classes. The Signature and SignaturePoint classes’ definitions look like this:
[Serializable]
public partial class Signature
{
public virtual SignaturePoint[] Points { get; protected set; }
public virtual SignaturePoint TopLeft { get; set; }
public virtual SignaturePoint BottomRight { get; set; }
public int Width
{
get
{
return BottomRight.X - TopLeft.X;
}
}
public int Height
{
get
{
return BottomRight.Y - TopLeft.Y;
}
}
public int XDpi { get; set; }
public int YDpi { get; set; }
private Signature()
{
}
public static Signature CreateFromTT8500String(string signaturePoints)
{
var sig = new Signature();
sig.Points = sig.CreateFromTT8500PointsData(signaturePoints);
return sig;
}
public virtual byte[] Serialize()
{
using (var ms = new MemoryStream())
{
var bf = new BinaryFormatter();
bf.Serialize(ms, this);
return ms.ToArray();
}
}
public static Signature Deserialize(byte[] serializedSignatureBytes)
{
using (var ms = new MemoryStream(serializedSignatureBytes))
{
var bf = new BinaryFormatter();
return (Signature)bf.Deserialize(ms);
}
}
protected virtual SignaturePoint[] CreateFromTT8500PointsData(string signaturePoints)
{
List list = new List();
// Do a lot of work here to change the strange format that we get
// as a string into bytes and transform them into an array of my custom class
return list.ToArray();
}
protected void CropPoints(List list)
{
foreach (var point in list)
{
point.X -= TopLeft.X;
point.Y -= TopLeft.Y;
}
BottomRight.X -= TopLeft.X;
BottomRight.Y -= TopLeft.Y;
TopLeft.X = 0;
TopLeft.Y = 0;
}
}
[Serializable]
public class SignaturePoint
{
public int X { get; set; }
public int Y { get; set; }
public bool PenUp { get; set; }
public SignaturePoint()
{
}
public SignaturePoint(int x, int y)
: this(x, y, false)
{
}
public SignaturePoint(int x, int y, bool penUp)
: this()
{
X = x;
Y = y;
PenUp = penUp;
}
public Point ToPoint()
{
return new Point(X, Y);
}
public override string ToString()
{
return ToPoint().ToString();
}
}
As you might have noticed, Signature is marked as Serializable, and that’s exactly what we’re doing to store the “perfect” information in the database. We call the Serialize() method on my Signature class, and store the resulting byte array in the database as VARBINARY(MAX). It works fine when we pull that back with ADO.NET and re-hydrate a Signature object with my static Deserialize() method.
To actually draw a signature on a picture box on a Windows form for example, we call my ToBitmap() method, that is in another file (partial class), and it generates a bitmap of the requested width, height, and pen width, suitable for display on a receipt, screen, report, etc.
However, as a limitation of our ridiculous reporting engine (and we are currently trying to work around it’s oddities), for an upcoming beta, we are trying to get SQL server to create the bitmaps for passing back up to our reporting engine. Yes, I do know that is application tier logic and shouldn’t be performed at the database level. We are still trying to work around it using custom controls with the reporting engine.
So, I create a SQL CLR scalar function in my Signature class (SqlBytes CreateBitmap(SqlBytes serializedSignatureBytes, SqlInt32 width, SqlInt32 height)), moved Signature and SignaturePoint to a CompanyName.SignaturePad.Common assembly, added a reference to System.Drawing. I added the assembly to SQL Serverand fought it a bit (setting TRUSTWORTHY to ON for the database). I had to manually add System.Drawing as well, because the version on my computer didn’t match exactly on the server, yet another pain and indication I shouldn’t be doing this. And another indication was SQL server warning me that System.Drawing hadn’t been tested and that the universe will indeed explode if they change something in it. I accept the risks, at the moment.
Everything worked great… for 3 signatures. As soon as the 4th signature is added, I get this:
Msg 6532, Level 16, State 49, Line 1 .NET Framework execution was aborted by escalation policy because of out of memory. System.Threading.ThreadAbortException: Thread was being aborted. System.Threading.ThreadAbortException: at System.Drawing.Graphics.CheckErrorStatus(Int32 status) at System.Drawing.Graphics.DrawImage(Image image, Int32 x, Int32 y, Int32 width, Int32 height) at CompanyName.SignaturePad.Common.Signature.ToBitmap(Int32 width, Int32 height, Int32 penWidth, Color foregroundColor, Color backgroundColor) at CompanyName.SignaturePad.Common.Signature.ToBitmap(Int32 width, Int32 height) at CompanyName.SignaturePad.Common.Signature.CreateBitmap(SqlBytes serializedSignatureBytes, SqlInt32 width, SqlInt32 height)
Looking at that code, I’m doing everything that I know to do. I’m making sure to Dispose() all Bitmaps, Graphics, Streams, etc. I even tried explicitly setting this to null. SQL Server just runs out of memory after about 3 signatures. I’m not sure if this is a side effect of its execution plan (perhaps forking to multiple workers?).
So… now I’m stuck, and I’m not really sure what to do, until we get a workaround in the reporting engine, which someone else is currently working on. I’ll update this post if we get a working solution.

Chris,
Is this on a 32 bit or 64 bit SQL Server? Your problem is likely that you are running out of virtual address space before the GC is triggered in SQLCLR. If you are on a 32bit server this can be more prominent because of the limited VAS which is likely getting fragmented by the operation that you are doing. Post some more details, or contact me through my blog and I’ll offer some ideas about how you might get around this.
@Jonathan, thanks for the reply. We are running 32 bit SQL Server 2008. The method in question creates a Graphics object, draws a large set of lines (signature) to it, then scales the image down to a smaller image (creating 2 Bitmaps and 2 Graphics objects in the process, but I am disposing them all). If you can help, I would be most gracious.
Your limitation here is linked to being on a 32 bit server. Without really getting into all the details, there is only 4GB VAS on a 32 bit server split 50/50 into user and kernel modes unless you use the /USERVA or /3GB switches in the boot.ini file. Because there is only 2GB of user VAS, SQL Server creates a reservation in VAS called the MemToLeave which is generally only 384MB of memory. The MTL is used for worker threads, connections with a network packet size > 8000 bytes, multi-page allocations, linked servers, OLE Automation, and SQLCLR. Your process is likely causing VAS fragmentation and since allocations from the MTL are required to be contiguous allocations, you could have 100MB of available VAS but if the largest contiguous available block is only 4MB and you need an 8MB allocation, you will get an OOM exception.
What you can try to do to get around this issue is bump up the base VAS reservation size from 256MB to a larger value with the -g startup parameter. This might work, but it also might not, it depends on how heavy a consumer your code is, and how fastly fragmented the VAS becomes. if you wanted to bump the base VAS reservation to 384MB which would equate to roughly 512MB total VAS on most servers, you would simply add ;-g384 at the end of the start parameters for the SQL Server Service.
To check the size and max contiguous block size of VAS on your server you can grab the code example on Christian Bolton’s blog post:
http://sqlblogcasts.com/blogs/christian/archive/2008/01/07/sql-server-memtoleave-vas-and-64-bit.aspx
If you bump the base reservation up, you are making a trade off with the size of your Buffer Pool, so you need to be careful with how far you increase the VAS reservation. Try bumping it to 384 and see if that solves the problem. I wouldn’t go anything beyond 512MB. Since you already know that this isn’t really something you should be doing in SQL, I won’t go down that road.
By chance have you looked at the memory usage of these methods external to SQL Server using the .NET profiler or other tool? You might find that it is just to big no matter what you do.
Jonathan,
Our admin increased the MemToLeave (as he had already suggested might be the case) to 448 last night, and now I am able to return 1536 rows in 1 min 21 seconds, including the conversion that is in my example code.
Thank you.
Did you need to reboot after adding the ;-g384 to the parameters?
Gus,
Nope! Just restarted SQL server and it was good to go.
If I have 32 GB RAM on a 32 bit system with PAE (bit not the 3 GB switch) enabled and AWE configured, what should be the value of the -g switch? Our DBA tried 2Gb!
We are seeing that memory consumption goes on increasing for SQL CLR (probably memory leaks from SPs