Memory Leaks in ShowDialog

In an earlier post I talked about Memory Leaks in .NET due to events. This time I'm going to talk about memory leaks that may occur in the use of Form.ShowDialog().

I'm talking about a WinForms Form here, so this doesn't apply to WPF or GTK or any other tool kit (which is not to say they don't have the same behavior, just that I don't know).

When you open a Form with Form.Show() it gets disposed automatically when the user closes it or when you programmatically call the Close() method. However, when you open a Form with Form.ShowDialog() it doesn't get disposed when the user closes it, nor when you programmatically close it. It just gets hidden. Presumably this is so you could re-show it if you wanted to. The downside of this is you have to dispose it manually.

You may be thinking that this isn't a big deal because the garbage collector will take care of it for you when the variable goes out of scope. This is not the case. If you don't call the Dispose method that form, and anything referenced by the form, will remain in memory until your program ends.

Because of this, my best practice recommendation is anytime you're opening a Form with ShowDialog, consider doing some like this:

using( SomeDialog sd = new SomeDialog() )
{
sd.ShowDialog();
}

If you can't constrain the scope of your form to just one method then you're stuck remembering to dispose it before you finish with it.

This behavior is, at least to me, somewhat unexpected. However, it is fully documented in MSDN:

When a form is displayed as a modal dialog box, clicking the Close button (the button with an X at the upper-right corner of the form) causes the form to be hidden and the DialogResult property to be set to DialogResult.Cancel. Unlike modeless forms, the Close method is not called by the .NET Framework when the user clicks the close form button of a dialog box or sets the value of the DialogResult property. Instead the form is hidden and can be shown again without creating a new instance of the dialog box. Because a form displayed as a dialog box is not closed, you must call the Dispose method of the form when the form is no longer needed by your application.

Posting Code on blogger

I finally found a decent way to post code on this blog. Its not hard, I just didn't know how...

Here's how. Just add the following css class definition in your blog's template:

.code {
color: $textcolor;
white-space: pre;
background: #ebebeb;
overflow: auto;
border: 1px inset;
padding: 4px 4px 4px 4px;
font-size: 85%;
font-family: courier new;
}

Then when you want to post some code, switch to the Edit Html view and wrap the code with a div class like so:

<div class="code">DisplaySampleCode();</div>

The "white-space: pre" part makes it so the browser will display all the spaces you've typed (normally it removes excess spaces). "pre" stands for pre-formatted. This is a reference to the <pre> tag in html.

The "overflow: auto" part tells it to display a scroll bar if the content of the div is too big to fit within the area of the div.

Fun With Circles

I recently decided that I wanted to create a "Progress Spinner." That's a stupid name for a control that looks like its spinning around to indicate that something is happening in the background. In other words, its a progress bar without the bar. The picture shows my end result (I was too lazy to try to make an animated gif).

I decided I wanted this because I hate progress bars that just go back and forth endlessly. Progress bars should be reserved for times when you can actually estimate how far along in a process you are as a discrete percentage. When you have no idea how far along you are, then just display a "spinny thingy" (an alternate name).

Progress spinners are pretty popular today. You see them in Micrsoft software like the Reporting Services WinForms viewer, or Apple's iPod, or the new iPhone. They're everywhere.

Anyway, I thought it would be a fun little side project to create one. Mostly because I would have to re-teach myself some trigonometry, all of which I had forgotten.

Here's the details: Display x number of circles (with r radii) in a circle which fills the size of the user control. Use two colors, a base color and an active color, which can be changed by the user. The active color appears to move clockwise around the outer circle. Allow the speed of the rotation to be changed.

The math was the fun part. When I started, I had no idea how to calculate the location of each of the circles (henceforth referred to as markers). It turns out its quite easy though, just use Polar coordinates and convert to Cartesian coordinates.

In case you can't remember, just like I couldn't, Polar is given by the distance r from the center to the point and the angle theta from what would usually be the positive x axis to the point. In my case, r is constant so I only need to calculate theta for each marker. 2 * PI / num markers gives the angle between each marker. So just loop for each marker and increment theta by that amount.

So that was fun and easy. Of course, it couldn't remain that simple... You also have to account for the radius of the markers in r so your markers don't leave the visible space of the control. And you have to convert from Cartesian coordinates to Computer coordinates (cause 0,0 is the top left in a computer, not the center). And finally, the biggest annoyance: account for borders on the WinForms UserControl.

If you have BorderStyle set to anything other than none, all kinds of weird things starts happening. It turns out that if you draw a line from (0,0) to (0, width) ( top-left to top-right) it will be visible. But if you draw a line from (0, height-1) to (width-1, height-1) (bottom-left to bottom-right) it will not be visible. To make that line visible you actually have to use (0, height-3) to (width-3, height-3). The borders take up 2 pixels, even when they're not using them as in BorderStyle.Single. The borders also only steal space at the bottom and right of the control, not the left or top. It makes no sense to me what-so-ever.

Anyway, it was a fun little project. While I was at it, I added the ability to draw the markers as lines instead of circles:

I also added the ability to register a paint event and paint your own markers however you please.

The control also includes the concept of color profiles allowing you to select from green, black, blue, or red so you don't always have to figure out your own colors.

Just for kicks I also added a feature so that as you resize the control, not only does the diameter of the main circle grow, but the diameter of the markers grows proportionally as well.

Finally I got to use a really nice trick with IEnumerable. I created a MarkerPositionManager : IEnumerable class. This allows me to write code like this in the OnPaint method:

foreach( MarkerPosition mp in markerPositionManager )
{
// draw ellipse in mp.BoundingRect
}

I like that because it completely abstracts the logic for positioning the markers from the logic for drawing the markers. I love using this pattern because its very effective and easy to refactor to.

The end result is pretty fun to play with.

Is WinForms DataBinding currently in progress?

I've recently been looking into WinForms DataBinding. We have an in house databinding solution that we use, but it is vary narrow in scope, so I wanted to learn how WinForms does it.

I started with an example where I have a very simple user control with 5 or 6 fields (combo boxes, text boxes). I wanted to bind these fields to data from a .NET class which has properties which exactly match the fields I'm displaying. Using WinForms databinding to accomplish this was startlingly easy. But then I hit a snag.

I've registered the ValueChanged event on all my controls (These are in house controls which implement an interface so they all have a ValueChanged. For the most part, all it does is forward the TextChanged and SelectedValueChanged events) and hooked it up to a single event handler. When this event handler fires I enable a "reset" link. Thus if the user changes any of the values, they can click reset to put the values back as they were.

The snag is that when the data binding initially loads the controls, the value changed events fire and I enable my reset link. But when its the data binding that has caused my value changed event to fire, I don't want to enabled the reset link. In other words, I need to know if the data binding caused the value changed or if a user editing the field caused the value changed.

Ideally, I'd like to be able to write something like this:

void Fields_ValueChanged( object sender, EventArgs e )
{
if ( !bindingSource.IsUpdatingControls )
resetLink.Enabled = true;
}

Unfortunately for me, nothing like this seems to exist. And even more infuriating, I can't find anyone else complaining about this on the web...

Now from what I've read I understand that if you're using data binding you're not supposed to depend on the controls. Instead you're supposed to use the object that contains your data. This is for MVC purposes apparently. This means: don't use the control's ValueChanged event. But this doesn't work in my case for 3 reasons: 1) I don't need two way binding here, so my underlying data object shouldn't even get updated. 2) To use two way binding I'd have to change the update mode to OnPropertyChanged instead of OnValidation, which I just don't want to have to do. 3) From an MVC perspective, this is View logic which should remain in the View layer and not be dependent on what update mode the underlying data binding is using.

Am I missing something huge here? Is there any way to do this? Or do I have to abandon WinForms data binding? Seems like such a silly thing to force me to write the binding code manually.

Theory of Software Usability

As software matures software designers are putting more and more thought into "usability" or "user experience." This is clearly a good thing. But I've noticed that this strange concept of usability isn't really that well defined.

There are a few distinct factors that all contribute to how usable a given piece of software is overall. I refer to these as:

• Ease of Learning - how easy is it for a user to learn to use the software
  
• Ease of Use - how easy is it for a user to use the software to achieve their goals
  
• Familiarity - how familiar is the user with the software or patterns used by the software
  
• Functionality - how many of a user's goals can be accomplished with the software
  

Of these 4 factors, Familiarity is the only one which changes over time (for a given software release). However, all of these factors depend on a combination of the user and the software. That is, different users will not only have different "values" for Familiarity but for all three as well.

In the case of Ease of Learning this is simply because some people learn faster than others or will take to certain concepts easier than others. Ease of Use will vary because different people may want or need to accomplish their goals in different ways. Functionality will vary for a similar reason, different people may have completely different goals.

So clearly, there is no way to look at a piece of software and assign these 4 factors static values. However I think they are still helpful as a guide to understanding what makes something usable. Its also interesting to consider how these factors relate to each other.

Probably the most important relationship is that between Ease of Learning and Ease of Use. Firstly, they are not the same thing. Take the third generation iPod for example. The iPod is frequently used as an example of an extremely usable interface. Its very simple in that it has only a few UI concepts and a few controlling buttons. One of my main goals when using an iPod is to find a certain artist, album, or song that I want to listen to. To do this I need to navigate through my music collection, reading the names until I find what I'm looking for. The third generation iPod has everything sorted alphabetically which makes scrolling around fairly easy. But wouldn't it be easier if it had a search feature? It didn't. It also didn't scroll the names of songs and albums when you had them selected (it only scrolled while it was playing), which made it very hard to tell if you were looking at the song you wanted. So while the third generation iPod had the functionality I was looking for, it wasn't terribly easy to get there.

Easy to learn, but not as Easy to Use as I'd have liked. These deficiencies have been corrected in later versions of the iPod of course, but it still serves as a great example.

This demonstrates that Ease of Learning and Ease of Use can very easily be in opposition to each other. By making an interface easy to learn you may be crippling how effectively it can be used.

Another interesting point here is that Ease of Learning can only contribute so much to the overall Usability of the software. You can imagine a text editor that is very easy to learn because it only consists of a text box that you can type in. But if the functionality of the text editor is so crippled such that it doesn't support cut and paste, return characters, capital letters, etc... Clearly, this will never be a usable text editor no matter how easy it is to learn.

Also, Ease of Learning becomes less important the more familiar the user becomes with the software. Consider the Vim text editor. This is a very powerful editor which is far more usable to an experienced user than an editor like Notepad. So while Notepad is exceedingly easy to learn, it can't compare to Vim overall. This is because Vim is both easier to use to accomplish certain goals (ex: delete all text to the end of the line) and because it has much more functionality.

So why is so much emphasis placed on Ease of Learning these days when it can clearly only take you so far?

Consider Vim again. Vim is not easy to learn at all. It requires a lot of memorization and completely new UI/control patterns to be learned. In fact a user who knows nothing about it may not even be able to figure out how to type any text into it.

Obviously some kind of balance must be struck. The next logic question then is where do you strike the balance. The only answer (as usual in Computer Science) is it depends.

If your user base contains skilled users with a lot of familiarity, you want the Ease of Use. If your user base is using your software to accomplish very specific goals in a controlled environment where they will be supplied with training, you probably want the Ease of Use. But if your user base is mixed with many different skill levels, many different goals to accomplish, and no structured environment, Ease of Learning is going to be very key.

Not surprisingly, we're right back to where we started with a not very rigidly defined concept of Usability. But at least recognizing these 4 factors as distinct factors can help in the design process as you size up your user base and your goals for the software.

Shared Assemblies, Components, and Applications

Here's the situation:
You have a shared assembly which contains a lot of useful utilities called SharedAssembly.dll

You create a customized component that uses SharedAssembly.dll called ComponentA.dll

You create a full application which uses SharedAssembly.dll and ComponentA.dll called App1.exe

In VS2005 you setup your solution to include all the projects: App1, ComponentA, and SharedAssembly. You setup the references as Project references. Now, when you build the solution, VS figures out it needs to build SharedAssembly first, then ComponentA, then App1. The end result is App1 and ComponentA both use the same version of SharedAssembly. This is wonderful.

But what if you wanted to change it so that ComponentA wasn't included as a project in the solution file? Instead you want to release ComponentA as dll files that can simply be referenced by App1. The problem is that you now have two different versions of SharedAssembly.dll. If you set it up this way you'll get a runtime error because the framework detects that the version of SharedAssembly.dll is not what was expected by ComponentA.dll. Is there any way to get a setup like this working?

I want ComponentA to be like its own little mini-application, with its own versions of dlls, completely independent from App1.

My mind is immediately drawn to Java's jar files, but I don't really know how they work.

I've tried a little utility from MS Research called ILMerge which takes a set of dlls and combines them into a single dll. However, this didn't work. An exception was thrown at run time when the second SharedAssembly namespace was encountered.

I also tried installing the dlls in the GAC. However, it turns out you can't reference a dll in the GAC from Visual Studio: "This is because the GAC does not have full support for all design-time pieces of an assembly -- while you have access to the DLL itself, you will not have access to PDB symbols nor XML documentation files." (link)

Any ideas?

Blob performance concerns

Maybe someone can help me answer this question. I originally wrote about this on 3/2/2006 but never really found a good answer.

Suppose you are writing a program in C# that uses SQL Server 2005 and you'd like to store some files as Blobs (Binary Large OBjectS) in the database.

This is a very simple thing to do. Take the bytes, send them to SQL Server 2005 where you store them in a varbinary(max) column.

The code for this in C# looks like:

FileStream fs = new FileStream( "file" );
byte[] buff = new byte[fs.Length];
fs.Read( buff, 0, fs.Length );

SqlCommand sc = new SqlCommand(...);
sc.Parameters.Add( ...buff... );

My concern is that this code is going to read the entire file into memory. Then its going to send all that memory to SQL server in one shot. If this is a sufficiently large file I'd expect there to be problems on both the client and the server because of this.

If the client is somewhat lacking for memory, or if the file is really large, then I'd expect this code to cause a lot of disk paging. Possibly even paging out the beginning of the file you just read from disk to make room for the end of the file, just to turn around and page out the end of the file to make room for the beginning so it can send those bytes to SQL. I'm not clear such a thing could ever actually happen, but I do suspect some paging will occur.

What I'd really like to do is specify a stream as the parameter instead of a byte array. That way I could trust the driver was only loading a portion of the full file into a buffer at a time. It seems the JDBC driver supports this, but the SqlClient driver for ADO.NET doesn't.

So my questions are, am I blowing this way out of proportion? Is there a better way to insert a Blob in SQL from C#? Why does no one else on the internet address this concern?

Partly because of this problem, along with some other performance concerns, I ended up storing the files on a shared file server instead of as Blobs in SQL. This allowed to me to handle the transfer myself using the typical buffered approach.

Memory Leaks, Garbage Collection, and .NET

I have frequently read and been told that .NET (and Java) do not have memory leaks. Of course, this depends on the definition of a memory leak.

One definition says a memory leak occurs when a program allocates memory from the operating system but never gives it back. This is pretty clear, so I'm going to stick with it.

Why does .NET not have memory leaks? It has a garbage collector. The garbage collector checks your memory for you every so often to determine if it is still referenced anywhere. If it is, its assumed you're using it. If it isn't, you can't possibly be using it, and the memory is freed and given back to the system.

Therefore, languages with garbage collectors clearly can't have memory leaks, right? I'm going to show you why I disagree with that statement.

Lets look at a simple example in C#. Suppose you've written a class which acts as a DataTable cache. This will be a static class. The purpose of this class is to store DataTables which will be used by various controls throughout the application. If a DataTable gets updated, this class will update all the controls that use that DataTable.

public static class DataTableCache
{
Dictionary dtCache = new Dictionary();
Dictionary controlStore = new Dictionary();

public static void AddDataTable( string key, DataTable table, Control ctl )
{
if ( dtCache.ContainsKey( key ) )
{
dtCache[key] = table;
// update controls using key, if any exist yet
}
else
{
dtCache.Add( key, table );
// update controls using key, if any exist yet
}
AddControl( ctl, key );
}

public static void AddControl( Control ctl, string key )
{
controlStore.Add( ctl, key );
}

public static DataTable LookupDataTable( string key )
{
if ( dtCache.ContainsKey( key ) )
return dtCache[key];
else
return null;
}
}

I'm keeping this example as simple as possible, but it is modeled on real code. There is a "memory leak" problem here. Suppose you create a Form, f. On that form you add a ComboBox, cb. You obtain a DataTable which you use as cb's DataSource and you add them both to the DataTableCache. When f is closed it will be disposed along with all of its controls, including cb. However, because cb is still in the controlStore dictionary it will never be freed by the garbage collector. It has been disposed, but it is still referenced by our static DataTableCache.

No, don't worry, I'm not claiming this qualifies as a memory leak. This is simply programmer error. The programmer should have accounted for this problem. There are two obvious ways to fix this. The first might be to add a RemoveControl method and call that when the form closes. The second, and better approach, is to register the control's Disposed event in DataTableCache and remove it from the dictionary when that event handler is fired. And while we're at it, we'll also make the event handler check to see if any other controls are still using the disposed control's key after it is removed. If none are, we'll remove the DataTable too.

AddControl would now look like:

public static void AddControl( Control ctl, string key )
{
controlStore.Add( ctl, key );
ctl.Disposed += new EventHandler( ctl_Disposed );
}

And the event handler would look like:

private void ctl_Disposed( object sender, EventArgs e )
{
string key = controlStore[(Control)sender];
controlStore.Remove( (Control)sender );
if ( !controlStore.ContainsValue( key ) )
dtCache.Remove( key );
}

Problem solved. No more memory leak.
Actually, wrong. We still have a memory leak. The DataTable will be disposed and freed by the garbage collector, but the Control wont be. If you don't believe me, try it out yourself. You can use a memory profiler to see that the combo box remains in memory no matter how many times you run the garbage collector.

This is the memory leak! No where in our code do we have a reference to the combo box, so why isn't the garbage collector working? Well, it turns out its because we registered the Disposed event. Any time you register an event you get this little intermediate object behind the scenes which holds a reference to the object you registered the event on and the object you registered the event from. So if you have ever registered an event between two objects, where one of them had a longer lifetime than the other, then you have a memory leak. And if one of those objects is static, as in this example, then the memory wont be given back to the system until the application is closed.

How serious is this? It depends on the circumstances, but most of the time it will be pretty serious. In our example its very likely that the Form, f, had registered the ValueChanged event of the ComboBox, cb. That's a very common need. Because of this, both cb and f will never be freed. And imagine how many controls could potentially be on f. Now none of them can be freed either.

How do we solve this? Its very easy, we just modify the Disposed event handler as follows:

private void ctl_Disposed( object sender, EventArgs e )
{
string key = controlStore[(Control)sender];
controlStore.Remove( (Control)sender );
((Control)key).Disposed -= new EventHandler( ctl_Disposed );
if ( !controlStore.ContainsValue( key ) )
dtCache.Remove( key );
}

I consider this a memory leak because 1) its hard to consider this programmer error since its actually the .NET framework which contains the references and 2) this problem is very very hard to remember to avoid. This means you really do need to use a memory profiler on your applications before releasing them. And therefore I claim that C#, at least, does suffer from memory leaks.

I haven't had a chance to test this on any other languages. I'd be very interested to hear if the same is true in Java, Python, and Ruby for example.

Fair Witness

In the book Stranger In a Strange Land, Robert Heinlein introduces the concept of a Fair Witness. In the book a Fair Witness is someone with complete memory recall (they remember EVERYTHING!) who makes no assumptions about anything. They're used for various legal things because they are infallible observers.

There's a quote from the book which I always liked. I can't remember it exactly but it goes something like:

"Anne, function as a Fair Witness! What color is that house over there?"
"This side is blue."

Obviously the other sides could be any color at all. But it wouldn't occur to you to think that way, and in most cases it wouldn't make sense to.

However, attempting to assume this kind of mind set can be enormously helpful for a software developer. How many times have you been tracking down a bug for hours and hours and hours, unable to figure out what's causing it. "Everything looks right!" you keep saying to yourself. Finally you discover the problem: you had made an assumption about some part of your code that turned out not to be true.

Sometimes its an assumption that you programmed into the code that turns out to be false. These are usually easier to find. Other times its an assumption you made in your mind about the behavior of the code. Could be, "I know this method is working fine" or "It will never do xyz because of abc." Then when you find the bug you realize that not only was your code wrong but you were wrong too. It was two bugs in one.

Of course, being human we can't avoid making assumptions. Assumptions are powerful for the same reason abstraction in Object Oriented Programming is powerful. It lets you forget about the details and focus on a higher level idea. But in my limited experience I've noticed that being human also comes with a tendency to want to rush. And nothing helps you rush more than making assumptions. So now when I set out to find a bug I always remind myself to function as a Fair Witness. I'll still make incorrect assumptions from time to time, but I'll avoid making a countless number of rush-assumptions. And ultimately, I'll end up saving myself time and frustration.

That being said, making assumptions in debugging can be a good thing too. You just have to make your assumptions in a very conscious manor. As in, this side of the house is blue, so I'm going to assume the other sides are as well. Then if you run into an inconsistency later on you'll remember you made the assumption and you can go back and examine if it was the right one. The author of The Old New Thing blog calls this Psychic debugging. I call it functioning as a more flexible Fair Witness.

Why?!

There are a few reasons for this blag.

1. I like to write my opinions about software and technology like things.
   
2. There is a wide community of people on the interweb who are writing similar things. This blog might allow me to join them.
3. I have a website which includes a forum that I wrote in ASP over 5 years ago (as of this posting). Its been great fun, but it doesn't allow me to blog, as it were. Its too open and public. Its also hosted on a computer in my basement, which isn't terribly reliable.
4. I thought I should learn how this blogging software worked.

I may port things I wrote on my original website to this blog if I feel like I'm getting something out of this. If I do, I'll indicate that's where it came from along with the original date in the interest of full disclosure.

And with that...