If I replace an image in a PictureBox control, should I dispose the original image first?

Yes, you should dispose the old object before you create a new image on top of the same variable. By creating a new image with the same variable, you are removing a reference to it. If there are no references to the old object, you are signifying that it should be picked up by the GC (Garbage Collector). Although technically, this "should" eventually result in the memory being freed assuming that the finalizer makes sure that non-managed resources are taken care of, this is a big assumption (You can't even really assume that the finalizer will be called), and it causes more work for the system. Non-default finalizers causes extra work for the GC in terms of garbage collection level promotion, resulting in taking longer for the memory to be deallocated, and the number of times the GC has to run to do so.

This is assuming that is all written to make sure the finalizer handles it. Anytime an object has a Dispose method (anything which implements IDisposable which BitMap does), it should be called before removing reference to the object (falling out of scope, removing reference to the object etc.).

Here is an article on how the Garbage Collector works in .net

http://www.devx.com/dotnet/Article/33167

Here is how MS says the dispose / finalizer should be implemented:

http://msdn.microsoft.com/en-us/library/b1yfkh5e.aspx

When changing the image associated with a PictureBox, one should call Dispose on the image that was there if and only if nothing else is ever going to use that image. In order to know that, one would have to know where the old image came from. In some cases, the image will have been created just for the purpose of being assigned to the PictureBox. In other cases, the image might be one which is intended to be shared and/or reused. If image was created solely for the purpose of assignment to the PictureBox, it should be Disposed if the PictureBox is disposed or given another image. If the image is supposed to be shared or reused, such conditions must not cause it to be disposed.

The proper way to resolve such issues in general would for classes which have IDisposable properties (like the PictureBox, with Image) to use an explicit SetImage method rather than having a mutable Image property, and for the SetImage method to include a parameter indicating whether the PictureBox should assume responsibility for disposing of it. Calling SetImage or Dispose on the PictureBox should call Dispose on the held image if and only if the previous SetImage call gave it that responsibility. Unfortunately, PictureBox doesn't work that way, but I would highly recommend using that as a pattern for future classes you write which hold IDisposable objects.

Yes you should. It implements IDisposable.
As general rule of thumb, dispose all objects that implements IDisposable. Do not leave it to GC.