How do you customize the address bar icon in a visualforce page?

The terminology for that image in the address bar is a favicon.

It's not mentioned in the question but if your Visualforce pages are exposed through a Site, you can upload your favicon into a Static Resource and then in the site configuration you can specify this resource to be used as the Site Favorite Icon and all pages which are displayed through the Site will use this as their favicon image.

Documentation:

• Introduction to Force.com Sites
• Creating and Editing Sites

Add this to the top of your Visualforce page, with the appropriate static resource link of course:

<link rel="icon" type="image/png" href="{!$Resource.Favicon}" />