Android - What is USB debugging? Can I keep it ON forever?

Enabling USB debugging essentially starts up the adb daemon on your device, which allows it to communicate with adb on another device to enable debugging commands. It's used when developing and debugging applications, and allows you (primarily) to:

• Transfer data between a computer and your device (both ways)
• Read log data easily from logcat
• Debug applications, including breakpoints and heap monitoring
• Install and uninstall applications
• Access a stripped-down shell on the device, for command-line interaction.

Leaving it enabled all the time will have a negligible impact on battery, if any at all. Some other things to consider about leaving it on (or not):

• PRO: If your device has a hardware problem that prevents you from accessing the screen, USB debugging could be helpful in recovering data (especially on a rooted device).
• PRO: You can install applications, reboot your device, and generally do some other convenient things without having to actually interact with your device physically, even with the screen locked/off.
• PRO: On a rooted device it may be possible to regain entry if you have forgotten the screen lock password/pin/pattern (also a potential CON, see below).
• PRO: You can take advantage of port forwarding over USB debugging to use tethering apps like PDANet (I believe that's how it's implemented since it does require it).
• CON: If your device is lost or stolen, an unscrupulous individual could attempt to steal data from the device regardless of whether or not you've got a screen lock. If you're rooted, they can get pretty much everything.
• CON: I recall reading that some devices can have trouble with accessing the SD card as a mass storage device when USB debugging is enabled. I've personally never seen this issue, but I have seen people allege that it was a problem.

See also: Using Hardware Devices from the Android SDK documentation which discusses debugging applications on physical devices (rather than the emulator). There is also a documentation page regarding the Android Debug Bridge (ADB) which explains its purpose and its commands.

To make one of eldarerathis' cons less critical:

One could use Tasker or Locale together with the Secure Settings plugin to automatically toggle USB Debugging on for safe locations (based e.g. on cell towers, network/GPS location, WiFi networks connected or in reach), and toggle it off (exit-task with Tasker) when leaving such. With a NFC tag and the Locale NFC Plugin it could also be switched it on/off on demand.

All that even without touching the screen -- so it would work even if the screen is broken. And that "unscrupulous individual" then would have to be in your "safe location" or have your tag, and must know those circumstances -- which is much less risky.

If your device does not belong to that small group having trouble mounting their storage with USB Debugging turned on, this would leave almost no "Con" then.

The biggest downside is not being able to mount it as a mass storage device when you plug it into the computer. With it turned on it will always assume you want to debug the device. You really only need it turned on if you are developing software for the phone or plan on using adb with your phone. The adb program is only available through the Android SDK so its not something average users need. That being said I keep my turned on all the time, but that's only for connivence for doing Android development because I do that more often that use it as a mass storage. You mileage may vary.