How do I configure iTerm2 to open URLs in a specific browser?

If you are trying to open http://localhost:#PORT, it might be necessary to add in Smart Selection Rules a new rule with Regular Expression of https?://localhost(:[a-zA-Z0-9]+)? and Precision of Very High, before applying @SushiHangover's answer.

The default HTTP URL rule does not appear to recognize http://localhost well.

You can add a Context Menu option for the smart selection of a URL for Chrome via the Edit Actions of the Smart Selection Rules (within a given profile) with a parameter of:

open -a "Google Chrome" \0

Once you do this, this action will override the built-in default of using the system defined app (Safari in your case).