Go frame-by-frame through a movie with a precise timer

Give Avidemux a try. It should be able to do what you want and is freely available for Windows and Linux.

I don't know if this feature was available at the time the question was asked, but the open source MPC-HC (Media Player Classic Home Cinema) video player definitely has both these features available now.

There is a context menu option to enable millisecond precision for time indicator, and you can easily step between frames in both forward (CTRL+Right) and backward (CTRL+Left) directions.

You can also jump to exact time point OR frame in video file via Navigate -> Go To... menu (or CTRL+G).