Why doesn't the evening/morning sun feel much hotter on your face?

In general, the sun's light (particularly the UV that causes sunburn) has to pass through a lot more atmosphere (or a greater amount of air mass) in the morning and evening to get to a vertical surface than it does when it is at zenith to a horizontal surface.

An example is shown in the generalised image below (all graphs are obviously generalised):

The caption from the source (The Daily Cycle of Sunlight: Part 1) is (the relevant statements are in bold):

When the sun is close to the horizon, its rays pass through a thicker slice of the atmosphere, which warms the color of sunlight (orange arrow). During the middle portion of the day, the sun’s rays pass through a thinner slice of the at-mosphere (white arrow). We define noon sunlight as being neutral white—neither warm nor cool.

As such, the solar radiation is absorbed and scattered more in the atmosphere in the morning or evening. The net effect can be shown on the following diagram:

Source: It's Coldest After Dawn

In terms of sunburn, the erthemal dose rate (related to UV radiation) follows the same pattern, as seen below:

Image source: NOAA, where the define erythemal dose rate as

indicates the instantaneous amount of skin damaging UV radiation.