How does the difficulty level affect the gameplay in Skyrim?

According to the UESP, the difficulty level affects the damage you deal and the damage you receive:

Difficulty | Damage Dealt | Damage Taken
-----------------------------------------
Novice     | x2           | x0.5
Apprentice | x1.5         | x0.75
Adept      | x1           | x1
Expert     | x0.75        | x1.5
Master     | x0.5         | x2
Legendary  | x0.25        | x3

Based on this, there are a few things you can extrapolate as the difficulty level increases:

• Skills based on damage increase more slowly, and in turn, the slower you level up (thanks yx.)
• Followers can be knocked out, and in turn, killed, more easily
• Your effective health decreases
• Enemies' effective health increases

Possible note: Weapon Skills aren't based on final damage; they are based on base damage of the weapon, which is unaffected by difficulty. Increasing the difficulty means that you have to hit each enemy more times, which speeds up skill growth. This is very noticeable when you increase the difficulty as my one-handed increased about 6 levels after 2 dungeons.

One thing I noticed is that at lower difficulties, when your follower uses a charged weapon, the weapon does not expend charges. At higher difficulties, it does, and so they start to require constant maintenance.

For example, I gave my follower a sword with +20 fire damage. Played a while on adept, and after hours and hours of use, its charge never went down. Same with staffs.

Knocked the difficulty up to Master, and the charges depleted so fast that we couldn't get through a single floor of a dungeon. It's gets pointless to equip enchanted equipment or staffs to your followers - just becomes another soul-hungry mouth to feed.

I think charges also don't deplete at the expert difficulty. Not sure. Haven't checked.

To add to this answer:

Patch 1.9 adds a sixth difficulty level: Legendary. It reduces damage dealt by the player to x0.25 and increases damage taken by the player to x3.