How do I clear xmodmap settings?

xmodmap has no notion of state, so it has no way to reset state directly. You can simulate it by using xmodmap -pke >.xmodmap.orig before making any changes (although it doesn't save the modifier map, which you would have to save and restore manually) — but it's a bit too late for that.

Modern systems don't generally use xmodmap to configure the keyboard, though. setxkbmap is the modern way to do it; and that does reset bindings when run. So you may be able to use setxkbmap -layout us to reset things to normal. More complete would be to check for the default configuration in /etc/X11/xorg.conf. For example, on my system

jinx:718 Z$ sed -n '/Identifier.*Keyboard/,/EndSection/p' /etc/X11/xorg.conf
        Identifier      "Generic Keyboard"
        Driver  "kbd"
        Option  "XkbModel"      "pc105"
        Option  "XkbLayout"     "us"
        Option  "XkbOptions"    "grp:alt_shift_toggle"
EndSection

The corresponding command is

setxkbmap -model pc105 -layout us -option grp:alt_shift_toggle

If there were an XkbVariant entry in the output, you would pass its value with -variant. One thing to watch out for is that options are handled specially: you can only set one option per -option parameter, and you need to use -option '' to reset parameters first. So to fully reset when there is something like XkbOptions "grp:alt_shift_toggle,grp:ctrls_toggle" you would need

setxkbmap -model pc105 -layout us -option '' -option grp:alt_shift_toggle -option grp:ctrls_toggle

setxkbmap -option resets the meta keys to default.

Experimenting with xmodmap, I messed up my key settings by using xmodmap -en "keysym BackSpace = Delete". Thought the -n flag would cause no action to be taken because the man xmodmap page stated that

-n      This  option indicates that xmodmap should not change the mappings, but should display what it would
               do, like make(1) does when given this option.

However, the command caused my Delete key to be useless.

After reading the above, I just typed setxkbmap, hoping it would show me the options, whereas in fact it returned immediately with no output, and then my Delete key was miraculously working again!

So it looks as though

setxkbmap

alone will do the job...