How do I change the guest session defaults

This has become much easier in 11.10.

First you need to create a folder: sudo mkdir -p /etc/guest-session/skel/. The -p switch creates /etc/guest-session so you can place /skel in it in one go. Now you just add the files you want to /etc/guest-session/skel and it will be copied onto the guest account whenever it's created.

The program that sets this up is /usr/sbin/guest-account in case you want to learn more about what's going on.

The tutorial CustomizeGuestSession elaborates on this topic. In particular it describes how you can create a "special purpose user" for a convenient way to customize the guest session appearance and behavior, without having to know which files to edit.

The guest session is recreated every time as a "default new user" with the /usr/share/gdm/guest-session/guest-session-setup.sh script, which is part of the gdm-guest-session package. I don't see any clean way to make it include your settings into the guest session.

You could of course change the way every new user's $HOME gets created, but I'm not sure that's really what you want (I won't consider it a "clean" solution to do it that way).