Set "older" kernel as default grub entry
First, make a backup copy of /etc/default/grub. If something goes wrong, you can easily revert to the known-good copy.
sudo cp /etc/default/grub /etc/default/grub.bak
Then edit the file using the text editor of your choice (e.g. gedit, etc.).
sudo -H gedit /etc/default/grub
Find the line that contains GRUB_DEFAULT - this is what you'll want to edit to set the default. You must know the full name of the kernel you want - e.g. Ubuntu, with Linux 3.13.0-53-generic - along with the full name of the "advanced menu" - e.g. Advanced options for Ubuntu.
You then combine those two strings with > and set GRUB_DEFAULT to them as: GRUB_DEFAULT="Advanced options for Ubuntu>Ubuntu, with Linux 3.13.0-53-generic" (including quotes).
Save it, then build the updated grub menu.
sudo update-grub
See also: GNU GRUB Manual 2.02 or more specifically section 15.1.0 on setting the default.
Note: There is a method utilizing numbers to access kernels and menus but this is not recommended as it is unreliable when kernel updates occur.
The best solution for me was to set (in /etc/default/grub):
GRUB_DEFAULT=saved
GRUB_SAVEDEFAULT=true
With this settings the last selected value is kept for the next boot. If you have a new kernel you don't have to edit your config.
Don't forget to re-run sudo update-grub
The solution is from reading: info grub-mkconfig
Now that an Advanced menu is default in Ubuntu, you'll need to select it before the kernel you want using the > character.
Set e.g.:
GRUB_DEFAULT="1>7"
in /etc/default/grub and re-run sudo update-grub.
It is important to note that for the GRUB menu entries numbering starts with 0. Therefore the 1 above points to the Advanced menu. As a precaution, you may want to initially set GRUB_TIMEOUT=5. Some may be unable to access GRUB by hitting a key at boot time. This is a safety net in case you accidentally point to something like Memory test instead. It is also necessary to include the above numbers in quotes. It will not work otherwise.
I tested on 16.04 LTS.