Is It Worthwhile To Use A Separate MySQL Server?
We've used a separate MySQL server in several instances where the shops were facing high traffic. There are a couple of advantages to it
- Dedicated database servers can be tweaked to the specific needs of MySQL which are different from a webserver
- It's easy to add a second (load balanced) database server to the cluster when needed
- When your database goes down it doesn't crash the frontend so you can display a decent warning or error page.
When Magento is properly cached with Varnish or any other FPC extension the main bottleneck will be the database from what I have experienced. The real power will be required for your database. In this way you can start out with a relatively small web server and invest more in the database server.
Security:
In addition to Sander's answer, I would add that at certain levels of PCI compliance this is a requirement:
Separate Web and Database Servers CHD is stored en mass in the database, making it a high-value target to an attacker. Separate database server means that access can be strictly controlled (limited exposure). Required by Section 1 of the PCI DSS.
Source: http://www.focusonpci.com/site/index.php/PCI-101/technical-requirements.html
By separating web and database duties you're limiting your exposure. Typically your db is in a private segment of your network, and inaccessible publicly.
A static VPN connection is also suggested, in PCI, between your web/db and intrusion detection is strongly suggested on your networking equipment. In the event of a compromise the db will be isolated and the VPN connection terminated such that, even though your application and your encryption key are now compromised, access to the data store has been locked down and is inaccessible.
High Availability / Disaster Recovery:
Sander's on point here. Go vote him up. I would add that in the event of even a small maintenance task like taking a filesystem backup or a database dump your db will, at best, be limited to reads-only for some time. In extreme cases I've seen lock wait timeouts and process queues fill up that all available connections are queued or dropped. The site "goes down", effectively.
You can mitigate this by separating your db and scheduling your web server to place the site into maintenance mode during these backup windows without any consequence to the filesystem of the web server.