How do I make Windows 10's File Explorer open "This PC" by default?

As of build 9926 (at least - it may have been an earlier build), this is configurable via the GUI.

Open Explorer and go to View, then click Options (or Change folder and search options). In General tab you can change Open File Explorer to This PC

The registry setting that controls this behavior is the DWORD value name LaunchTo under HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Explorer\Advanced

Set the DWORD accordingly:

1 = This PC
2 = Quick access

Win 8.1:

If you don't use Libraries at all, you can hide them. This makes your default "This PC".

1. In File Explorer, select the View tab and click the Options button.
2. In the General tab, under Navigation Pane, uncheck "View Libraries"
3. Click OK, then restart File Explorer.

Win 10:

1. In File Explorer, select the View tab and click the Options button.

2. Look at "Open File Explorer to:" and click the dropdown.

3. Select 'This PC'

4. Click OK.