How to change SQL Server authorization mode without Management Studio

Here is what Management Studio does to change the authentication mode from mixed to Windows only:

EXEC xp_instance_regwrite 
    N'HKEY_LOCAL_MACHINE', 
    N'Software\Microsoft\MSSQLServer\MSSQLServer', 
    N'LoginMode', 
    REG_DWORD, 
    1;

And from Windows only back to mixed:

EXEC xp_instance_regwrite 
    N'HKEY_LOCAL_MACHINE', 
    N'Software\Microsoft\MSSQLServer\MSSQLServer', 
    N'LoginMode', 
    REG_DWORD, 
    2; -- only difference is right here

You can call the same command from various sources that can connect to SQL Server such as SQLCMD, PowerShell, VBScript, C#, etc. Or you can log directly onto the server, navigate to that registry key, and change the value manually (as @marc_s suggested).

Note that in all cases you have to restart SQL Server in order for the changes to take effect. You can view the first several entries in the new error log on restart to validate that the authentication mode is correct. It will say (for mixed):

date/time    Server    Authentication Mode is MIXED.