How can I make my .NET application erase itself?

You will never be able to guarantee that this will work, as long as you require a physical presence on the machine. For example:

• What if the app fails to release a resource in a timely fashion while you're trying to delete it? An error occurs, and the app remains.
• The behavior of one app starting another which then deletes the first app is very suspicious from an AV perspective. You are likely to trigger defenses on a user's machine which may kill the process that's trying to kill your original app.
• If you do something like delete a file at reboot, what if the user moves your file in between or makes a copy? It's not in the original spot anymore, and the app remains.

If your application requires this level of security, consider hosting it on a machine you control (e.g., by providing a web service and letting a stub client access it that way).

On a somewhat related note, one is also tempted to speculate about the motives of someone who (1) requires a physical presence on someone's machine and (2) wants to delete the evidence that the app existed.

A correction to @Bobby answer, in case people will find it useful - executable path needs to be quoted. Additionally, below is setting cmd.exe window to be hidden (otherwise it flashes as a black console window) and converted to run without relying on System.Windows.Forms assembly (the Application class).

 
var exepath = Assembly.GetEntryAssembly().Location;              
var info = new ProcessStartInfo("cmd.exe", "/C ping 1.1.1.1 -n 1 -w 3000 > Nul & Del \"" + exepath + "\"");
info.WindowStyle = ProcessWindowStyle.Hidden;
Process.Start(info).Dispose();
Environment.Exit(0);

There's a great CodeProject Article about this topic.

Edit: Basically it's a simple cmd-call which will delete the specified files after some seconds.

Process.Start("cmd.exe", "/C ping 1.1.1.1 -n 1 -w 3000 > Nul & Del " + Application.ExecutablePath); 
Application.Exit();

There's a MoveFileEx API, which, when given a MOVEFILE_DELAY_UNTIL_REBOOT flag, will delete specified file on next system startup.