How to make \{ and \} automatically have \left and \right

It's a very bad idea, really. See Is it ever bad to use \left and \right?

However, we can find the definitions of \{ and \} in latex.ltx and see

\DeclareRobustCommand{\{}{\ifmmode\lbrace\else\textbraceleft\fi}
\DeclareRobustCommand{\}}{\ifmmode\rbrace\else\textbraceright\fi}

so it's quite simple: add

\let\{\relax \let\}\relax
\DeclareRobustCommand{\{}{\ifmmode\left\lbrace\else\textbraceleft\fi}
\DeclareRobustCommand{\}}{\ifmmode\right\rbrace\else\textbraceright\fi}

to your document preamble. The first line is to avoid warnings about the redefinition. This still allows using \{ and \} in text mode (without \left and \right, of course, which wouldn't make sense in that context).

Much better is to load mathtools and do

\DeclarePairedDelimiter{\braces}{\{}{\}}

so you can do

\braces{x} \braces[\big]{x} \braces[\Big]{x} \braces[\bigg]{x} \braces[\Bigg]{x}

for manual size selection of

\braces*{x}

for automatic size selection.

Your attempts

Attempt 1

For \edef\{{\left\{} we need to check the various expansion steps. The first level expansion of \{ is

\x@protect\{\protect\{•

(the bullet stands for a space in the control sequence name). The next expansion step is

\ifx\protect\@typeset@protect\else\@x@protect\{\fi\protect\{•

In the document preamble, \protect is the same as \@typeset@protect (both are \relax), so the true branch is followed, yielding nothing, so you end up with

\protect\{•

that becomes

\protect\ifmmode \lbrace \else \textbraceleft \fi

(because \protect is not expandable). Since you are not in math mode, the false branch is followed, leading to

\protect\textbraceleft

(there should be a trailing \fi, but that's eventually disappearing, because \edef does full expansion). It's not finished, yet, because we need to know what \textbraceleft does. We get

\protect\OMS-cmd \textbraceleft \OMS\textbraceleft

Oh, well, what's this? \OMS-cmd is something that should never appear inside an \edef, because it wants to perform assignments.

You're doomed.

Attempt 2

Your \edef\{{\left\string{} defines \{ to be \left{ (where the brace is turned into a “printable character”). This is illegal, because { as a printable character has its \delcode set to −1, so you get a “missing delimiter” error at \{.

Attempt 3

With \edef\{{\left\string\{} you get a puzzling output. You are actually applying \left to a backslash of category code 12 and this has a positive \delcode, namely "26E30F meaning that character "6E from math family 2 is used if no sizing is needed (the symbol font has indeed a backslash there) or character "0F from math family 3 otherwise (and again you get a backslash). The left brace tells TeX to print use the math code "007B, so the character in slot "7B from math family 0, which is the ordinary text font and contains the en-dash at that slot.

Why does \edef\langle{\left\langle} work?

Because \langle is defined in a very different way: its definition is

\delimiter"426830A

(with a space at the end) and so the \edef just does the same as

\def\langle{\left\delimiter"426830A }

because the expansion of \langle consists of unexpandable tokens.

The usual trick is to \let the old command to an "orig" version and use it in the definition of a changed command:

\let\leftbraceOrig=\{
\let\rightbraceOrig=\}
\def\{{\left\leftbraceOrig}
\def\}{\right\rightbraceOrig}

Result:

\begin{equation}
\{\int_{-\infty}^\infty \frac{e^{-x^2}}{2} dx \}
\end{equation}

The cause of your problem is that LaTeX defines \{ as \ifmmode\lbrace \else\textbraceleft\fi. If you are using LaTeX then your \edef goes to the branch \textbraceleft.

But you can define:

\edef\{{\left\lbrace}
\edef\}{\right\rbrace}