How to format a number with an error?

Update

This seems to do what you want a bit more simply than your own solution.

f[{a_, b_}] :=
  NumberForm[
    SetAccuracy[a ± b, Accuracy @ SetPrecision[b, 2]],
    ExponentFunction -> (Null &)
  ] // ToString // Quiet

Your test:

f[{12.3456, 0.0123}]
f[{12.3456, 0.123}]
f[{129.3456, 1.23}]
f[{-129.3456, 12.3}]
f[{12.9999, 0.02}]
f[{1, 100}]
f[{-1, 100}]
f[{12345.6789, 345}]
f[{12345.6789, 3456}]

"12.346 ± 0.012"

"12.35 ± 0.12"

"129.3 ± 1.2"

"-129. ± 12."

"13.000 ± 0.020"

"0. ± 100."

"0. ± 100."

"12350. ± 340."

"12300. ± 3500."

f[{a_, b_}, prec_] := 
 Block[{x = {a, b}, y}, y = SetAccuracy[x, prec]; 
  ToString @ y[[1]] <> " ± " <> ToString @ y[[2]]]

f[{12345.6789, 0.0012345}, 4]

"12345.679 ± 0.001"

f[{12345.6789, 3.456}, 2]

"12345.7 ± 3.5"

f[{12345.6789, 345}, 1]

"12346. ± 345."

f[Round[#, 5]& @ {12345.6789, 345}, 1]

"12345. ± 345."

Building on @Mr.Wizard♦ 's answer I have come up with the following. I use SetPrecision to set the significant figures of the error to two (ala Mr.W). I then use RealDigits, which tells me about the number of significant figures either side of the decimal place, to calculate the necessary precision for the number, a. I can the use SetPrecision again, instead of Round, but I have to catch, using If, the cases where the required precision is $\le 0$.

I avoid using InputForm as I think this is what leads to the 0.0012000000000000001 problem. But I do use NumberForm to convert 1.*10^2 to 100.. The Quiet is there to turn off the warning about the presence of trailing zeros - which I actually want!

My function:

f[{a_, b_}] := Module[{aa, bb, p},
  bb = SetPrecision[b, 2];
  p = Last[RealDigits[a]] - Last[RealDigits[bb]] + 2;
  aa = If[p > 0, SetPrecision[a, p], 0];
  Quiet[
   ToString@
    NumberForm[PlusMinus @@ {aa, bb}, ExponentFunction -> (Null &)],
   {NumberForm::sigz}
   ]
  ]

This passes all the test cases I have outlined so far above, please let me know if you find any tests you think it fails. And obviously if you find a more elegant implementation!

f[{12.3456, 0.0123}]
f[{12.3456, 0.123}]
f[{129.3456, 1.23}]
f[{-129.3456, 12.3}]
f[{12.9999, 0.02}]
f[{1, 100}]
f[{-1, 100}]
f[{12345.6789, 345}]
f[{12345.6789, 3456}]

"12.346 ± 0.012"

"12.35 ± 0.12"

"129.3 ± 1.2"

"-129. ± 12."

"13.000 ± 0.020"

"0 ± 100."

"0 ± 100."

"12350. ± 340."

"12300. ± 3500."