Did the researchers at Fermilab find a fifth force?
Let me add an experimentalist's opinion. From the plots shown by Lubos above, one sees that two distributions are being subtracted in order to bring up the signal. A Monte Carlo background from expected interactions with a number like 500 events/8GeV, and a similar number for the experimental data. As the Monte Carlo background has no error bars, I presume the statistics are much higher and the histogram is just normalized to the number of events in the data.
The statistical error in a number of events for each data bin is about $\sqrt{500}$=22.4 events. If I measure the error in the difference plot you show above, it is not larger than this number for each bin. This means they have not included systematic errors in their error estimates. One such is the error produced by the shift in energy as discussed by Lubos. This should have been added to the errors by varying the Monte Carlo background according to the 1-sigma error of the energy of the jet and added to the error bars. There are other systematic errors one can think about in subtracting data from Monte Carlo events and in cut decisions. The effect of each cut should be in a systematic error. Each variable used in the cuts (including the 8 Gev binning above) should be varied in the Monte Carlo within the error bars of the variable and the error estimate.
Note that systematic errors are added linearly and not in quadrature.
If this bump is not a statistical fluctuation but a result of underestimation of systematic errors, even if CDF doubles the statistics the problem will remain. It is independent experiments that will inform us of whether it is a statistical fluctuation, an analysis artifact or a true signal.
The likely answer is No, the whole signal is just an artifact of the difficult statistical manipulations. A Weizmann Institute physicist has noted that the peak has been shifted by one bin and the whole discovery could therefore be due to a pile-up effect or jet energy calibration; a small shift of the jet energy removes the effect. See also
http://motls.blogspot.com/2011/04/fermilab-cdf-new-force-press-conference.html
for more details. Additional doubts about the valid statistical procedures were mentioned on Tommaso Dorigo's blog, too.
An adjustment of jet energy by 3 percent is enough to make the signal totally insignificant. Animation by Tommaso Tabarelli de Fatis, a member of CMS.
The text above also reviews some theoretical literature and clarifies that if the signal happens to be real - and the D0 Collaboration is going to release its own verdict about the phenomenon in a few weeks - the most likely explanations are
a new, fifth force - one mediated by a Z' boson (with the mass of 144 GeV or so) which is the particle that was decaying to two jets in those events. It must be leptophobic - (almost) no interactions with the leptons - and such Z' bosons, messengers of new $U(1)$ groups similar to the electroweak Z' boson but independent from them, are predicted by a very large fraction of grand unified theories and/or string theory models; articles by Alon Faraggi and many others are listed above; the simplest (but not the only) way to get a new leptophobic $U(1)$ is to obtain it as a piece of a color $SU(4)$ broken to $SU(3)\times U(1)$
a technipion, a particle analogous to a pion in technicolor theories that break the electroweak symmetry by similar composite particles, and not necessarily scalar ones (so there is no true Higgs particle in those theories); a paper by Kenneth Lane et al. is linked above, too; technicolor has been thought to be nearly dead for years - the optimistic proposal to interpret the bump as a technicolor effect doesn't solve all the detailed problems with technicolor theories
a stop squark in a supersymmetric theory - but it must be an R-parity-violating version of supersymmetry (e.g. because the new superpartners were not produced in pairs) which is unattractive for many other reasons (in simple terms, R-parity-violating theories are only consistent with proton stability given immense new assumptions, and they don't produce natural well-behaved dark-matter candidates); an article is linked to by the weblog above, too