It's actually a bit hyped for particle physics. It ignores the fact that you're very likely to see a lot of "significant" effects in physics because there are tens of thousands of data analyses performed every year, so we're bound to find some chance occurrences at this significance level. See the diphoton excess a few years ago.

This is why the particle physics community has a very strict unofficial standard of 5 sigma for significance. People don't generally publish "serious" papers at 3 sigma.

We've only observed a small handful of black hole collisions though so seeing something at 2.9 sigma would be somewhat more surprising, right? Definitely not rising to the level of a discovery yet of course.

Only a small handful of observations (only one, last I heard, but I may be out of date), but lots and lots of analyses. And for every scientist that has their own model and does their own data processing, there's a chance that the model lines up with some arbitrary noise in the data.

You also have to add in that once these experimental results were released that a hundred or so theoretical physicists immediately started working on massaging the data into supporting their pet theories. After every anomalous result there's immediately hundreds of published papers by someone trying to simply be the first to publish in case their idea happens to pan out. It is a kind of shotgun approach to winding up being the next Dirac discovering the positron from math or whatever.

Since there's 99 other PhDs who probably looked at this data and found their pet theories didn't match the data and haven't been able to publish you have to account for that filtering effect that this was the 1-in-100 paper that managed to match the data. Adding that "Look Elsewhere Effect" to the 2.9 sigma would push the global significance (in the literal sense of global -- meaning all the research teams across the whole world) of this result down into meaninglessness.

Of course its likely that any discovery would start out looking like something on the edge of significance exactly like this. The safe bet is that this disappears, but all we can do is wait for more data to come in and see if the significance improves or disappears.

And I do really hope that someone finds something like this via the LIGO data. I'm convinced there's something very interesting out there to find, and sooner or later it should pop up experimentally and shake up our model of the universe.

Yes, this is basically a distributed version of the issues discussed in psychology regarding researcher degrees of freedom with a single data set. If you throw enough models at the data, one of them is bound to stick, whether it is predictive or not.

You're right, of course - while not a particle physicist, I know about the sigma standards, and mistakenly assumed others would see the "2.9 sigma" and immediately knew what that implied.

it wasn't obvious to this layman. thanks for clarifying.

I assume one reason for that high standard is that particle physics experiments are repeatable and can be observed with very high precision.

None of that is true of LIGO observations of black hole collisions.

yea, 1 in 270 seems fairly likely given the circumstances.

There was a time when Nature was a highly respected publication. Now, their headlines are at tabloid level.

TFA is in "news & comments" rather than "research" (which is the respected part).

It's a for-profit publication, and I agree -- sometimes it shows.