Point/Counterpoint

Saturday Night Live

Back in March, a project known as BICEP2 held a press conference where they announced the discovery of inflation in the early universe. This created quite a stir in the press. When the announcement was made, the results had just been made public, and their paper had not been peer reviewed. As everyone started analyzing the work, what initially looked like a pretty strong result started to look less strong. Then there started to be murmurings that perhaps the announcement had been premature.

The key aspect of the BICEP2 results is the analysis of polarization within the cosmic microwave background (CMB). Specifically there is a type of polarization known as B-mode polarization. This type of polarization can be caused by gravitational lensing of the CMB by galaxies and the like, but it can also be caused by primordial gravitational waves formed during the earliest moments of the universe. Distinguishing between these two types of B-modes requires a detailed analysis of the CMB data, which is very tricky to do. The BICEP2 team ran their analysis, and found a polarization signature too strong to be caused by gravitational lensing alone, thus pointing to inflation as a cause.

But the real challenge is making sure your data is relatively free of foreground effects. The CMB is the most distant thing we can observe in the universe, and that means everything else is between the CMB and us. All of that stuff can distort the CMB, which can give you a false positive result. This is particularly true in the region around the plane of our galaxy, which is sometimes referred to as the zone of avoidance. These foreground effects are the Achilles heel of any inflation result.

Soon after the BICEP2 announcement, some initial results from the Planck telescope mapped our galaxy’s magnetic field through the polarization of gas and dust within the Milky Way. This is exactly the type of foreground effect that can distort results. Another paper noted that an effect known as radio loops could produce similar B-mode polarization, and it wasn’t clear whether BICEP2 had taken this into account. Later it was found that the BICEP2 team had used some tentative data from Planck by extracting the data from a PDF slide.

Now a couple¹ of new papers² argue that there are sufficient foreground effects to wash out the results of BICEP2, thus the BICEP2 result neither confirms nor denies the existence of early inflation. The BICEP2 team argues that even with foreground corrections the results are still strong enough to be valid.

It is important to keep in mind that the BICEP2 results are still going through peer review. Whether the result holds up or not is an ongoing question. But this is happening very publicly, and it was initiated with a very public announcement that a great discovery had been made. If it turns out the results don’t pan out, it can give an impression that we really don’t understand the universe, or that this kind of public point-counterpoint is how peer review works. If that’s the case, how is that any different from the public debates we see on global warming, or the safety and effectiveness of vaccines?

Of course there is a difference between intense discussion over BICEP2, which is an initial discovery of a new effect, and the endless debate over arguments that have been refuted by the evidence again and again. I have in the past compared peer review to whacking a result like a piñata, but there also comes a point where you stop whacking and come to a conclusion. Both are a part of peer review.

Much of this public drama began because BICEP2 decided to make a public announcement. Maybe they shouldn’t have. Maybe the professionals should keep results to themselves until it passes peer review. Only then should results be announced. It would reduce the drama, and maybe we wouldn’t have such skepticism on controversial topics strongly supported by the evidence.

While I can see the benefit of that approach, I’m not sure that I agree. I think there is value in discussing results that are tentative, and in a public discussion of its strengths and weaknesses. Perhaps we shouldn’t be so eager to have press conferences, but once the paper was released, it was public. That’s true of most research these days. In my field preprints appear on the arxiv long before they are published in a journal. Trying to keep results private would only make it more difficult to peer review work. I also think there is a responsibility to ensure science is done publicly. Most of the work in astrophysics is funded through government support. The general public pays for it, and should have access to not only the results, but also the data supporting it.

That said, I think there is also a responsibility to be honest and thoughtful about scientific discoveries. Journalists need to move beyond copy-pasta press releases and point-counterpoint simply for the sake of argument. When results are tentative that should be made clear, and when the data is conclusive, that should be made clear as well. As scientists we also need to be willing to communicate ideas and results clearly. We can’t simply disseminate our work among our peers and consider the job done. As a society we’ve moved beyond that. And as readers we need to be mindful not to feed the hype machines so prevalent among online media. Science is not simply about data and facts. It is a process that pushes us to be better. To be honest and to think critically.

I’m not sure whether the BICEP2 results will hold up or not. What I can say is that as it whatever the outcome you’ll hear about it here. Because it’s not about the results, it’s about the process. In the end, that’s what makes science so cool.

ht to Phillip Buckhaults for prompting this post.