Also, Gelman's improved list of reproduceability-enhancing measures is hilarious, as it boils down to "Do higher quality work." But said more tactfully.

Some years ago I started outlining something I wanted to write in a sort-of-professional context* and realized the conclusion was going to be along the lines of "we talk about [name for an approach to the work that has been influential for a number of years] in methodological terms but it looks like what people really want to say is that they wished people would do (or would have done) their jobs better." I didn't really want to go there and fortunately I'd already basically abandoned my blog so it wasn't hard to not write.

*personal blog, but the post would have been about a couple of articles in my then-field and ensuing discussion in various places online

Here's another Hullman post from partway through the process that lays out some more of the concerns as they initially arose.

Apparently the one guy really does think there might be some supernatural reason that effect sizes decline over time. I wondered if he was inspired by quantum mechanics, and sure enough:

Less likely, but not inconceivable, is an effect stemming from some unconventional process. Perhaps, just as the act of observation has been suggested to affect quantum measurements, scientific observation could subtly change some scientific effects. Although the laws of reality are usually understood to be immutable, some physicists, including Paul Davies, director of the BEYOND: Center for Fundamental Concepts in Science at Arizona State University in Tempe, have observed that this should be considered an assumption, not a foregone conclusion.

Top Way of Increasing Reproducability

Eliminate regulations on child car seats.

"Although the laws of reality are usually understood to be immutable" is a hell of a way to start a sentence in a top-tier scientific journal.

The Beyond Center at ASU seems to be pretty much what you'd expect it to be.

Kind of bothers me that I don't see a retraction notice on the pre-print linked to from the original Gelman blog post. Maybe I'm missing something. The published paper clearly shows the retraction.

"Do higher quality work."

Yeah, sadly or amusingly, this is surprisingly pertinent to a lot of science.

2: thanks for filling in that gap. That was one of the posts I read but I missed it when compiling these for a post.

2,9, in fact that's the core post, isn't it? Start there anyone who wants the deets.

Yeah, I think that's the one that really lays out what's going on.

It is pretty amazing that even open science advocates writing about open science end up here. It shows you just how powerful the forces are pushing you to tell a plausible popular story from the results you get. The social structure of research, and possibly human nature, dictate these outcomes. Having five labs replicating large studies multiple times - that's a huge investment and effort. It's very difficult to end with a shrug and a low tier publication.

It reminds me of a quote from an old econometrics textbook I had where the author jokingly mentions at his own department (Wisconsin, I think) how the "high priests of statistics on the upper floors" (where seminars were taught and given) would become "wanton sinners in the basement" where the computing terminals to do actual analysis were located.