Re: Guest Post: "Reproduceablity-enhancing" measures

1
Also, Gelman's improved list of reproduceability-enhancing measures is hilarious, as it boils down to "Do higher quality work." But said more tactfully.

Some years ago I started outlining something I wanted to write in a sort-of-professional context* and realized the conclusion was going to be along the lines of "we talk about [name for an approach to the work that has been influential for a number of years] in methodological terms but it looks like what people really want to say is that they wished people would do (or would have done) their jobs better." I didn't really want to go there and fortunately I'd already basically abandoned my blog so it wasn't hard to not write.

*personal blog, but the post would have been about a couple of articles in my then-field and ensuing discussion in various places online


Posted by: fake accent | Link to this comment | 09-27-24 11:11 AM
horizontal rule
2

Here's another Hullman post from partway through the process that lays out some more of the concerns as they initially arose.


Posted by: teofilo | Link to this comment | 09-27-24 11:21 AM
horizontal rule
3

Apparently the one guy really does think there might be some supernatural reason that effect sizes decline over time. I wondered if he was inspired by quantum mechanics, and sure enough:

Less likely, but not inconceivable, is an effect stemming from some unconventional process. Perhaps, just as the act of observation has been suggested to affect quantum measurements, scientific observation could subtly change some scientific effects. Although the laws of reality are usually understood to be immutable, some physicists, including Paul Davies, director of the BEYOND: Center for Fundamental Concepts in Science at Arizona State University in Tempe, have observed that this should be considered an assumption, not a foregone conclusion.

Posted by: teofilo | Link to this comment | 09-27-24 11:26 AM
horizontal rule
4

Top Way of Increasing Reproducability

Eliminate regulations on child car seats.


Posted by: peep | Link to this comment | 09-27-24 11:27 AM
horizontal rule
5

"Although the laws of reality are usually understood to be immutable" is a hell of a way to start a sentence in a top-tier scientific journal.


Posted by: teofilo | Link to this comment | 09-27-24 11:41 AM
horizontal rule
6

The Beyond Center at ASU seems to be pretty much what you'd expect it to be.


Posted by: teofilo | Link to this comment | 09-27-24 11:51 AM
horizontal rule
7

Kind of bothers me that I don't see a retraction notice on the pre-print linked to from the original Gelman blog post. Maybe I'm missing something. The published paper clearly shows the retraction.


Posted by: fake accent | Link to this comment | 09-27-24 12:07 PM
horizontal rule
8

"Do higher quality work."

Yeah, sadly or amusingly, this is surprisingly pertinent to a lot of science.


Posted by: chill | Link to this comment | 09-27-24 12:59 PM
horizontal rule
9

2: thanks for filling in that gap. That was one of the posts I read but I missed it when compiling these for a post.


Posted by: chill | Link to this comment | 09-27-24 1:00 PM
horizontal rule
10

2,9, in fact that's the core post, isn't it? Start there anyone who wants the deets.


Posted by: chill | Link to this comment | 09-27-24 1:01 PM
horizontal rule
11

Yeah, I think that's the one that really lays out what's going on.


Posted by: teofilo | Link to this comment | 09-27-24 1:10 PM
horizontal rule