The Nonlinear Library LW - Linkpost: They Studied Dishonesty. Was Their Work a Lie? by Linch
Oct 2, 2023
02:43
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Linkpost: They Studied Dishonesty. Was Their Work a Lie?, published by Linch on October 2, 2023 on LessWrong.
This is a linkpost for Gideon Lewis-Kraus's New Yorker article on the (alleged) Ariely and Gino data fraud scandals. I've been following this situation off-and-on for a while (and even more so after the original datacolada blog posts). The basic story is that multiple famous professors in social psychology (specializing in dishonesty) have been caught with blatant data fraud. The field to a large extent tried to "protect their own," but in the end the evidence became too strong. The suspects have since retreated to attempting to sue datacolada (the investigators).
Despite the tragic nature of the story, I consider this material hilarious high entertainment, in addition to being quite educational.
The writing is also quite good, as I've come to expect from Gideon Lewis-Kraus (who locals might have heard of from his in-depth profiles on Slate Star Codex, Will MacAskill, and the FTX crash).
Some quotes:
If you tortured the data long enough, as one grim joke went, it would confess to anything. They called such techniques "p-hacking." As they later put it, "Everyone knew it was wrong, but they thought it was wrong the way it's wrong to jaywalk." In fact, they wrote, "it was wrong the way it's wrong to rob a bank."
Ziani [a young grad student] found Gino's results implausible, and assumed that they had been heavily p-hacked. She told me, "This crowd is used to living in a world where you have enough degrees of freedom to do whatever you want and all that matters is that it works beautifully." But an adviser strongly suggested that Ziani "build on" the paper, which had appeared in a top journal. When she expressed her doubts, the adviser snapped at her, "Don't ever say that!" Members of Ziani's dissertation committee couldn't understand why this nobody of a student was being so truculent. In the end, two of them refused to sign off on her degree if she did not remove criticisms of Gino's paper from her dissertation. One warned Ziani not to second-guess a professor of Gino's stature in this way. In an e-mail, the adviser wrote, "Academic research is like a conversation at a cocktail party. You are storming in, shouting 'You suck!' "
A former senior researcher at the lab told me, "He assured us that the effect was there, that this was a true thing, and I was convinced he completely believed it."
The former senior researcher said, "How do you swim through that murky area of where is he lying? Where is he stretching the truth? What is he forgetting or misremembering? Because he does all three of those things very consistently. So when it really matters - like with the auto insurance - which of these three things is it?"
(Meme made by myself)
Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org
This is a linkpost for Gideon Lewis-Kraus's New Yorker article on the (alleged) Ariely and Gino data fraud scandals. I've been following this situation off-and-on for a while (and even more so after the original datacolada blog posts). The basic story is that multiple famous professors in social psychology (specializing in dishonesty) have been caught with blatant data fraud. The field to a large extent tried to "protect their own," but in the end the evidence became too strong. The suspects have since retreated to attempting to sue datacolada (the investigators).
Despite the tragic nature of the story, I consider this material hilarious high entertainment, in addition to being quite educational.
The writing is also quite good, as I've come to expect from Gideon Lewis-Kraus (who locals might have heard of from his in-depth profiles on Slate Star Codex, Will MacAskill, and the FTX crash).
Some quotes:
If you tortured the data long enough, as one grim joke went, it would confess to anything. They called such techniques "p-hacking." As they later put it, "Everyone knew it was wrong, but they thought it was wrong the way it's wrong to jaywalk." In fact, they wrote, "it was wrong the way it's wrong to rob a bank."
Ziani [a young grad student] found Gino's results implausible, and assumed that they had been heavily p-hacked. She told me, "This crowd is used to living in a world where you have enough degrees of freedom to do whatever you want and all that matters is that it works beautifully." But an adviser strongly suggested that Ziani "build on" the paper, which had appeared in a top journal. When she expressed her doubts, the adviser snapped at her, "Don't ever say that!" Members of Ziani's dissertation committee couldn't understand why this nobody of a student was being so truculent. In the end, two of them refused to sign off on her degree if she did not remove criticisms of Gino's paper from her dissertation. One warned Ziani not to second-guess a professor of Gino's stature in this way. In an e-mail, the adviser wrote, "Academic research is like a conversation at a cocktail party. You are storming in, shouting 'You suck!' "
A former senior researcher at the lab told me, "He assured us that the effect was there, that this was a true thing, and I was convinced he completely believed it."
The former senior researcher said, "How do you swim through that murky area of where is he lying? Where is he stretching the truth? What is he forgetting or misremembering? Because he does all three of those things very consistently. So when it really matters - like with the auto insurance - which of these three things is it?"
(Meme made by myself)
Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org