The Nonlinear Library LW - Some (problematic) aesthetics of what constitutes good work in academia by Steven Byrnes
Mar 11, 2024
19:25
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Some (problematic) aesthetics of what constitutes good work in academia, published by Steven Byrnes on March 11, 2024 on LessWrong.
(Not-terribly-informed rant, written in my free time.)
Terminology note: When I say "an aesthetic", I mean an intuitive ("I know it when I see it") sense of what a completed paper, project, etc. is ideally "supposed" to look like. It can include both superficial things (the paper is properly formatted, the startup has high valuation, etc.), and non-superficial things (the theory is "elegant", the company is "making an impact", etc.).
Part 1: The aesthetic of novelty / cleverness
Example: my rant on "the psychology of everyday life"
(Mostly copied from
this tweet)
I think if you want to say something that is:
(1) true,
(2) important, and
(3) related to the psychology of everyday life,
…then it's NOT going to conform to the aesthetic of what makes a "good" peer-reviewed academic psych paper.
The problem is that this particular aesthetic demands that results be (A) "novel", and (B) "surprising", in a certain sense. Unfortunately, if something satisfies (1-3) above, then it will almost definitely be obvious-in-hindsight, which (perversely) counts against (B); and it will almost definitely have some historical precedents, even if only in folksy wisdom, which (perversely) counts against (A).
If you find a (1-3) thing that is not "novel" and "surprising" per the weird peer-review aesthetic, but you have discovered a clearer explanation than before, or a crisper breakdown, or better pedagogy, etc., then good for you, and good for the world, but it's basically useless for getting into top psych journals and getting prestigious jobs in psych academia, AFAICT. No wonder professional psychologists rarely even try.
Takeaway from the perspective of a reader: if you want to find things that are all three of (1-3), there are extremely rare, once-in-a-generation, academic psych papers that you should read, and meanwhile there's also a giant treasure trove of blog posts and such. For example:
Motivated reasoning is absolutely all three of (1-3). If you want to know more about motivated reasoning, don't read psych literature, read
Scout Mindset.
Scope neglect is absolutely all three of (1-3). If you want to know more about scope neglect, don't read psych literature, read
blog posts about Cause Prioritization.
As it happens, I've been recently trying to make sense of social status and related behaviors. And none of the best sources I've found have been academic psychology - all my "aha" moments came from blog posts. And needless to say, whatever I come up with, I will also publish via blog posts. (
Example.)
Takeaway from the perspective of an aspiring academic psychologist: What do you do? (Besides "rethink your life choices".) Well, unless you have a once-in-a-generation insight, it seems that you need to drop at least one of (1-3):
If you drop (3), then you can, I dunno, figure out some robust pattern in millisecond-scale reaction times or forgetting curves that illuminates something about neuroscience, or find a deep structure underlying personality differences, or solve the
Missing Heritability Problem, etc. - anything where we don't have everyday intuitions for what's true. There are lots of good psych studies in this genre (…along with lots of crap, of course, just like every field).
If you drop (2), then you can use very large sample sizes to measure very small effects that probably nobody ought to care about.
If you drop (1), then you have lots of excellent options ranging from p-hacking to data fabrication, and you can rocket to the top of your field, give TED talks, sell books, get lucrative consulting deals, etc.
Example: Holden Karnofsky quote about academia
From a
2018 interview (also excerpted
here):
I would say the vast majority of what is g...
(Not-terribly-informed rant, written in my free time.)
Terminology note: When I say "an aesthetic", I mean an intuitive ("I know it when I see it") sense of what a completed paper, project, etc. is ideally "supposed" to look like. It can include both superficial things (the paper is properly formatted, the startup has high valuation, etc.), and non-superficial things (the theory is "elegant", the company is "making an impact", etc.).
Part 1: The aesthetic of novelty / cleverness
Example: my rant on "the psychology of everyday life"
(Mostly copied from
this tweet)
I think if you want to say something that is:
(1) true,
(2) important, and
(3) related to the psychology of everyday life,
…then it's NOT going to conform to the aesthetic of what makes a "good" peer-reviewed academic psych paper.
The problem is that this particular aesthetic demands that results be (A) "novel", and (B) "surprising", in a certain sense. Unfortunately, if something satisfies (1-3) above, then it will almost definitely be obvious-in-hindsight, which (perversely) counts against (B); and it will almost definitely have some historical precedents, even if only in folksy wisdom, which (perversely) counts against (A).
If you find a (1-3) thing that is not "novel" and "surprising" per the weird peer-review aesthetic, but you have discovered a clearer explanation than before, or a crisper breakdown, or better pedagogy, etc., then good for you, and good for the world, but it's basically useless for getting into top psych journals and getting prestigious jobs in psych academia, AFAICT. No wonder professional psychologists rarely even try.
Takeaway from the perspective of a reader: if you want to find things that are all three of (1-3), there are extremely rare, once-in-a-generation, academic psych papers that you should read, and meanwhile there's also a giant treasure trove of blog posts and such. For example:
Motivated reasoning is absolutely all three of (1-3). If you want to know more about motivated reasoning, don't read psych literature, read
Scout Mindset.
Scope neglect is absolutely all three of (1-3). If you want to know more about scope neglect, don't read psych literature, read
blog posts about Cause Prioritization.
As it happens, I've been recently trying to make sense of social status and related behaviors. And none of the best sources I've found have been academic psychology - all my "aha" moments came from blog posts. And needless to say, whatever I come up with, I will also publish via blog posts. (
Example.)
Takeaway from the perspective of an aspiring academic psychologist: What do you do? (Besides "rethink your life choices".) Well, unless you have a once-in-a-generation insight, it seems that you need to drop at least one of (1-3):
If you drop (3), then you can, I dunno, figure out some robust pattern in millisecond-scale reaction times or forgetting curves that illuminates something about neuroscience, or find a deep structure underlying personality differences, or solve the
Missing Heritability Problem, etc. - anything where we don't have everyday intuitions for what's true. There are lots of good psych studies in this genre (…along with lots of crap, of course, just like every field).
If you drop (2), then you can use very large sample sizes to measure very small effects that probably nobody ought to care about.
If you drop (1), then you have lots of excellent options ranging from p-hacking to data fabrication, and you can rocket to the top of your field, give TED talks, sell books, get lucrative consulting deals, etc.
Example: Holden Karnofsky quote about academia
From a
2018 interview (also excerpted
here):
I would say the vast majority of what is g...