The rise of citation tracking and institutional metrics

In this eye-opening conversation, Malcolm and Simone Collins expose the myth of "trust the science" in today's world. What people really mean is "trust the peer-review bureaucracy"—a system that's only about 50-70 years old, riddled with failures, and openly admitting its own decline.

They dive into the landmark 2023 Nature study showing scientific papers and patents have become dramatically less disruptive since the mid-20th century (decline 90% in disruptiveness for papers). New ideas are incremental, not revolutionary. Metrics like the H-index (invented 2005) and citation farming reward safe, iterative work over bold breakthroughs. Real progress? It's happening outside academia—through tight-knit communities of independent researchers, Substacks, patrons, and informal networks (think biohackers, geneticists like Emil Kirkegaard, or sex/arousal researchers like Aella).

They contrast this with historic "hold my beer" science (e.g., self-experimenting spinal cocaine for epidurals), discuss why bureaucracy killed disruptiveness, and explain why renegade cliques (in genetics, governance, crypto, history) are already replacing rotting university systems. Bonus: thoughts on journalism parallels, prediction markets for kids, and why mainstream media/academia memory-holes inconvenient truths.

If you've ever wondered why innovation feels stalled despite more scientists and funding than ever—this is why. The old system is dead. The new one is already here.Timestamps below. Like, subscribe, and share if you're tired of bureaucratic "truth."

Episode Transcript

Malcolm Collins: [00:00:00] Hello Simone. I’m excited to be here with you today.

Today we are gonna be discussing something that I have brought up in episodes in the past, but it is one of the largest and most systemic fundamental misunderstandings of how our world currently works. That is common in society, which is. How truth is determined and the belief that the system that we have for determining what is true is an old system.

That it is a vetted system or even that it is a system that hasn’t been in constant failure since it came out. It’s a system that itself says it is not working. And here we are going to be talking about the academic system as we understand it right now, when somebody’s like, well, trust the science.

They want you, they’re, they’re, they’re trying to get you to believe that what they’re saying is like, trust the scientific [00:01:00] method, trusted the thing that gave us cars and railroads and industrialization and computers. But what they’re actually saying is trust is this very specific, pure review system and academic bureaucracy for sorting information.

And I wanna point out to them that, that very same bureaucracy they’re asking us to trust they’re, the, the, the height or one of the, the most respected magazines is nature, right? Nature did a landmark study in 2023 on this very issue and we’ll get to it in a bit, but basically they show that since this system has been in place, scientific research has only declined.

It has just been getting worse and worse and worse every year. By

Simone Collins: what measure? By. Its ability to be replicated by,

Malcolm Collins: By its ability. Disruptiveness is [00:02:00] what they were looking at. So like genuinely new rather than iterative ideas.

Simone Collins: Ah, okay. Okay. Like germ theory and antibiotics, that kind thing.

Malcolm Collins: Well, you also see you know, research like the, the cost of research.

So basically the research you get per dollars has been going down dramatically. We’ll go over. This system basically was put in place in the 1950s and in pharma, new drugs per r and d dollar hald every nine years since the 1950s. So it’s, it’s, it’s accumulated getting worse the further we go from the inception of this system.

Simone Collins: That’s horrible. And

Malcolm Collins: the other funny that no Halfing every nine years

Simone Collins: is, I mean, I’m, I’m sure a lot of that’s bureaucratic morass. I bet AI is really going to disrupt that, but also to a certain extent, for example, in the United States. You almost are prevented from getting a new drug introduced without spending a certain [00:03:00] ridiculous amount of money because of the regulatory morass that you’re bogged down by with the FDA.

I

Malcolm Collins: don’t think this is regulatory issues. It correlates way more with the implementation of citation, the citation system, I guess I’ll call it,

Simone Collins: really,

Malcolm Collins: and we’ll go over how that system works, the various variants of that system that have come. Mm-hmm. And people might be surprised how new. So the system that is used most frequently today to judge professors this is the H index and the G index.

Okay. These systems were invented in 2005 and 2006 respectively.

Simone Collins: Oh my gosh. So around the time you and I were graduating from high school. Like,

Malcolm Collins: yeah, that’s when the system that underlies pretty much all of current academia’s hierarchy was invented.

Simone Collins: Right. ‘cause you actually mentioned these, these systems to me only this today, and I knew that citations [00:04:00] mattered to researchers.

I didn’t know that they were tied to their ability to get tenure. And I looked it up and you were right, and I was shocked that. That was one of the most important factors, and it wasn’t just like organizational fit and the extent to which you contribute to the advancement of your field and to which you are able to get, for example, grant money to your department for the university.

I figured that would matter more, even though it’s not necessarily like the best thing in the world at, you know, at least it’s more practical.

Malcolm Collins: What I find so ironic is, you know, there’s that like Redditer who’s like for science,

Speaker 4: As you know, tomorrow is Peace Day and nobody is as excited for the big celebration as I am. I’m not scientifically possible.

Speaker 2: . I’m a super scientist. My father was a super scientist. His father was a super scientist, and his father was, no wait. No. I think he was a milliner. Either way. I’m just not impressed with your tricks.

Malcolm Collins: You know, and then they’ll have like a doctor membrane [00:05:00] here. And the irony is, is the ones who say that the ones who worship what they consider quote unquote science mm-hmm.

Are actually worshiping something of an inversion of what was practiced by the type of historic scientists that would’ve shouted for science before pulling a lever or something like that. Right. Like the, the Frankenstein scientist.

Speaker 3: It’s lie. It’s.

Simone Collins: Yeah.

Malcolm Collins: They actually hate that. Form of science. Mm-hmm. The you know, biohacker lab in your house.

So, you know, sharing

Simone Collins: and it, it was those people who gave us so many of the medical and otherwise interventions. Like light bulbs, like, epidurals that make such a huge difference in our lives.

Malcolm Collins: Dive with epidurals recently. Explain how those are invented. Oh,

Simone Collins: yeah. I mean, I asked the anes anesthesiologist giving me my spinal when I was getting my latest [00:06:00] C-section.

Like, Hey, how are these invented?

Malcolm Collins: He’s attempting to learn Simone pathologically here. I love it.

Simone Collins: Well, you, you’d rather, when they’re sticking a giant needle like into your spinal cord, not think about the fact that they’re doing that. So, yeah, I’m asking other questions. But they, they like checked and they looked it up and they’re like, oh yeah, I remember it was these two guys, and they’re like.

They were experimenting with, with different things and they they just decided to experiment on each other using like, you know, giant, huge gauge needles. ‘cause at the time they didn’t have smaller needles, but I think what it was, was they tried injecting just cocaine directly to their spinal cords.

‘cause what could go wrong? But they were just, they did it on themselves. It wasn’t like, I have a theory about this. We’re gonna test this on lab bias for like the next 10 years. They’re like, Hey man. What would happen if we just put this directly into our spinal cord?

Malcolm Collins: You wanna try to stick this in our spine

Simone Collins: with giant needles?

Oh,

Malcolm Collins: that is, that is fine. That’s the type of thing that [00:07:00] they’re imagining when they say,

Simone Collins: yeah, it, it wasn’t, yeah, it, it was hold my beer. Like, that’s the kind of like attitude and that

Malcolm Collins: is, it’s funny. Same type of redditer who will say that unironically on Reddit is the same person who freaks out at us for doing like polygenic selection or like, germline gene editing on humans.

They’re like, how dare you edit human DNA, that that’s, that could do something new that would be dangerous. Have you sought approval from the authority before flipping that switch? You know, and I, I. I want to get into when they say for science, what they actually are worshiping is a provably failed. And the, and when I say provably failed, I mean the system itself has said,

Simone Collins: yeah,

Malcolm Collins: this has failed.

This is

Simone Collins: not working.

Malcolm Collins: Our own metrics.

Simone Collins: Yeah.

Malcolm Collins: A provably failed bureaucracy, not actual science or the scientific message. So let’s get, get in here. [00:08:00] So the system that we call the, the academic bureaucracy right now, that basically, and the reason I say like this determines truth within the urban monoculture.

This really does when they’re like, what is true and what is not true? This is the system they’re looking to. Right. It is that their final point of this is the fundamental way reality is structured.

Simone Collins: Yeah.

Malcolm Collins: It came together specifically post 1945, fueled by massive government funding, specifically the USGI bill and the National Science Foundation.

In 1950, enrollment exploded from 1.7 million people in 1940 to 7.9 million in 1970, creating quote unquote big science with more papers necessitating rigorous evaluation, peer-reviewed, standardized. Slowly nature in 1967 becoming universal by 1970s to handle volume and ensure quality [00:09:00] amid funding scrutiny.

So the concept, if you hear like, well, you know, what did peer reviewed whatever say this concept wasn’t even made universal till the 1970s. It wasn’t fully adopted by nature until 1967. We are dealing with a fundamentally new system here.

Simone Collins: So how was the publication of something deemed like or, or selected before there was peer review then?

Malcolm Collins: Yeah. We’ll get into that in a second. But basically just the editors of publications like, looked at it and they were like this looks good.

Simone Collins: Okay,

Malcolm Collins: this looks interesting. And that

Simone Collins: is, so that peer review, it was, it was just editor review, which makes sense.

Malcolm Collins: Yes. But people were like, well that’s, you know, it’s, it’s, it’s not fair enough.

It’s not basically, it’s not bureaucratic enough. They needed to add a bureaucratic layer into how papers were chosen and approved instead of handling it like it was actually. So let’s talk about why having it be editor approved is much better than having it be peer reviewed.

Simone Collins: Hmm.

Malcolm Collins: [00:10:00] When you had it be editor reviewed, you were dealing with a few things.

Scientific publications back then made their money based on how many people were paying for them,

Simone Collins: right? Yeah. Yeah.

Malcolm Collins: That is not true anymore, right? You don’t get money based on how many people view your article. You don’t get money, like, and the article, like the big journals themselves do not really get money based on their quality.

They get money based on the universities that have them in their pipeline. And so much of that is sort of. Pre bureaucratic setup that is kind of irrelevant. So the people who are making judgements about whether or not a paper got improved actually don’t really care outside of being Karens, they have no real skin in the game, say, yeah.

But in, if you were an editor back in the day, you needed to actually care. Is this gonna destroy the. Reputation of my journal.

Simone Collins: Yeah.

Malcolm Collins: If it is, then I’m not gonna publish it. Is this something that people are gonna actually care about and want to buy and want to read?

Simone Collins: Mm-hmm. Is this useful to people like tactically useful, such that if they don’t have a subscription now, [00:11:00] will they bother to pay for one because they know they need this?

Yeah.

Malcolm Collins: Yeah. And that was a very good way of handling things. Yes. But then the bureaucrats come in and are like, well, we need to systematize this. We need more layers, we need more standardization. Now this isn’t what caused all of the the breakdown, but I’m giving you an example of one layer, and you have to remember here is this bureaucratization happened at every single layer at the process that puts out what we today call truth.

Simone Collins: Mm-hmm.

Malcolm Collins: Okay.

Simone Collins: Yeah. From funding to ethics boards to promotion, to even just getting your PhD.

Malcolm Collins: And, and by the way, do you want to do, can you guess when even the concept of the academic citation was formalized?

Simone Collins: Was it recent?

Malcolm Collins: Yeah, it was Eugene Garfield’s 1955 proposal, which led to the science citation index, which was [00:12:00] put out in 1964.

It wasn’t even proposed until 1955. In nailing in. Enabling impact tracking. By the 1960s and 1970s, citations became proxies for quality complementing peer review. Institutional research offices emerge Association for Institutional Research in 1965 to analyze metrics for hiring slash promotions.

Tenure density peaked at around 75% in the 1960s, but is now only at around 15% within universities.

Simone Collins: Hmm.

Malcolm Collins: So. And this was made so the universities couldn’t arbitrarily fire someone based on them publishing something that was politically unpalatable. However it basically does not serve that purpose at all anymore.

Tenure doesn’t really mean that much important anymore, like it used to. And the whole system’s pretty corrupted. But to continue here, and I know you’re like, well, AI will remove some of this bureaucracy. No AI’s just replacing it in the same way that we have seen. Citizen journalists. Before I go further, we have our piece on like, [00:13:00] how, how I don’t even like the term citizen journalists, influencers, I’m just gonna call them.

Influencers, have created a system for delivering news to the end reader or listener that is strictly more factual, better and less bias than the journalistic system was at its very height. By. Creating a very interesting sort of layered approach. And I pointed out in that episode, the way that that works is basically at the bottom layer you have the tus who nobody really they don’t have large follower ships but they just produce.

Tons and tons and tons of detailed content that’s very clearly not about putting out a particular agenda or message. It’s just like I am obsessed with military drones, so I have analyzed every, the military drone footage from ex Ukraine, or I am obsessed with like military operations. So I completely analyze like pec of the raid on Maduro and like how this.

Satellite equipment could have gone. [00:14:00] And then at the next layer you have people like us who comes through and synthesize that into something that is entertaining and actually has a large viewership. And then on the layer top of us, you have your, you know, your asthma golds, and your leaflets and your next, and well next does a little bit more original research.

And, from there they, they then synthesize lots of people like us who are intern, synthesizing the layer below us into like a cohesive full, like this is what’s happening in the space. Mm-hmm. In the space of research, we are seeing a similar thing happen. Before I go further with this in the space of research, we are seeing, because I’m seeing this in the people who are like actually interested in advancing our knowledge of what is true.

And this is why we have ala on frequently. Like we might not agree with all of her lifestyle choices. But she is a phenomenal researcher. And people who criticize her research, I think show their. Like naivete in terms of how actual research is happening. When they’re like ALA samples, you [00:15:00] know, I’ll point out, she’ll get like a million.

Respondents to something and they’ll be like, well, her sample is biased. And I’m like, if you look at the way that she normalizes her data what you will see is her less biased than the average sample used by peer reviewed sex research. Like. The, the, the fact that you would say that about her samples and not look at her normalization work shows that either you don’t understand how any of this works, or you don’t understand how corrupted the mainstream publications are at this point.

Yeah. Think around this is, if you look at how sex research is working right now, right? The people who are actually interested in moving it forwards and understanding how people can be like, why does next research matter? It matters a lot in understanding. The parts of the human brain, and it’s the reason why I find it so fascinating and may output deleterious or non beneficial impulses.

And the better we can categorize these impulses and understand what is [00:16:00] actually causing them and how they actually function at like a, a, a baser level we can not only. Better understand ourselves and better control ourselves and better sublimate those animalistic instincts. Obviously that’s not why she does the research, but that’s why I find it so interesting.

But we can also understand wider trends in society. Why do trans people keep doing mass shootings? Like, there was just another one.

I remember when we did the episode on that so many people were like, this isn’t a phenomenon, this isn’t, and there have been two trans mass shootings since that.

And it was so interesting to see the news. Completely disappeared that this had happened. The moment, like I looked at the day that this happened, and it was 10 people died in this one. So it’s a really big one too.

So I double checked and nine people died due to the one recent trans mass shooter. And in Columbine it was 13 people died with two mass shooters. So on a per shooter level, this trans individual was way above the Columbine numbers, and yet the news media immediately blacked [00:17:00] this out. And this happened incredibly recently too.

, This was the one that happened in February. 10th, 2026. The fact that Columbine was a major civilizational issue for about a decade, and this immediately gets memory hold, shows us how much the medium manipulates the conversation on this topic. As we have said, , with the statistics we’re looking at these days, somebody admitting that they’re trans. It’s a bit like saying.

Speaker 7: Nice to meet you, . Listen, if you ever need anybody murdered. Please give me a call and you, you’re giving him card. No.

Code of ethics. I will kill anyone anywhere. Children, animals, old people, doesn’t matter. I just love killing

Speaker 6: you.

Malcolm Collins: And it was not anywhere on the judge report that day. It was not on some of the front pages of like major newspapers.

Right. And I was like, this is wild. Like the extent to which the moment. Disappears. But this is what I mean, like this is why people don’t trust these sources of evidence. Like an academic could not publish a piece investigating why are [00:18:00] trans people more likely to commit mass shootings? Mm-hmm. That is a very interesting question to be asking, right?

Mm-hmm. Now the, the, the, the point I’m making here is if you look at the people actually interested in like this arousal impulse has this evolved everything like this, you have a community that is actively working on this. It is actively pushing things forwards. And the funny thing is it’s a community that’s doing it is very much like one of the historic science communities, right?

Like they all know each other, they all talk like, for example. Diana Fleischman and Jeffrey Miller. And Simone and Malcolm and Ayla may seem like, we have pretty different ideological backgrounds and everything like that, and yet, you know, they’ve all been on our podcast, right? Like we all chat regularly.

And we all are aware of each other because of our social community. As competent researchers. So I don’t even really bother to look at the academic fields that much. I know the people who are actually [00:19:00] producing the new stuff. Right. And I know, and this is if I’m looking at like genetic science, right?

Like if I’m like, oh, what’s the latest genetic science? There’s like five or six names I can go to, right? With like the leading one. I’m sorry, I know you’re not allowed to say this. Is Emil Ard. He produces really good, like I all of his substack posts on the human genetic science because I know that he’s pushing things forwards in ways and I, I don’t mean like, journaling, gene editing here, I’m talking about like.

Genetic correlations in the evolution of human genes. Like, the great study that he and a few other guys did recently, and this is interesting because there were some other researchers involved in this that aren’t as famous as him, but they know that everybody knows, oh, well if this guy’s involved in it, basically almost like he’s a researcher running one of those old journals and he’s putting his name to something, right?

Simone Collins: Mm-hmm.

Malcolm Collins: The great study that they did where they looked at the, full genome sequence of [00:20:00] corpses in Rome over time. Oh yeah.

Simone Collins: Look

Malcolm Collins: at the the genetic markers for the probability that this person was in a modern context. Like what, what level of academic, what. Achievement. Would they have, like how, how far would they go with their schooling?

Basically this is what they is used often when people say intelligence markers and genetics. And what they found was that it did actually collapse at the height of the Roman Empire, there does appear to have been a high amount of disc genetic selection that contributed to the fall of the Roman Empire.

And even today, the people in Rome born have a lower probability of having these. These markers associated with high intelligence than people at the height of the Roman Empire. Yeah. That is how long the DYS genetic impact lasted. Now it has rebounded to some extent. It just hasn’t gotten back up to the height of the Empire.

Mm. And now they’re in a state of DYS genetic class again, so it’s never going to unless they get involved in like genetic engineering tech or genetic selection tech. But the point being. Is that’s a really cool [00:21:00] study, right? And that could have gone through mainstream academic sources, but the people involved in it actively chose not to because they knew the people pushing ahead.

Meaningful genetic variation research in human populations are going to read a meal’s stuff. I note here, because isn’t me endorsing every single meal Kirk Guard’s ever said. I’m just saying that in terms of like, where’s the cutting edge? Who is verifying the cutting edge research? It’s him. And note it’s not just him with that crew, which also cluster of people that all know each other.

You know, and they publish things and they go together and they all get canceled together. But it’s, it’s just something right. And we are already seeing this shift to what is very transparently and obviously gonna replace the university system. And if you’re like, this doesn’t work for potentially more advanced or more nuanced fields that is also just wrong.

We do investing and obviously it’s been leaked and we do investing in the human dream. Really. Gene [00:22:00] editing space, right? Like. And like we know, most of the companies in the space and the companies that are in the space know most of the other companies in the space. They know basically everyone in the space and they all are aware of each other’s research.

They’re aware of how, how each other’s cutting edge methods work. And they talk together basically. Humanity ended up solving the problem of the explosion of scientists going into the academic system. As it turned out that the vast majority of those quote unquote scientists were not scientists, like the scientists of the old day.

They were just bureaucrats and were largely irrelevant from the perspective of advancing the field of science. And then. When it came to the people who were actually interested in advancing particular subdomains of science, the ways that we handled the explosion of population and smart people, because there’s more people doing that today than there was, you know, during the, you know, Victorian period or whatever, right?

Where you had the, the science societies and everything is these people still form their [00:23:00] cliques. They still all hang out and talk together. But the cliques are based around scientific subjects. Which has been actually very interesting to be part of multiple of those cliques because today if you guys look at us and you hear Malcolm and Simone have done.

X or Y in terms of research, I think a lot of our audience would expect that to be on demographics or evolutionary slash cultural anthropology. That is, that is where I think our audience expects it, right? They’re like, okay. Or they’re like, maybe if it’s not that, it’s on, like speculative theology, right?

Mm-hmm. When in reality there was a period this was early in our podcast as well. So like even after we had started popping off as ISTs, we were still like a lower known as ISTs. And certainly our anthropological work wasn’t known as well yet. And our theological work, we hadn’t even gotten into it yet.

But when [00:24:00] we would be asked to go on a podcast or go speak, I always needed to ask them before we went up and spoke. ‘cause I’d say. I think we’re famous for gosh, about half the time it would be Tism but about the other half the time. And our audience may be surprised about this ‘cause we do not cover it on our show as much.

They’d be like, oh, I am really fascinated by your work on governance models and crypto. And we’d occasionally, I’d say like 10% of the time we’d get something in sex research or human evolution. And about 10% of the time just to add some others, we’d get something in the space of positive psychology or, or psychology more broadly.

‘cause we did a lot of stuff in that space.

Simone Collins: Oh, I would say entrepreneurship as a couple was a really big thing to start.

Malcolm Collins: Yeah. Yeah. And they’d have us on for, oh, you’re, you married, couple entrepreneurship invented the ser. People don’t know that. Like our audience doesn’t know that we invented the married search fund model, which is now taught at Harvard and Safer Business schools.

Like we’ve had periods of our life where we [00:25:00] changed the way multiple fields work. But the, the point I’m making here is that these fields are fairly isolated and they’re isolated from each other to the extent where you will get some overlap, like the evolutionary. Arousal research field is aware of like the human variation research field, and we, we talk and socialize occasionally.

But we are still two very different social circles. Mm-hmm. And these two fields are actually completely unaware usually of what’s going on in the crypto field. And that’s why because our, our models in the crypto space, the guy who invented quadratic voting

Glen Weil is who I was thinking of.

Malcolm Collins: said that we had. I think it was one of the best or the best books on governance models he’d ever read.

And this is important because quadratic voting is used in a lot of crypto governance. It’s sort of like one of the foundations of the way. Crypto governance structures work now. And because of that, a lot of our ideas, because he liked our book and the [00:26:00] ideas in our book a lot of the ideas in our book ended up being taken our book on governance, a fragmented guide to governance number one Wall Street Journal bestseller, by the way, at one point, ended up being taken and used in various parts of other crypto projects. And these, these crypto people, when they are looking for governance theory and people who are like on the cutting edge of governance theory, they do not go out and say, what are the academics saying right now? What they do is they go to their.

Leaders in crypto governance, Siri, and they go, what projects do these people affirm? Like, what other thinkers do these people like? I’m gonna go to these other thinkers now.

Simone Collins: Mm-hmm.

Malcolm Collins: But to continue before I go further here but basically what I’m saying is what’s replacing academia is already underway.

Academia’s, just a rotting behemoths file, flailing around at this point and slapping at things. Right? And it’s hilarious that people will cite like, but what does the behemoth think? And I’m like, you mean [00:27:00] that thing that has like its skin falling off and is like barely able to lift its hand anymore?

But to give you an idea. Of how bad things are here. The paper in nature that I mentioned before was called Papers and patents are becoming less disruptive over time. A landmark 2023 nature study analyzed 45 million papers, 1945 to 2010, and 3.9 million patents. 1976 to 2010 using the CD index, a metric of how much new work disrupts prior citations.

It found disruptiveness decline by over 90% for papers and 78% for patents. Oh. Papers, increasingly building incrementally rather than veering in new directions. Similarly, a 2021 Technological forecasting and social change analysis showed the quote unquote flow of ideas. New concepts entering scientific canon declined since the 1970s and [00:28:00] research productivity output per researcher since the 1950s.

Patent quality and Nobel Prize worthy breakthroughs have also waned with fields like physics and medicine showing no upward trend at all. Post to 1950s.

Simone Collins: Ouch.

Malcolm Collins: And then remember, I, I mentioned that doubling every nine years.

Simone Collins: And you’re not even mentioning like the number of patents now that are filed just by patent trolls

Malcolm Collins: Yeah.

Simone Collins: That are actively stifling innovation. It makes me so mad. Patents basically just don’t make sense anymore as far as I am concerned.

Malcolm Collins: Yeah. And so, if we go further here, we’re like, are there people fighting against us? Before I get into the nuances of, of how this stuff works because I don’t know, everybody cares about that. So I try to keep the more boring stuff towards the end of the video and the more interesting stuff towards the beginning of the video.

And there is a group that is fighting against us called Dora. And it, it has actually gained a significant amount of traction [00:29:00] and specifically what they push for is, be completely transparent about evaluation criteria in terms of like hiring and stuff like that. Explicitly state that the scientific content of the work is what matters more than the journal or metrics value all research output data, software, et cetera, and use.

Qualitative impact measures, eg. Influenced by policy, clinical practice, public understanding. This is for funders and institutions. And then for publishers, they say, stop promoting gif. We’ll get to in, in a second as the main selling point. That’s basically how many people cite you. Provide rich article level metrics and encourage proper authorship and controversial statements.

Make reference lists freely usable and reduce artificial limits on references. So primary literature gets proper credit. Hmm. For metrics providers this is Cultivate s, scopist, et cetera. They say be fully transparent about data and methods. Allow unstructured reuse of the data and clearly define and punish metric violation for individual researchers.

When you sit in committees, [00:30:00] judge the science, not the journals, site primary papers and not just reviews. Use diverse article level metrics in your CV or statements. Actively challenge bad assessment practices when you see them.

Simone Collins: Hmm.

Malcolm Collins: I don’t really think that this is a meaningful thing that they’re pushing here.

I think that the truth is, is that you cannot reform academia. Because academia is full of bureaucrats, and bureaucrats are not interested in doing actual science. The, the renegades who are doing actual science are already basically living off of patrons in the way that they did historically. You look at somebody like, like, say if you’re looking at like the human differences, genetic differences, crowd, one of our good friends in that space is Reeb Khan.

Who’s been on the podcast? Great guy. Really good researcher. He used to work for the New York Times, right? But then he, I, I find this hilarious. EB Kahan became accused of being a white. A, a, a white [00:31:00] supremacist, I guess, because he was going through just genetic data. The reason why this is hilarious is if you cannot tell from his name, he is not remotely a white person.

But they decided that he, he went into genetics data that you’re not supposed to talk about. And he got in trouble for that. And now he makes his money off of Substack, right? Like off of, off of actual patrons, right? And that’s the world that we live in now, right? Like we have a, and what’s funny is, you know, he’s not making as much money.

He’s not getting as much money as these big organizations. And if I was president, I would start cutting off the funding. When people are like, oh, we need this r and d funding. Oh, we need this. We need systems that are not corrupt like darpa, that can basically only go to crony insiders. We need systems that are not corrupt, like the existing university funding system.

That fund, you know, academics, quote unquote academics, bureaucrats, really and build systems that can fund the sort of [00:32:00] truly independent and meaningful research that everyone who is an actually active player knows is happening, right. And it’s also funny to me how you know, people like you and me can go out there and have field changing theories.

Which we, we’ve had multiple of on this show. Like, but my favorite, if you wanna go into our most controversial theory it is our one Civilization hypothesis.

Simone Collins: Knew you were gonna say that.

Malcolm Collins: But we had another one recently that I really liked where I was like, I’m disappointed. I can’t, like click a button and be like, everybody needs to know.

This is like an actually pretty big theory in terms of changing how things work.

This is the episode on coop frequency by religious or ethnic background and, , how that changes which countries end up winning wars. I.

Because it changes how competent you can allow your generals to be and how much power you can allow them to have in terms of unilateral thinking and all the way down the military, [00:33:00] you can allow much more autonomy the more you can trust a group to not coup. I.

Simone Collins: I don’t know. You come up with so many different concepts. I have no idea about this one.

Malcolm Collins: I’ll end it in post, I guess. But I remember being like, yeah, this is another big one where it’s just like, it, it changes the way an entire field is structured and it’s weird that we’re able to have these, but it’s mostly because people in these other fields just aren’t thinking anymore.

Mm-hmm. And actually you see this a, a great example of you’re like, well, where else do you see this? Do you just see this in the sciences? It’s like, no. You see this in the in the history as well. So like, if you’re a history nerd these days like, you know who the people who are defining, rethinking how we see history are this generation as people like Ruby Art and Sam Oberg, right?

And, and, and that’s who you’re watching. That’s who you care about, right? You don’t care. About what’s being published by Ex bureaucrat guy, right? Like, if you’re an actual history nerd, that’s who you could because you’re like, oh, this is, this is where we’re getting new [00:34:00] novel theories.

We should, we should go on one of Ruby Yard’s show and push our one civilization hypothesis. So you go how much you push us back on that, because that, that’d be a fun one.

Simone Collins: Well, one thing I think in terms of returning to how things used to be, what’s interesting about how truth used to be determined or where people turned.

To various sources. For truth is the most common on a local level was a tribal elder or following tradition. And the only reason why that worked was basically if the truth that someone believed didn’t work back then and it mattered, you would die. So, you know, a tribal elder was kind of a good bet because they’ve lived so far.

They’ve survived. Meaning that, you know, if, if you do the solution that they did when you cut your leg or when you get sick, you’re more likely to survive. And all of the elders who did a dumb thing when they felt sick or [00:35:00] got injured, probably didn’t survive so they wouldn’t be elders. And that worked.

And also the same thing happened with traditions, right? Like if, if a village or a tribe or some other, a town, a culture, whatever, had a tradition. That ultimately was not true. Doesn’t matter. They’d die, then it wouldn’t be a tradition anymore. So like traditions existed because they correlated with survival.

And it’s interesting how after a while things became bureaucratized and they were dances done around them. I mean, obviously people turned to. Academic, or really, I mean, originally church institutions for higher truths, we’ll say, you know, stuff that people got through Revelations, but it was more about these big picture questions where if you were wrong, it didn’t really mean that you weren’t going to survive.

And I find it very interesting how the original universities, how academia in its. OG Sense was religious. It was a place where priests [00:36:00] studied. That is what Oxford and Cambridge were for. It was for the clergy. They were governed literally by different laws that only affected the clergy. And it is interesting how over time that bureaucratic body took over.

This scientific method as it was innovated that at first the scientific method was a, a very, or, and just in, in general, empirical knowledge was very grassroots or renegade thing. Something that went alongside, passed down traditions or that was just done by weird artists in society is like as an enthusiast’s thing.

And if, if it worked out for the. Enthusiast experimenters who tried these things out, then they would become traditions, they would become things that tribal elders passed down. It was kind of like the artists were the like YOLO people in a society. And then the tribal elders were the ones who solidified whatever worked of the YOLO class that was disposable.

[00:37:00] And then, I don’t know, there was this sort of vague period where academia became increasingly secular.

And this is a really important point of view, having studied the history of knowledge or education, is that academia and the church used to be synonymous. It was the same system. This was true for most of the history of what today we call science. , Where you would go if you were a precocious young person was to study at one of the colleges that trained seminaries.

I mean, that’s what Oxford and Cambridge, and St. Andrews and all of that were. Four originally.

And I mean, obviously there was almost no scientific progress under the Catholics when they controlled these institutions, but as soon as Protestantism came around, this is when we had our greatest leap forwards in terms of scientific progress.

Simone Collins: And I feel like that’s a key turning point when our method for generating truth, that was the beginning of the end. Like if we talk about demographic collapse really being [00:38:00] downstream of the industrial revolution and the atomization of family family jobs, you know, like basically taking everything that used to be produced in the household and having it being made piecemeal outside by a larger economy.

I think the thing that really predated. The crisis of generation of truth was when we took that, which happened within the village and with, you know, renegade experimenters and passing down of knowledge through survival. And, and, and had it be done by a bureaucratic institution instead?

Malcolm Collins: Yeah. Well, I mean, it basically we, we.

Ran the communist experiment on the entire field of science. Mm-hmm. And you know, communism has never worked economically, but we’re like, well, we’ll make it work in terms of information. And it demonstrably didn’t work by the way. I remember the other new theory that I was kind of shocked when I was going over it that no one had had before

which is pointing out.

First, why Islamic Forces hadn’t won a war of [00:39:00] conquest such a long time. And then I pointed out the secondary point to this theory, which is that Catholic Forces actually hadn’t won many wars of conquest in, in a very long time. And that. This appears to be due to the frequency of coups, which then it’s like, why do Protestants have coups so rarely.

And why do Catholics and Muslims have coups so frequently? And you know, I posited a theory there, but what was wild to me wasn’t even just like my theory there. It wasn’t no mainstream academic researcher had even noticed that this was a phenomenon. And yet it’s, it, it’s like, it’s not like a. A p value of like one thing.

It was like there were like three Protestant coups, like successful coups in all of human history. And for Catholics it was like one a generation in most countries. And it’s like how. Why?

Simone Collins: Yeah. It’s kind of a big deal.

Malcolm Collins: Yeah, just a little bit. It’s an interesting question and, and I [00:40:00] realize that the reason is because academics just were not interested in asking questions that could shake things up.

They were interested in iteratively asking something, right? Like, oh, well X has already been established, so like, let’s find out. How did they cook bread back then? Or what were Roman supply lines like? Or, you know, what does, was this formation really like, you know, all of that stuff matters, but you know, there’s bigger picture things you could be getting wrong that are really important to establish.

The two final things I wanted to go over here Jif I talked about a bit, or JIF it’s basically just how many citations does a publication get and publications like display this to show how good they are. Then you have the H index, which it was the thing I mentioned that is the fairly recent 2005 thing.

The H index, A researcher’s papers are ranked in descending order by the number of citations they each have received. The H index is the highest number [00:41:00] H, where the researcher has at least H papers that can each be cited at least H times, while the remaining papers have fewer H citations each. For example, if a scientist has an H index of 20, it means they have 20 papers cited at least 20 times each.

Wow. First cited paper has fewer than 21 citations. This metrics balances quantity, number of publications and quality, the impact via citations, avoiding extremes. Like somebody who had a b bone paper that was cited just a ton of ton at times. Or somebody who was, i’m putting out just tons and tons of papers.

And this is actually very easy to exploit through things like citation farming and stuff like that. And then somebody created a counter system actually just a year later called the G Index, but I don’t think it’s gotten as big as the H index. Hmm. That, that attempts to solve this with a bit of math that you don’t care about.

But it still runs into many of the same problems. Anyway, [00:42:00] Simone, final thoughts here.

Simone Collins: I’m glad that we’re returning to truth in its more natural format, and I find it notable how much this rhymes with journalism, but I think they’re both related. Journalism is just the truth about what’s happening in the world.

And this is just the truth about anything from medical. Science to mathematics, physics, et cetera.

Malcolm Collins: What I find very interesting about the way this has been happening is that the people who are in these sort of knowledge circles where real science and real academics is still happening.

Simone Collins: Mm-hmm.

Malcolm Collins: It’s not just that they all like know each other and are friendly, and it’s actually a really chill scene if you end up working your way into it.

But the ways that they’re employed is so diverse. A number of them, for example, still work in traditional academics. Take an individual like Robin Hanson who’s been on the show. Great, great thinker. And you know. He [00:43:00] is a traditional academic still, he still works at a professor at a university.

Yet he is sort of like brought into these fields as like, oh, obviously you belong in these fields. Because he is generating the type of work he does as a traditional academic actually does revolutionize multiple domains of knowledge through new ways of looking at. Things.

Simone Collins: Hmm.

Malcolm Collins: You know, like the grabby alien hypothesis for example, that’s not his job.

Aliens. That has nothing to do. But he comes up with this hypothesis that everyone who is interested, like super interested in aliens is immediately like. Oh, that’s a really good way to look at things. Like let’s dig into that. Right. And, and I think that it’s really heartening for me to see, and I think the people who are like, how do we save academics?

The reason they wanna save academics is because academics cares about them and the real people moving forward to human knowledge and innovation don’t care about them.

Simone Collins: [00:44:00] Yeah.

Malcolm Collins: Anyway, love you to decone.

Simone Collins: I love you too, Malcolm. I’m glad things are, I think getting better.

Malcolm Collins: Yeah. I mean, how do you, have you noticed this in terms of the way that you look at information, update yourself with new information about what’s true, what’s not true?

Simone Collins: Yes, absolutely. But you know, we’re in the same household, so Of course,

Malcolm Collins: Well, I mean, you used to like keep track of all the new studies that came out and I think you just stopped doing that.

Simone Collins: I didn’t stop doing that because I don’t trust them. I still think that peer reviewed academic research is broadly speaking better than anecdotal information that I have from life experience or from someone else’s sample of one or two.

You know it, it is more rigorously done than that you don’t end up in a peer reviewed publication because you. We’re super, super shoddy in your work. And as much as there is a lot of academic fraud, [00:45:00] there’s also a lot of people making stuff up online. So you have to choose the lesser of various evils.

Malcolm Collins: Yeah, but I, I, I’d point out that the difference in terms of information, like the quality of the information

Simone Collins: Yeah.

Malcolm Collins: Between let’s talk about the, the field of like arousal and sex, because that’s one of the ones I, I am, I’m very familiar with. Sure. Of your. Average peer-reviewed piece versus anecdotal evidence?

Oh, those two things are closer to each other than the average peer reviewed pieces to an ALA Substack post.

Simone Collins: No, that is 100% true, but also sex research has a long, especially in the academic field, a long history of just being abysmal. So I don’t think that’s a fair, I, I’m referring mostly to medical science and I think that.

Despite so many headwinds in that space when it comes to academic complications that generally I, I turn to it to try to find fairly reliable information because [00:46:00] the information of Renegade researchers, while very interesting is, has its own series of problems, not just small sample sizes and not just.

Maybe less rigorous methods and less in, in most cases, transparency, but also a lot of hidden agendas. And that is. You know?

Malcolm Collins: Yeah,

Simone Collins: I, I know the agendas in academia, at least, you just talked about a lot of them and that it at least is helpful when, when you trust something, it’s not because you know it’s going to give you good results.

It’s because you can predict it and better understand its incentive structures. You can’t something when you cannot trust something, when you cannot accurately and consistently predict it or understand its incentive structures.

Malcolm Collins: Mm-hmm. Alright. Have a day. Bye.

Simone Collins: It occurred to me that it’s gonna be really fun [00:47:00] when I’m sure we can probably figure out how to do it now, but our kids are still a little bit too young for it. But to have prediction markets just for household things.

You know, in, in how many months is, you know, so and so gonna reach No, because

Malcolm Collins: we have enough kids to get a real

Simone Collins: prediction market going. Yeah. And then also like, you know, what are the odds that mom emerge just from the bathroom after using the toilet unclogging gun covered in excrement? It will be very fun.

And it will also get our kids thinking more in a like Bayesian mindset instead of just yes and no. And I think, or I don’t think. And I want that. I want our kids to think in probabilities. I’m 90% sure whatever.

So

Malcolm Collins: well, an interesting concept I saw recently that I’m a little disappointed. We didn’t do an episode. Like I didn’t focus more on, [00:48:00] on the episode about how news is dying.

Simone Collins: Mm-hmm.

Malcolm Collins: And as gold was actually going over, this is when people are like, oh, that’s, that’s horrible PR or This is a PR

Simone Collins: nightmare,

Malcolm Collins: disaster.

Nightmare. They’re often talking about news media, and news media is largely irrelevant. Like they’re talking about a Karen hate mob. And then when

Simone Collins: no one cares about their opinion,

Malcolm Collins: published by sort of that class of people, and nobody cares. Nobody cares. Yeah. And that, that organization or mindset has not made.

Festival boycotts or anything like that in a long time. If anything, their, their boycotts more likely to just blow up. Like it’s, it’s, it is both not an audience we’ve seen with like the, the game about stealing African artifacts that just came out and had like zero players,

Simone Collins: Bruce.

Malcolm Collins: But it’s also this is, it is not an audience that you can court in any meaningful context, nor is it an audience that will abandon your products in any meaningful context.

And this [00:49:00] is coming up

Simone Collins: because I think it’s one of those things that Yeah. Is dead and people have not realized it’s dead. Similar to publishing or a variety of other domains. Yeah. Like advertising. You know, people, I think even to a certain degree, continue to advertise when. 95% of advertising doesn’t re, re reduce.

Sorry. Does not produce an ROI.

Malcolm Collins: Here’s a A. A, so people freak out so much when they get tied to us, or there’s some sort of negative mention in like a mainstream media outlet, like the New York Times or something. And in that episode we went over how few people actually watch those or read that these days.

Simone Collins: Yeah.

Malcolm Collins: When in reality it’s. A bigger impact to them if one of their friends Facebooks about something, because more of their friends are going to see that than if it’s in the New York Times. More people in their extended social network are gonna see that if somebody has an angry LinkedIn post about that.

Simone Collins: That’s such a good point. Yeah. Especially if it’s an influencer in their network. [00:50:00] Even if it’s literally the head of your homeowners association. But if that’s your world, that’s really all you should care about. Good point.

Malcolm Collins: Yeah, influencer in your network.

Speaker 8: here?

Speaker 5: I.

Speaker 8: Octavian, you gotta be careful when you attack them. You’re getting bigger, okay? You can’t jump on them.

Speaker 5: Okay?

Speaker 8: Octavian, did you understand me? Why? Because you could accidentally really hurt them.

Speaker 5: Okay? I hurt the subscribers.

Speaker 8: You’ll hurt the subscribers. Yeah, no, only hurt the non-subscribers.[00:51:00]

Is this where you’re training to battle the non-subscribers to people who don’t like and subscribe?

Speaker 5: Hey.

I,

Speaker 8: oh, it is.

Speaker 5: Look behind you because you picked your, [00:52:00]

Speaker 8: oh, okay. Okay. Okay. So I just gotta look behind me and you won’t attack me. You promise? Yeah,

Speaker 5: I promise.

Why?

Speaker 8: I’ll do it later. Okay. Why? Oh God. Stop. I gotta recharge my batteries.

Speaker 5: No, no, no.

Speaker 8: Gotta recharge my batteries. Stop [00:53:00] it.

This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit basedcamppodcast.substack.com/subscribe