The Nonlinear Library LW - the subreddit size threshold by bhauth
Jan 24, 2024
06:31
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: the subreddit size threshold, published by bhauth on January 24, 2024 on LessWrong.
Nobody goes there anymore. It's too crowded.
Yogi Berra
In the early days of the internet, people on Usenet complained about the influx of new users from AOL making it worse. I always thought the evolution of online communities with growth was an interesting and important topic. Do they really get worse with size? According to who? Why would that happen? What can be done about it?
Today, Reddit has over 1 billion monthly active users. It's divided into smaller communities called subreddits, all using the same software. This provides an unprecedented amount of data on the dynamics of online communities.
I haven't done a systematic study of every subreddit, but sometimes I read things on Reddit myself. I mainly do that by using a browser shortcut to see the weekly top posts of a particular subreddit, using the old site version. In doing that, I've gotten a decent idea of how particular subreddits differ, and I've noticed that very large subreddits tend to have lower quality than smaller ones. I'm not the only one; this has been widely noted.
Naively, one might expect that the week's best posts from a larger group of people would be better, and that does seem to be the case up to a point - and then the trend reverses. At 100k users, the derivative of quality vs size is clearly negative. That raises the obvious question: why? Why would large subreddits be worse? Here are the possible reasons I've thought of.
reasons for decline
selection bias
Maybe I'm selecting high-quality subreddits to read, and there are more small subreddits, so some of them will randomly be better.
I certainly do select what subreddits I look at, but I don't think that's the reason here, because:
I've seen changes in quality over time as subreddits grow.
The variation seems mostly consistent across different ways of selecting subreddits to read.
memes
A common thing that relatively high-quality larger subreddits do is remove meme posts, which are mostly popular images with a few words added on them.
I think the problem with those meme posts is that time spent on posts varies but every upvote is worth the same. Most people who see posts don't even vote on them, and there's some fraction of people who will see a meme, look at it for 2 seconds, upvote, and move on. That upvote is worth the same as an upvote from someone who spent 10 minutes reading an insightful essay.
A similar problem happens with titles that confirm people's preconceptions. For example, if someone really hates Trump, and sees a title that implies "this shows Trump is bad", they might upvote without actually looking at the linked post.
There have been a few attempts at mitigating this by making vote strength variable. Some sites have "claps" instead of "likes", which can be clicked multiple times. There are sites like LessWrong where users can make stronger votes by pressing the vote for a couple seconds. The problem I have with such systems is, while individual votes more accurately represent the voter's opinion, the result is a worse average of overall user views. For example, there might be a thread of 2 people arguing, and then 1 person strong-downvotes every post of the other person to make their argument look relatively better, and then the other person gets mad and does the same, and then those strong votes can outweigh votes from other people.
new post visibility
When you make a new post on a smaller subreddit, it goes directly to the front page, where ordinary users see it and vote on it. On a larger subreddit, new posts are only visible on a special "new" page, which only a small fraction of users visit.
One uncommon thing TikTok did was showing new videos from creators with few followers to a hundred or so people. Videos that got some like...
Nobody goes there anymore. It's too crowded.
Yogi Berra
In the early days of the internet, people on Usenet complained about the influx of new users from AOL making it worse. I always thought the evolution of online communities with growth was an interesting and important topic. Do they really get worse with size? According to who? Why would that happen? What can be done about it?
Today, Reddit has over 1 billion monthly active users. It's divided into smaller communities called subreddits, all using the same software. This provides an unprecedented amount of data on the dynamics of online communities.
I haven't done a systematic study of every subreddit, but sometimes I read things on Reddit myself. I mainly do that by using a browser shortcut to see the weekly top posts of a particular subreddit, using the old site version. In doing that, I've gotten a decent idea of how particular subreddits differ, and I've noticed that very large subreddits tend to have lower quality than smaller ones. I'm not the only one; this has been widely noted.
Naively, one might expect that the week's best posts from a larger group of people would be better, and that does seem to be the case up to a point - and then the trend reverses. At 100k users, the derivative of quality vs size is clearly negative. That raises the obvious question: why? Why would large subreddits be worse? Here are the possible reasons I've thought of.
reasons for decline
selection bias
Maybe I'm selecting high-quality subreddits to read, and there are more small subreddits, so some of them will randomly be better.
I certainly do select what subreddits I look at, but I don't think that's the reason here, because:
I've seen changes in quality over time as subreddits grow.
The variation seems mostly consistent across different ways of selecting subreddits to read.
memes
A common thing that relatively high-quality larger subreddits do is remove meme posts, which are mostly popular images with a few words added on them.
I think the problem with those meme posts is that time spent on posts varies but every upvote is worth the same. Most people who see posts don't even vote on them, and there's some fraction of people who will see a meme, look at it for 2 seconds, upvote, and move on. That upvote is worth the same as an upvote from someone who spent 10 minutes reading an insightful essay.
A similar problem happens with titles that confirm people's preconceptions. For example, if someone really hates Trump, and sees a title that implies "this shows Trump is bad", they might upvote without actually looking at the linked post.
There have been a few attempts at mitigating this by making vote strength variable. Some sites have "claps" instead of "likes", which can be clicked multiple times. There are sites like LessWrong where users can make stronger votes by pressing the vote for a couple seconds. The problem I have with such systems is, while individual votes more accurately represent the voter's opinion, the result is a worse average of overall user views. For example, there might be a thread of 2 people arguing, and then 1 person strong-downvotes every post of the other person to make their argument look relatively better, and then the other person gets mad and does the same, and then those strong votes can outweigh votes from other people.
new post visibility
When you make a new post on a smaller subreddit, it goes directly to the front page, where ordinary users see it and vote on it. On a larger subreddit, new posts are only visible on a special "new" page, which only a small fraction of users visit.
One uncommon thing TikTok did was showing new videos from creators with few followers to a hundred or so people. Videos that got some like...