How does Stack Overflow actually manage to work given ~100x higher traffic than here, and what can we learn from it?

Question

This question is inspired by @Glen_b's worry yesterday that we [are] seeing a dramatic drop in answers per question and is also related to my own question from almost a year ago about How not to miss interesting answers?

I have the feeling that many (most?) active users on Cross Validated are more or less monitoring the whole forum. They look at the front page several times per day, which makes them see pretty much all the appearing and updated questions. They definitely can monitor all the bounty questions that rarely exceed 10 at any given time. Questions and answers mostly get views and upvotes while on the front page (and some further views/upvotes if they get into the weekly/monthly top, but enough initial views/upvotes is necessary for that).

So overall I get the feeling of a reasonably small and attentive community, like a big permanent seminar in a busy room where anybody could come in and ask a question (sort of a coffee lounge in a stat department, like it should be).

My question (linked above) from a year ago was motivated by the fact that it seemed to me that this model almost breaks down now, because the number of questions gradually becomes too high. As @Glen_b now points out, this trend is likely to increase in the near future. Which makes me wonder: how does our bigger (much bigger!) brother Stack Overflow manages to function at all, given that they are much, much beyond our activity level?

For comparison. We have ~50k questions, they have ~8000k. We have ~10 bounty questions at any given time, they have ~500. Our front page covers the last ~5 hours, their front page barely covers 1h.

Maybe some people here frequent both forums and can comment on how this HUGE activity influences how the forum functions and in particular how active contributors deal with it? How can each individual question or answer possibly get enough attention, views and upvotes? Are people only following specific tags, and so the forum is basically split in some barely overlapping ones? But there isn't even a way to see all the questions with one's "favourite" tags on one page! Do people rely on email notifications? Etc.

Answer 1

I think the biggest factor that allows SO to function is that there are a lot more people contributing answers and working to maintain the site. That is, the number of people asking is larger, but the site as a whole maintains some sort of balance between the number of askers and the number of active answerers. It seems that we are slipping out of balance, which I take to be @Glen_b's concern (although it may be just a seasonal effect, as @whuber noted).

Other than that, my impression is that SO has evolved to become a loose federation of tag-based fiefdoms. I occasionally look at the [r] tag page on SO, @NickCox and @StasK only follow the [stata] tag page, etc. I'm sure there is some overlap for some of the tags (e.g., [html], [css], [php], and [javascript] logically go together), but you still have semi-isolated islands in an archipelago. I certainly never follow the main page on SO and I suspect (almost?) no one does.

Here, on the other hand, I do follow the main page, and I'm glad I do. This has exposed me to a lot of statistical (ML, etc.) topics that I'm still not expert in but would otherwise be totally unfamiliar with. This has been quite valuable to me and I'm glad it remains manageable; I rue the day when it ceases to be.

Answer 2

In my case, very simple. I follow the tag "Stata" exclusively. Occasionally that leads me elsewhere, but I don't systematically follow anything else.

Incidentally, that's one reason I am quite aggressive on migrations to Stack Overflow. If it's a Stata question, I will certainly look at it on SO. (If it's an R question, there are many spectacularly competent people on SO; and so on for most of the leading statistical platforms. I find I don't care much about supporting statistics in environments that don't allow programming; or else life is short, and some things have to be somebody else's problem.)

Stata users count as a tiny niche community within SO. That's why it's manageable to monitor. Posts are of the order of 1 per day. But no moderators on SO know anything about it, so far as I can tell. Occasionally, this causes bizarre problems, as when perfectly good answers are deleted by moderators under pressure from high-reputation users because they don't recognise key words, and the answer looks like spam or irrelevance.

Answer 3

To a greater extent than here, SO seems to divide up into mostly independent topic areas, though individual questions might cross several such topics that usually don't sit on the same question.

In my case, I read the r tag there, and rarely look at anything else, so stackoverflow looks to someone like me like a largish but quite manageable site with questions on R. Sometimes I read Python questions (but can't answer any - it's a way of trying to learn more). On rare occasions I venture into looking at other tags.

I imagine most people would look at a wider range of tags than I do, but would still keep themselves to only a few.

Here on CV, I don't limit myself by tags. There are certainly some tags that I'm not qualified to say much about, and topics I'm not actively trying to answer, but I try to make sure they all show up when I look at the site. That's partly because stats questions are much less compartmentalized than typical SO questions (and partly because both my interests and background are substantially broader). It's the same when I go on math.SE (though the topics I'm qualified to answer on there are more restricted). Since math.SE has quite a bit more traffic than here and it's not quite to the overwhelming stage there, I expect I'd be able to see a substantial growth in traffic here without changing that habit very often.

However, if the volume were multiplied by ten, I imagine I'd pretty much have to cut that "read everything" down a lot of the time, either by searching on favorite tags or by using the ability to ignore some.

Answer 4

There are many colleagues in my workplace who uses Stack Overflow in the read-only mode. I try to encourage them to participate, but they always say that most things they look for are already there.

As time goes on, it is more difficult to ask different good questions (despite new subjects are always coming up). As a consequence, there are many closed and downvoted questions on that site, and the community handles them until a certain point.

A recent change in the SE system allowed users who carry a tag gold badge to cast binding votes on duplicate threads from that tag. I felt that questions labeled with Excel were more organized and easier to search, as simple questions were right away marked as duplicates to a canonical answer (e.g., vlookup questions). That somehow will make top answers more accessible to people coming from Google and other search engines.

This is a feature we could give more attention: linking recurrent questions as duplicates of canonical answers.

---

Update: SO is rolling out a promising feature to increase the quality level of its questions. First part of it, is the Triage process.