# How does Stack Overflow actually manage to work given ~100x higher traffic than here, and what can we learn from it?

This question is inspired by @Glen_b's worry yesterday that we [are] seeing a dramatic drop in answers per question and is also related to my own question from almost a year ago about How not to miss interesting answers?

I have the feeling that many (most?) active users on Cross Validated are more or less monitoring the whole forum. They look at the front page several times per day, which makes them see pretty much all the appearing and updated questions. They definitely can monitor all the bounty questions that rarely exceed 10 at any given time. Questions and answers mostly get views and upvotes while on the front page (and some further views/upvotes if they get into the weekly/monthly top, but enough initial views/upvotes is necessary for that).

So overall I get the feeling of a reasonably small and attentive community, like a big permanent seminar in a busy room where anybody could come in and ask a question (sort of a coffee lounge in a stat department, like it should be).

My question (linked above) from a year ago was motivated by the fact that it seemed to me that this model almost breaks down now, because the number of questions gradually becomes too high. As @Glen_b now points out, this trend is likely to increase in the near future. Which makes me wonder: how does our bigger (much bigger!) brother Stack Overflow manages to function at all, given that they are much, much beyond our activity level?

For comparison. We have ~50k questions, they have ~8000k. We have ~10 bounty questions at any given time, they have ~500. Our front page covers the last ~5 hours, their front page barely covers 1h.

Maybe some people here frequent both forums and can comment on how this HUGE activity influences how the forum functions and in particular how active contributors deal with it? How can each individual question or answer possibly get enough attention, views and upvotes? Are people only following specific tags, and so the forum is basically split in some barely overlapping ones? But there isn't even a way to see all the questions with one's "favourite" tags on one page! Do people rely on email notifications? Etc.

I think the biggest factor that allows SO to function is that there are a lot more people contributing answers and working to maintain the site. That is, the number of people asking is larger, but the site as a whole maintains some sort of balance between the number of askers and the number of active answerers. It seems that we are slipping out of balance, which I take to be @Glen_b's concern (although it may be just a seasonal effect, as @whuber noted).

Other than that, my impression is that SO has evolved to become a loose federation of tag-based fiefdoms. I occasionally look at the [r] tag page on SO, @NickCox and @StasK only follow the [stata] tag page, etc. I'm sure there is some overlap for some of the tags (e.g., [html], [css], [php], and [javascript] logically go together), but you still have semi-isolated islands in an archipelago. I certainly never follow the main page on SO and I suspect (almost?) no one does.

Here, on the other hand, I do follow the main page, and I'm glad I do. This has exposed me to a lot of statistical (ML, etc.) topics that I'm still not expert in but would otherwise be totally unfamiliar with. This has been quite valuable to me and I'm glad it remains manageable; I rue the day when it ceases to be.

• +1. Strong support here. Also I'd add that although I understand why many people want to ask software-specific questions here, here's a practical fact: if they were allowed, the volume of questions would go up dramatically and many of us would have a harder job wading through questions on software we don't use. – Nick Cox Dec 4 '14 at 18:35
• Huge +1 to this, @gung. You are confirming exactly what I suspected: that SO has in a certain sense turned into a number of disjoint forums, corresponding to different tags (even though how can anybody monitor tags such as [c#] with 700k questions, I am still not sure). I have exactly the same feelings as you about CV and I join your rue (special thanks for using this word with which I was unfamiliar and had to look it up; one learns more than only statistics on CV). A more practical question that I suggest we consider here, is can (and should) we do anything to prevent it?.. – amoeba Dec 4 '14 at 21:12
• gung I agree, and currently believe that the expertise required to successfully vet and answer questions on SO is much more broadly distributed within society than on CV: many more people code than statistic. – Alexis Dec 7 '14 at 18:29

In my case, very simple. I follow the tag "Stata" exclusively. Occasionally that leads me elsewhere, but I don't systematically follow anything else.

Incidentally, that's one reason I am quite aggressive on migrations to Stack Overflow. If it's a Stata question, I will certainly look at it on SO. (If it's an R question, there are many spectacularly competent people on SO; and so on for most of the leading statistical platforms. I find I don't care much about supporting statistics in environments that don't allow programming; or else life is short, and some things have to be somebody else's problem.)

Stata users count as a tiny niche community within SO. That's why it's manageable to monitor. Posts are of the order of 1 per day. But no moderators on SO know anything about it, so far as I can tell. Occasionally, this causes bizarre problems, as when perfectly good answers are deleted by moderators under pressure from high-reputation users because they don't recognise key words, and the answer looks like spam or irrelevance.

• I am pretty much in the same niche as Nick, as far as SO goes. There's a separate Statalist.org forum for Statafolk with may be 50 questions and answers per day; at some point, I suggested relocating it to SE platform, as I am missing the SE functionality there very badly (tags, ability to edit other people's posts, etc.), but it was never picked up. Given that there's such a strong and highly qualified Stata community elsewhere, I am surprised any Stata questions are getting asked on CV or SO (although they end up being answered by one of about five or ten people no matter where they land). – StasK Dec 4 '14 at 15:42
• @StasK I love being able to edit your posts. A detail that may bite with some is that Statalist has an expressed strong preference for posters using their real full names. Here and indeed on talkstats identifiers are at choice and that may be crucial to those who wish to ask what may appear to be ignorant or stupid questions in public without loss of face or reputation or prospects. Economists seem especially prone to a cost-benefit analysis approach to such matters, and indeed life in general. – Nick Cox Dec 4 '14 at 15:49

To a greater extent than here, SO seems to divide up into mostly independent topic areas, though individual questions might cross several such topics that usually don't sit on the same question.

In my case, I read the r tag there, and rarely look at anything else, so stackoverflow looks to someone like me like a largish but quite manageable site with questions on R. Sometimes I read Python questions (but can't answer any - it's a way of trying to learn more). On rare occasions I venture into looking at other tags.

I imagine most people would look at a wider range of tags than I do, but would still keep themselves to only a few.

Here on CV, I don't limit myself by tags. There are certainly some tags that I'm not qualified to say much about, and topics I'm not actively trying to answer, but I try to make sure they all show up when I look at the site. That's partly because stats questions are much less compartmentalized than typical SO questions (and partly because both my interests and background are substantially broader). It's the same when I go on math.SE (though the topics I'm qualified to answer on there are more restricted). Since math.SE has quite a bit more traffic than here and it's not quite to the overwhelming stage there, I expect I'd be able to see a substantial growth in traffic here without changing that habit very often.

However, if the volume were multiplied by ten, I imagine I'd pretty much have to cut that "read everything" down a lot of the time, either by searching on favorite tags or by using the ability to ignore some.

There are many colleagues in my workplace who uses Stack Overflow in the read-only mode. I try to encourage them to participate, but they always say that most things they look for are already there.

As time goes on, it is more difficult to ask different good questions (despite new subjects are always coming up). As a consequence, there are many closed and downvoted questions on that site, and the community handles them until a certain point.

A recent change in the SE system allowed users who carry a tag gold badge to cast binding votes on duplicate threads from that tag. I felt that questions labeled with Excel were more organized and easier to search, as simple questions were right away marked as duplicates to a canonical answer (e.g., vlookup questions). That somehow will make top answers more accessible to people coming from Google and other search engines.

This is a feature we could give more attention: linking recurrent questions as duplicates of canonical answers.

Update: SO is rolling out a promising feature to increase the quality level of its questions. First part of it, is the Triage process.

• Interesting point about golden tag badges, but I have to point out that they are EXTREMELY difficult to get on Cross Validated (because 1000 upvotes is an enormous lot here, unlike Stack Overflow). In fact, only two users currently have golden tag badges: Glen_b (in [regression] and [r]) and gung (in [regression]). – amoeba Dec 4 '14 at 20:23
• @amoeba, correct. I was trying to emphasize we could make more effort to find and link the dupes (in CV it is easier to find them comparing to SO). – Andre Silva Dec 4 '14 at 20:37
• And regarding the duplicates: if I am not mistaken, questions closed as duplicates are never automatically deleted (unlike other closed questions) and remain hanging around. At the same time, the "main" question displays all of them in the "Linked" list on the right. I am very fond of this list and always check it, but if gets filled with worthless duplicates, I imagine it loses any value. Does it happen on CV? Maybe the whole Stack Exchange network should better separate "Linked" into "Linked" and "Duplicates"... – amoeba Dec 4 '14 at 21:37
• @amoeba I have had exactly the same thoughts about "linked questions". I wonder whether CV takes a different attitude to "duplication" than SO does, since our questions tend to be more conceptual and not just about implementation details. It's difficult to define a metric, but I feel as if many of our duplicates tend to be "less similar" than those on SO. – Silverfish Dec 4 '14 at 22:30
• Duplicates do not get deleted because they serve a purpose, which is to lead users to a good answer. It is common users with different background asking the same thing in different ways, so they are kept as pathways for future readers. About the linked questions, it is a SE network-wide feature. Suggestion: in cases of duplicates, one can go first to the main post and look to the linked question from there. (cc/ @amoeba). – Andre Silva Dec 4 '14 at 23:03
• @Andre, but that is exactly what I meant: "linked" questions list in the main post will be filled up with duplicates (that are useless there)... On the other hand, duplicates usually have not many upvotes, so they should be in the bottom of the "linked" list. – amoeba Dec 4 '14 at 23:23
• @amoeba, sorry. That is correct. I overlooked it. It should be like you say (no links of duplicates in the main post, under the 'Linked' section; or at least they could be in the bottom). Tks. – Andre Silva Dec 5 '14 at 12:42