14
$\begingroup$

I am having the feeling that since 'recently' in the vague sense of the word there has been an upsurge in questions on CV whereas the number of people who can give qualified answers (to sometimes very specialized question, esp. from the field of machine learning) has stagnated.

It is a very subjective impression that perhaps is related to recent interest and MOOCs on 'data science' and machine learning. I am wondering if anybody else had this experience.

$\endgroup$
13
  • 3
    $\begingroup$ This can help answering the title of your question: sese.evbpc.com/Sites/…. Perhaps you could use it to narrow the question. $\endgroup$
    – Andre Silva
    Commented Sep 6, 2016 at 20:34
  • $\begingroup$ Me like it. I may be right $\endgroup$
    – tomka
    Commented Sep 6, 2016 at 20:42
  • 3
    $\begingroup$ I am very new, so do not have a good sense of the trend. However I have noticed a common tendency for questions to be "answered" in their comments. Quite frequently these answers seem sufficient, but they are never codified into an official "Answer". $\endgroup$
    – GeoMatt22
    Commented Sep 6, 2016 at 21:14
  • 1
    $\begingroup$ This has been discussed several times, notably here: meta.stats.stackexchange.com/questions/2242 (this was almost two years ago though). $\endgroup$
    – amoeba
    Commented Sep 6, 2016 at 21:50
  • 3
    $\begingroup$ @Geomatt About the tendency to answer in comments, where often a reasonably adequate answer is given: I agree -- it's an ongoing problem we have. $\endgroup$
    – Glen_b Mod
    Commented Sep 7, 2016 at 10:46
  • 2
    $\begingroup$ @Glen_b do we have a protocol for basing an answer on someone else's comments? I would not mind doing it occasionally but it feels dangerously close to plagiarism. $\endgroup$
    – mdewey
    Commented Sep 7, 2016 at 11:47
  • 5
    $\begingroup$ @mdewey It's an issue that has been discussed; I'll see if I can find the question. [My personal opinion was if someone posts an answer in a comment it's fair game (if they didn't want someone else to write an answer based on it, they should have done so themselves). The main requirement would be to credit the author of the comment; quoting them if you're quoting them or otherwise crediting them if you paraphrase] $\endgroup$
    – Glen_b Mod
    Commented Sep 7, 2016 at 11:52
  • 2
    $\begingroup$ @Glen_b: meta.stats.stackexchange.com/a/2642/17230 ? $\endgroup$
    – Scortchi Mod
    Commented Sep 7, 2016 at 12:02
  • 5
    $\begingroup$ @mdewey It's not plagiarism to quote and acknowledge somebody else's work. Neither is it plagiarism to explain somebody else's idea in your own words (although acknowledgment of the source of the idea is expected). This site, and SE in general, encourage people to convert comments into actual answers. $\endgroup$
    – whuber Mod
    Commented Sep 7, 2016 at 13:26
  • 1
    $\begingroup$ Perhaps relevant: I noticed this new question today, which is essentially the same as an earlier question that is unanswered. An authoritative answer to one of these would allow future instances to be closed as duplicates (or reduce future queries, if found). The new question is from a user who just joined, but the earlier one is from a longer-time user (7 months), so an authoritative answer to the earlier one might be accepted. $\endgroup$
    – GeoMatt22
    Commented Sep 7, 2016 at 14:05
  • 1
    $\begingroup$ @Scortchi That's the question I meant to point to, thanks (rather than just my own answer) $\endgroup$
    – Glen_b Mod
    Commented Sep 7, 2016 at 15:14
  • 1
    $\begingroup$ It's also worth noting that some Coursera videos recommend sending questions to CV. Trying to hold my opinions to a minimum, but I would guess that this leads to surge in questions that don't get answered. $\endgroup$
    – Cliff AB
    Commented Sep 15, 2016 at 18:08
  • $\begingroup$ From my experience, answering old unanswered questions usually yields little reputation compared with answering recent questions. Therefore, there is little incentive to answer old questions and unanswered questions are likely to remain unanswered forever. It would be good to incentive such answers, although I don't know how - I know we have a couple of badges about it, but it doesn't seem enough. $\endgroup$
    – Pere
    Commented Sep 19, 2016 at 15:50

2 Answers 2

Reset to default
14
$\begingroup$

I've been watching this for a long time now. While both questions and answers are growing, there's a somewhat faster increase in questions than answers, and over a period of a couple of years it can be quite alarming (and then there are occasional apparent rapid increases - "jumps" - in the level of questions that I have no good explanation for).

While the problem that the answer rate isn't keeping up is likely to have many sources (among other explanations see Why is our answer rate so low?), one source than we can do something about is that we are overly tolerant of poor quality and near duplicate questions. The biggest growth seems to be in that segment of poor and duplicate questions.

Duplicates can sometimes be hard to locate, even if you are confident you've seen one (it's usually worth a try, though because it makes the site much more useful when we do find them), but unclear/off-topic/ etc questions can often be spotted quickly and it takes very little effort to vote (or flag if you lack the voting privilege) to put them on hold.

I agree that there's a rapid growth of interest in data science and machine learning topics and that's certainly a part of the problem.

We have had a few new people answering a lot of questions recently -- that's great to see, but there's lots more questions that we can deal with.

I'd like to attract more people to answer, particularly in the areas of rapid growth though I am not quite sure how to do that. Bounties do seem to help a little to encourage budding answerers - and we've had a lot more bounties lately - but it's not going to be enough on its own.

All the curation activity that goes on -- people who to try to make the tags better, the people who quietly edit questions into good approximations of clarity and comprehensibility, the people who flag and vote to close and so on -- that all helps tremendously, and there are some people who contribute a great deal on that front. While that doesn't directly answer more questions, it keeps the site viable and useful and among other things makes it more attractive to answer questions, encouraging answerers to move from responding to one question to tackle more.

$\endgroup$
7
  • 6
    $\begingroup$ +1. It might be worth noting that there is a strong seasonal component to the rate of questions (and not much seasonality in the rate of answers). The "seasons" are, as one might guess, academic ones: a gradual increase throughout the fall; a sudden drop in late December; a gradual increase in the spring; and a month-long drop in May. Recently, then, people would be observing the beginning of the first semester of the 2016-17 academic year. It is also true that about half the growth in the gap between Q's and A's is due to Q's that get closed. $\endgroup$
    – whuber Mod
    Commented Sep 7, 2016 at 13:23
  • $\begingroup$ I have been wondering whether it would make any sense to have a separate SE board dedicated to machine learning and try to exclude the topic from CV. There are good reasons not to do this of course. Indeed ML does statistics, but it usually assumes very, very large samples (or it assumes you have seen the whole population). So this is the major difference to classical statistics. Also there are other SE boards that share a communality with CV, like the stackoverflow or mathematics boards and a board dedicated to ML would outsource computer science topics beyond the scope of this board. $\endgroup$
    – tomka
    Commented Sep 7, 2016 at 16:12
  • 6
    $\begingroup$ @tomka I think ML and statistics share a a great deal more than you seem to (ML tends to be used on larger samples, but that doesn't make the issues of classification, prediction etc non-statistical) -- and splitting would just move the large group of ML-learners questions away from a pool of experts able to answer many of them, making it someone else's problem; it wouldn't produce more people capable of answering those questions. [I actually think ML is better here, where the various kinds of statisticians / ML people can bring their overlapping expertise to these problems - it's fruitful.] $\endgroup$
    – Glen_b Mod
    Commented Sep 7, 2016 at 22:34
  • 2
    $\begingroup$ @tomka: There's already Computer Science, as well as Data Science & Computational Science in beta. And there used to be a machine learning SE site too, the questions from which were all migrated over here when it shut down. See ML questions: here or at Data Science? & links. $\endgroup$
    – Scortchi Mod
    Commented Sep 9, 2016 at 9:15
  • 2
    $\begingroup$ I'm also tempted to suggest we get a higher proportion of poor ML questions than of poor trad stats questions: vague appeals for help - "My classifier is bad" -, reams of uncommented, inscrutable code that "doesn't work", & that sort of thing. $\endgroup$
    – Scortchi Mod
    Commented Sep 9, 2016 at 10:50
  • 2
    $\begingroup$ @Scortch I think that's so, and again due to the increase in people learning ML - especially via MOOCs and such $\endgroup$
    – Glen_b Mod
    Commented Sep 9, 2016 at 10:55
  • 2
    $\begingroup$ @Glen_b I recently saw a comment on Reddit that went something like "data science is what engineers call statistics when they don't know enough statistics to realize they're doing statistics". I think you could swap out "machine learning" for "data science" in this case. $\endgroup$
    – shadowtalker
    Commented Sep 13, 2016 at 5:04
11
$\begingroup$

In answer to Glen_b, I think the site management is outstanding. It seems however that the group of users who have questions but cannot answer any is much larger than the group of people who can answer questions (and may have some own ones once in a while). Related, statistics as a field is rather wide, the number of potential applications is huge, and the scope of both is growing rapidly, especially with adjacent fields like ML being integrated. So it may be harder for CV to maintain a critical number of experts with wide enough joint knowledge (and time) to provide a sufficient amount of answers.

Two further factors may also play for some people and I have seen them being discussed here on meta CV before. First I believe CV has a rather disastrous upvoting problem. I have seen questions with five answers while the question did not have any upvote, nor the answers. In my humble opinion, if you answer a question it probably was good enough to deserve an upvote. If somebody takes the time to answer your question, you should upvote it (nearly) regardless of the quality (nearly = except if wrong, off topic etc., see comments below). Whether an answer is great is decided by the community by giving even more upvotes.

Second, I have caught myself not giving an answer to newbies because in 90% of the cases you never hear from them again (again no upvote but also no accepted answer). I suspect the upsurge in questions is related to an instream of new users. So potentially more experienced users may be discouraged from answering these questions by earlier experiences of this kind.

$\endgroup$
2
  • 7
    $\begingroup$ Except for the part an OP should upvote an answer to his/her question regardless of quality, agree with the other points made. Because sometimes the answer does not answer the question; it is wrong; is a link-only-answer, etc. And upvoting these will send a message that it is ok to respond like that, when it is not. Perhaps when you used the word "nearly" you wanted to disregard such situations? $\endgroup$
    – Andre Silva
    Commented Sep 7, 2016 at 11:09
  • 2
    $\begingroup$ Exactly. Thanks for making this more explicit! $\endgroup$
    – tomka
    Commented Sep 7, 2016 at 11:11

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .