Are we closing questions too fast?

Question

It seems to me that we have been closing more and more questions, and that we are moving them into the close votes review queue very quickly. This seems to have evolved naturally, but I wonder if we have moved past the optimum point without anyone stopping to think about whether we are moving in the right direction. I hope this thread will give us a chance to do so now.

Whereas I used to think that most treads coming through the close vote queue should be closed, I am increasingly voting to leave open or just not voting at all on threads there (perhaps leaving a comment to the OP instead). I am somewhat reluctant to list specific threads here, because people have voted on them and I don't want anyone to feel defensive. However, I can make some general comments and people may be able to recognize cases that fall into that pattern:

1. duplicate: (Actually, I have no problem with how this is working.)

2. off topic: I see two issues in this category:

   • self-study questions: My impression is that any question that seems like it might be an assignment is immediately voted to be closed as self-study. I do occasionally vote to close such questions immediately, but mostly I just leave a comment for the OP to add the tag and read the wiki. (There is suggested text that people can use here: How best to use the review queue?)

     Many self-study questions come with 'I have this question: ... I thought it might be ... but I can't figure it out'. Such a case is 90% of the way to meeting our requirements, in my opinion. True, it doesn't have the tag, but new users are very unlikely to know our policies and may well be trying to get help in a completely honorable way. Even if they have less than that, it may still be viable to leave a comment and let them (potentially) edit.

     Having been notified to add the tag, if they don't add it after a reasonable period of time (maybe a few hours), it seems appropriate to vote to close, but not before and not without any comment (or so it seems to me).

   • ... focuses on programming...: It seems to me that any question that even mentions R (e.g.) is often summarily voted to close. I do continue to believe that we should focus on questions about statistics, machine learning, etc., and not about how to use software or tech support, despite the fact that you have to use software to do statistics, machine learning, etc. Nevertheless, I often suspect the situation is more ambiguous than just whether code is listed with the question.

     For example, a post may have a mix of statistical and coding questions, in which case, I believe the post could stay open to answer the statistical questions, and we could leave a comment explaining that the coding questions may not get an answer. I have seen other cases where there was a coding error, but I suspect the reason the OP made a coding error is not because they misunderstood the code, but because they don't properly understand the underlying statistical concepts. In that case, I believe the Q should stay open so we can address the 'real' issue. Likewise, there are other cases where people are explicitly asking about code, but presumably wouldn't if they knew the statistical issues better. I often leave the "Ambiguous code / statistical question" comment from the meta.CV thread linked above in situations like that. Etc.

3. unclear what you're asking: A lot of threads have been voted to close as unclear without a comment explaining what is unclear and what the OP could do about it to fix their question. I recognize that there are some questions that are so thoroughly unclear you can't even state what it needs. I would estimate those are about 10% of the unclear questions, though.

   Depending on how bad it is, I sometimes leave a comment, and sometimes leave a comment and vote to close. (I rarely vote to close as unclear without a comment.) In some proportion of those cases, the OP comments in reply or edits their question to clarify the points at issue. There have been such cases where I have subsequently retracted my close vote. I think more can be done here...

4. too broad: (I probably overuse this myself.) I suspect there are relatively few threads that really are too broad, although some clearly do exist. (I remember one question that read, more or less, 'how do you fit models, and how do you select one, and how do you interpret it?', that's too broad.) There have been a sizeable proportion (albeit probably less than half) of questions voted as too broad, that I think could have been given a brief, but broad, answer that may well have be useful for the OP and for future users. The OP may then be able to ask a clearer, more specific question, or may find that even a one paragraph gloss is enough to get them started.

   I also wonder if such questions may sometimes / eventually pull out thoughtful answers that are real gems. I wonder if a question like, what basic tests can I run on each dataset to make sure that they don't have typos, might be closed as too broad today. I myself have thought about asking a question or two before, but opted not to, because I couldn't formulate the question any better than, 'how should I think about ...' or 'what do people think about ...', where a broad, preliminary answer, or some discussion in comments might have put me on the right path forward. As with unclear, I think a comment to the OP may often be a better first action than voting to close straightaway.

5. primarily opinion based: These are more commonly fine, in my opinion. I am aware that SE policy is that all such questions be closed, so I admit I'm on shaky ground. Nonetheless, I believe that opinion-based questions can sometimes be of real value to people and could be allowed here, if just made CW. (Unfortunately, I do not have any clear guidance on how to draw the line, though.)

Above, I have contrasted things I've seen with what I do. I don't mean to hold myself up as the ideal that others need to match. My own behavior is simply my most natural point of reference. My goal here is to raise the topic for discussion. I am open to changing my own policies as well.

---

Edit:
My title may be a little misleading. The suggestion implied by the question isn't that a thread should spend a long time in the close votes review queue, or something like that. If a thread is to be closed, I agree we benefit from it closing as quickly as possible. Perhaps a better way to put it might have been 'Are we moving too many questions into the close votes review queue?' There is some—difficult to articulate—line between voting to close and not doing so. I wonder if that line has become too stringent. Relatedly, I wonder if a comment might not be enough in some cases, and should often be added if we are going to vote to close.

For example, @Tim notes that he always does both, which strikes me as a good policy. Below that, @GeneralAbrial makes a good point:

Part of the problem I have with the re-open interface is that the comments discussion is not visible -- so on a post on some obscure topic, precisely 0% of the question is clear to me, but if the comments say "please clarify these three issues," I can usually figure out whether those points have been clarified. Absent those hints, I usually just Skip.

It can be difficult for an OP to know what / how to edit their question, and difficult for someone to subsequently determine if the problem has been rectified, if what is unclear was left unspecified.

We have moved from a state of affairs where I used to think almost all of the threads in the close votes review queue should be closed (and those I didn't think should be closed were nonetheless understandable), to a situation where a large proportion (maybe close to half) don't necessarily need to be closed, in my opinion. Yesterday, for example, a homework question (Why is a square root not a linear transformation?) was asked. It labeled the question as an assignment, and included what the OP understood thus far. So instead of voting to close without comment, I left a comment to add the self-study tag and read its wiki. The OP added the tag in approximately 30 seconds. Despite this, someone subsequently voted to close the thread as off-topic / homework. (I had said I would avoid listing any specific examples so as not to make anyone feel attacked, but perhaps this will help make the topic clearer. I apologize for any hurt feelings.)

Answer 1

Some excellent and thoughtful comments in existing answers from long-standing members still leave a little scope for additional remarks, from the Dark Side if you will.

Some respected colleagues here seem to be leaning towards an ideal of indefinite courtesy, namely that people posting poor questions may require and deserve support until their questions have been rewritten as acceptable and that it's our duty to provide that so far as we can.

I don't vote for being rude (who does?), but I vote for being firm and for spending time and effort in ways that promote the best long-term interests of the site, which pivot on building up a core of well-posed, well-answered questions.

I don't buy the obligation to indefinite courtesy as a matter of practicality. We have guidelines on the kinds of questions we support, which over time have shifted and which in the future may be changed again. But at any one time many questions are just off-topic and/or poor. Sometimes people just have not read the advice in the Help Center. Sometimes people are just trying to outsource their homework -- or even more frequently their basic thinking on Master's or research projects -- to the internet. Sometimes people are just naive or inexperienced and can't see that their question is not even barely answerable. It's not really a question of blame, although some questions do deserve the quick closure or the lack of answers they get. Poor questions don't help the forum's long-term goals. In fact, they dilute the forum with fluff and flab. That's the nub of it.

Any policy has to be tempered by practicality, not least the fact that people are busy and have other things to do. I don't think it's an especially good use of the time of people of high reputation (~10K, ~100K) to be writing the same kinds of comments again and again to explain to people why they've posted rather poor questions. We are not a help line on which every caller must be answered. If we don't explain that well enough, we need to revisit the documentation.

I don't claim to be consistent, because I share in the conflict of motives here. Sometimes I just vote to close, half-hoping that someone else will explain directly to the OP why there is a move to close. Sometimes I add a comment myself. It's really hard to get the balance right, because often explaining the (sometimes several) reasons why a post doesn't pass muster is likely to appear hostile rather than supportive.

Answer 2

We shouldn't be chary of using the mechanism we've got to deal with questions that we think can't be or oughtn't to be answered before they're improved:

• People tend to assume their questions are good ones & to discount somewhat comments to the contrary as quibbling. Closing a question kills false hopes of getting a useful answer without improving it.

• Giving specific guidance on how to improve a question in comments is often a great help, & never does any harm. But voting to close a question doesn't at all preclude this.

• Voting to close a question brings in other people, through the review queues, who can also help improve it. It sets the wheels in motion & doesn't rely on a single user to keep coming back to a question.

• If we're right that a question can't be or oughtn't to be answered before it's improved, then any answers it gets won't be good ones, or of the kind we want. Sometimes precipitate answers can lead to confusion, for the OP and for readers, especially if the question's edited later.

• When closed questions get re-opened it's usually within a few hours of their being edited, & rarely more than a day. (I think—it'd be interesting if there's any data available.) A faster response would be better, & I expect we'll get faster as the site grows, but that's not bad.

• Questions that remain unanswered, un-upvoted, & closed are automatically deleted after nine days, de-cluttering the site. (If they're not closed they hang around for a year; & with two or more comments, for ever).

Now I'm not saying we always draw the line in the right place (I agree with some of the points you make on whether some kinds of question ought to be closed at all), but it is a matter of drawing a line, & I can't see the advantage of shilly-shallying—at best, if someone edits quickly in response to a comment alone, an answer might appear a little sooner—rather than acting promptly.

Answer 3

I personally do not see the fail-fast policy that we implicitly adopted as something bad per se. I think that the problem lies in slow re-opening.

Often when I see off-topic-like questions (e.g., too broad, programming focused) I comment on them asking OP's for clarification and at the same time vote to close them. I do so because - from my experience - in the vast majority of cases either the OP responds or edits the question immediately (which leads me to retracting the vote), or he/she does not respond at all. The problem is when the question was edited, so that it is no longer unclear, or software-focused, but it still does not get re-opened because it did not receive enough votes. In my opinion the problem lies in us focusing more on closing than on re-opening questions.

We do not gain anything by leaving the off-topic questions opened. In most cases off-topic questions lead to off-topic answers, unclear questions lead to unclear answers, and low-quality questions lead to low-quality answers. In many cases such questions get one or two random upvotes and sometimes even get answered what makes them part of our large collection of low-quality questions. When such questions stay on the site they clutter the search results, which in effect leads to more duplicated questions since people do not find relevant search results among hundreds of off-topic questions.

I think we should close off-topic questions fast, but re-open them even faster if they got edited.

Answer 4

I believe my vote to leave open / vote to reopen rates are rather higher than the norm on CV. I have also raised several majority "close"/"leave closed" votes on Meta where I disagreed strongly enough with the result. So in that sense I naturally sympathise with @gung's position.

Nevertheless, I do prefer a quick close to a drawn-out one. The cost of a slow close is that answers begin to appear before the question is sorted out. The process of editing and reopening a question is slow and laborious. The process of sorting out a question once answers have been posted, though, tends to be even more laborious and the threads that result are often poorer in quality (messier and harder for later readers to work through).

I think some of the substantive points of this question are, ironically, too broad! I have intended to raise several of them as standalone items in Meta threads of my own, if I get the chance. Let me just add some thoughts, currently rather unstructured.

I agree that self-study questions are often closed too early - if the OP has clearly made an honest attempt at the Q, it's silly and premature to close it just because the [self-study] is missing (but it happens). On the other hand if the OP has only made a de minimis attempt - or claims to have made an attempt but puts no details in the Q - I'd rather close it and give the OP time and space to edit it. This is partly because self-study Qs seem to attract Math SE style "fastest fingers in the West" answers, and once one of these has arrived the OP loses incentive to improve the Q. Moreover, if the OP shows their attempt, it often illustrates a mistake or misconception that is worth addressing in its own right. This is often how self-study questions add the most educational value. If the OP doesn't show their attempt, this is lost.

A related and possibly more serious (IMO) issue with self-study questions is we are very prone to closing as duplicates when two questions have similar mechanics to solve - but I think we often do it in a rather broad way, so that someone with the mathematical or statistical sophistication of the OP may be unable to translate the application of one question to another.

Incidentally, I think this is more serious because the close-edit-reopen cycle seems to be even less agile for duplicates. So closing as duplicate requires particularly careful consideration, both on self-study questions and others. The page notice on questions closed as dupes doesn't seem to encourage OPs to edit if they feel what they really were trying to get at is distinguishable from the target.

There are quite a few close reasons that I think should usually (unless the question is beyond redemption and the OP does not seem to be acting in good faith) be accompanied with an explanation of what the close voter thinks needs to be changed for the question to be left open (and I'd extend this to reopen votes). I'd particularly say this for too broad, unclear what you're asking and self-study. In the review process I am sometimes saddened at lack of engagement with OPs, who must find the process rather mystifying and disappointing. It would help if more read the help pages and "how to ask a good question" guides first, of course, but some aspects of this site's culture are not immediately obvious from the help pages. This is especially true when standards have changed over time but OPs have seen grandfathered Qs and presumed a question in similar vein would still be acceptable.

I agree that some questions on programming are closed despite a statistical component. In some cases it is worthwhile doing so so that the OP has the chance to separate out the statistical and programming aspects of the question - for instance to make the coding part work on a more minimal reproducible example, which brings the Q more within the scope of our site and renders it more useful to future readers. On balance I think we are probably too aggressive closing questions that include code, or which discuss statistical programming practices but I can sympathise with the view that "there's nothing more boring than reading questions about software you don't know". Personally I think this is something that is worth a rehash of "what software issues should we consider on-topic here", which is a vexed question! (For my money - algorithm questions should almost always be on-topic though sometimes they are currently closed; I might stretch so far as to say package recommendations particularly for more complex tasks should be more welcome here, but I think I'd be a minority voice; I wouldn't want the site to become a repository of SPSS how-to tutorials. I think this is going beyond the scope of the current question though.)

I'd also add something about dataset request questions - for specific datasets they should clearly be off-topic but often people ask for datasets that are exemplars of certain statistical properties (but not caring so much what practical context the data are from), and yet their questions are often also being closed and they are being redirected to Open Data SE, which does not seem appropriate in my view (and has been discussed on Meta before).