# Are we closing questions too fast?

It seems to me that we have been closing more and more questions, and that we are moving them into the close votes review queue very quickly. This seems to have evolved naturally, but I wonder if we have moved past the optimum point without anyone stopping to think about whether we are moving in the right direction. I hope this thread will give us a chance to do so now.

Whereas I used to think that most treads coming through the close vote queue should be closed, I am increasingly voting to leave open or just not voting at all on threads there (perhaps leaving a comment to the OP instead). I am somewhat reluctant to list specific threads here, because people have voted on them and I don't want anyone to feel defensive. However, I can make some general comments and people may be able to recognize cases that fall into that pattern:

1. duplicate: (Actually, I have no problem with how this is working.)

2. off topic: I see two issues in this category:

• self-study questions: My impression is that any question that seems like it might be an assignment is immediately voted to be closed as self-study. I do occasionally vote to close such questions immediately, but mostly I just leave a comment for the OP to add the tag and read the wiki. (There is suggested text that people can use here: How best to use the review queue?)

Many self-study questions come with 'I have this question: ... I thought it might be ... but I can't figure it out'. Such a case is 90% of the way to meeting our requirements, in my opinion. True, it doesn't have the tag, but new users are very unlikely to know our policies and may well be trying to get help in a completely honorable way. Even if they have less than that, it may still be viable to leave a comment and let them (potentially) edit.

Having been notified to add the tag, if they don't add it after a reasonable period of time (maybe a few hours), it seems appropriate to vote to close, but not before and not without any comment (or so it seems to me).

• ... focuses on programming...: It seems to me that any question that even mentions R (e.g.) is often summarily voted to close. I do continue to believe that we should focus on questions about statistics, machine learning, etc., and not about how to use software or tech support, despite the fact that you have to use software to do statistics, machine learning, etc. Nevertheless, I often suspect the situation is more ambiguous than just whether code is listed with the question.

3. unclear what you're asking: A lot of threads have been voted to close as unclear without a comment explaining what is unclear and what the OP could do about it to fix their question. I recognize that there are some questions that are so thoroughly unclear you can't even state what it needs. I would estimate those are about 10% of the unclear questions, though.

Depending on how bad it is, I sometimes leave a comment, and sometimes leave a comment and vote to close. (I rarely vote to close as unclear without a comment.) In some proportion of those cases, the OP comments in reply or edits their question to clarify the points at issue. There have been such cases where I have subsequently retracted my close vote. I think more can be done here...

4. too broad: (I probably overuse this myself.) I suspect there are relatively few threads that really are too broad, although some clearly do exist. (I remember one question that read, more or less, 'how do you fit models, and how do you select one, and how do you interpret it?', that's too broad.) There have been a sizeable proportion (albeit probably less than half) of questions voted as too broad, that I think could have been given a brief, but broad, answer that may well have be useful for the OP and for future users. The OP may then be able to ask a clearer, more specific question, or may find that even a one paragraph gloss is enough to get them started.

I also wonder if such questions may sometimes / eventually pull out thoughtful answers that are real gems. I wonder if a question like, what basic tests can I run on each dataset to make sure that they don't have typos, might be closed as too broad today. I myself have thought about asking a question or two before, but opted not to, because I couldn't formulate the question any better than, 'how should I think about ...' or 'what do people think about ...', where a broad, preliminary answer, or some discussion in comments might have put me on the right path forward. As with unclear, I think a comment to the OP may often be a better first action than voting to close straightaway.

5. primarily opinion based: These are more commonly fine, in my opinion. I am aware that SE policy is that all such questions be closed, so I admit I'm on shaky ground. Nonetheless, I believe that opinion-based questions can sometimes be of real value to people and could be allowed here, if just made CW. (Unfortunately, I do not have any clear guidance on how to draw the line, though.)

Above, I have contrasted things I've seen with what I do. I don't mean to hold myself up as the ideal that others need to match. My own behavior is simply my most natural point of reference. My goal here is to raise the topic for discussion. I am open to changing my own policies as well.

Edit:
My title may be a little misleading. The suggestion implied by the question isn't that a thread should spend a long time in the close votes review queue, or something like that. If a thread is to be closed, I agree we benefit from it closing as quickly as possible. Perhaps a better way to put it might have been 'Are we moving too many questions into the close votes review queue?' There is some—difficult to articulate—line between voting to close and not doing so. I wonder if that line has become too stringent. Relatedly, I wonder if a comment might not be enough in some cases, and should often be added if we are going to vote to close.

For example, @Tim notes that he always does both, which strikes me as a good policy. Below that, @GeneralAbrial makes a good point:

Part of the problem I have with the re-open interface is that the comments discussion is not visible -- so on a post on some obscure topic, precisely 0% of the question is clear to me, but if the comments say "please clarify these three issues," I can usually figure out whether those points have been clarified. Absent those hints, I usually just Skip.

It can be difficult for an OP to know what / how to edit their question, and difficult for someone to subsequently determine if the problem has been rectified, if what is unclear was left unspecified.

We have moved from a state of affairs where I used to think almost all of the threads in the close votes review queue should be closed (and those I didn't think should be closed were nonetheless understandable), to a situation where a large proportion (maybe close to half) don't necessarily need to be closed, in my opinion. Yesterday, for example, a homework question (Why is a square root not a linear transformation?) was asked. It labeled the question as an assignment, and included what the OP understood thus far. So instead of voting to close without comment, I left a comment to add the self-study tag and read its wiki. The OP added the tag in approximately 30 seconds. Despite this, someone subsequently voted to close the thread as off-topic / homework. (I had said I would avoid listing any specific examples so as not to make anyone feel attacked, but perhaps this will help make the topic clearer. I apologize for any hurt feelings.)

• +1 The motivation expressed in the introductory paragraph is a little unclear. If a post ought to be closed, then there can be no such thing as "too quickly": we are doing a favor to the OP to take action as soon as possible. Thus, one measure of whether there actually exists a problem is to examine how many posts are acted on in some way (close, delete, etc.) but the action had to be reversed because it was incorrect. I am aware of extremely few such threads. The problem with merely commenting is that there is no mechanism to help us follow up later, so commenting really means "do nothing." – whuber May 20 '16 at 21:13
• @whuber, for me, commenting means 'alert the OP to the issue & give them a chance to fix it'. You're right that there isn't a good method to follow up. I sometimes check back later, but sometimes don't. Nonetheless, sooner or later someone will see the thread, see that there was a problem & that nothing was done about it, & can vote to close then. The downside to this would be that the thread remained open in the interim. A related question I've thought about, but didn't explicitly ask here, is how bad that would be. – gung - Reinstate Monica May 20 '16 at 21:22
• Not all of my points here are about commenting, so we could always focus on the others. But to make the commenting issues more stark, imagine we had the power to instantly close a question as unclear w/o needing 4 more votes. So a question is posted & closed as unclear w/i a minute or 2, w/o any comment specifying what was unclear, or how to fix it. What is the OP supposed to do? What impression of our site will result? Etc. – gung - Reinstate Monica May 20 '16 at 21:28
• (1) You write as if closing a question precluded the OP from having a chance to fix it, but (as you know) that's not correct. In fact, if a reasonable request for a fix has been made, would it not be better for the question to be on hold as an indication to readers that a change will occur? (2) The system displays a clear, albeit general, message when questions are put on hold for being unclear or too broad, etc. In cases where that message suffices (such as unresearched homework questions), why should we feel a need to add any comments? – whuber May 20 '16 at 21:33
• @whuber, I didn't intend to imply that the OP couldn't fix their post. You are right that they can. I suppose it seems to be that there are gradations of the level of (say) unclarity in a Q; w/ higher levels, closing is better, but w/ lower levels, a comment may be enough. A line must be drawn somewhere. Surely a post shouldn't be closed if it is missing a single comma. I am wondering if the line should be loosened somewhat from its current location. I am opening the topic for discussion. The answer may be 'no', & that would be fine, too. – gung - Reinstate Monica May 20 '16 at 22:12
• My concern w/ the system's messages is that I think they are typically too broad & unclear to be useful, in that they are unspecific. Here is the 'unclear' message: "Please clarify your specific problem or add additional details to highlight exactly what you need. As it's currently written, it’s hard to tell exactly what you're asking." What 'additional details' need to be added? W/o a comment to the OP, they may not be able to tell. The additional details needed are likely to vary from post to post. – gung - Reinstate Monica May 20 '16 at 22:16
• The current close -> edit -> reopen process isn't necessarily bad, but it certainly isn't very agile. It can take hours or days to play out. OTOH, I've had cases where I left comments to the OP, they edited, & I deleted my comment or retracted my close vote w/i 15 minutes. – gung - Reinstate Monica May 20 '16 at 22:19
• "Nonetheless, sooner or later someone will see the thread, see that there was a problem & that nothing was done about it, & can vote to close then." - A lot of our lingering unanswered questions haven't been automatically deleted because there are comments on them (that haven't been responded to) & yet they haven't been closed. – Scortchi - Reinstate Monica May 20 '16 at 22:34
• You're right, @Scortchi. That's the other side of this. So a follow up question is, how bad is that state of affairs? How much does it hurt us that we have lingering, unanswered questions that should be closed at this point but haven't been? It does seem to bother people (every so often, someone takes to meta.CV to complain about it), but why does it actually matter that much? Do we lose credibility with the SE power structure as a result? – gung - Reinstate Monica May 21 '16 at 3:00
• It's not so bad; & we shouldn't be changing what we do just to improve the site statistics. Still, if a question's not likely to get a decent answer because it's flawed, closing it sends a strong message to the OP that it needs improvement, & the automatic deletion of closed questions after some time removes clutter. And the quicker the better. – Scortchi - Reinstate Monica May 21 '16 at 8:46
• The square root Q was migrated to Math.SE which seems to make sense. So I am not sure it's a good example (or perhaps I misunderstood your point). – amoeba May 23 '16 at 19:10
• @amoeba, I don't object to it having been migrated to math.SE. I do object to it being closed as off topic -> homework. – gung - Reinstate Monica May 23 '16 at 19:18
• FWIW, recent (last month's) statistics show that the site moderators (a) close (queued) questions at substantially lower rates than most people and (b) spend slightly more time with each decision than most people. I suspect both are because we tend to act cautiously and conservatively, letting the community lead concerning the gray areas. Thus, this thread is of particular value both for the moderators and to help the community see where it's going and to reflect on that. – whuber May 26 '16 at 20:17
• on whuber's comment -- I know I'm much more hesitant to mod-hammer closure than when I was just one vote among many. If it seems quite obvious it should close, then I might do it alone, but often I'll wait for several votes first. I'm happiest if I can come in as the last needed vote (reducing my impact to that of an ordinary high reputation user). I'd vote to close with something nearer to my old frequency if I could turn off the diamond. – Glen_b Jun 2 '16 at 1:43
• I dunno, @Glen_b. I voted for you. I'm perfectly happy for you to close those threads you believe clearly should be closed, or to leave open those threads you believe clearly merit it. More recently, I have started voting (in the close vote queue) more like a moderator, in that I abstain from voting. However, I still vote 1 way or another on more than half. I would be comfortable w/ those votes if they were final. – gung - Reinstate Monica Jun 2 '16 at 1:50

Some excellent and thoughtful comments in existing answers from long-standing members still leave a little scope for additional remarks, from the Dark Side if you will.

Some respected colleagues here seem to be leaning towards an ideal of indefinite courtesy, namely that people posting poor questions may require and deserve support until their questions have been rewritten as acceptable and that it's our duty to provide that so far as we can.

I don't vote for being rude (who does?), but I vote for being firm and for spending time and effort in ways that promote the best long-term interests of the site, which pivot on building up a core of well-posed, well-answered questions.

I don't buy the obligation to indefinite courtesy as a matter of practicality. We have guidelines on the kinds of questions we support, which over time have shifted and which in the future may be changed again. But at any one time many questions are just off-topic and/or poor. Sometimes people just have not read the advice in the Help Center. Sometimes people are just trying to outsource their homework -- or even more frequently their basic thinking on Master's or research projects -- to the internet. Sometimes people are just naive or inexperienced and can't see that their question is not even barely answerable. It's not really a question of blame, although some questions do deserve the quick closure or the lack of answers they get. Poor questions don't help the forum's long-term goals. In fact, they dilute the forum with fluff and flab. That's the nub of it.

Any policy has to be tempered by practicality, not least the fact that people are busy and have other things to do. I don't think it's an especially good use of the time of people of high reputation (~10K, ~100K) to be writing the same kinds of comments again and again to explain to people why they've posted rather poor questions. We are not a help line on which every caller must be answered. If we don't explain that well enough, we need to revisit the documentation.

I don't claim to be consistent, because I share in the conflict of motives here. Sometimes I just vote to close, half-hoping that someone else will explain directly to the OP why there is a move to close. Sometimes I add a comment myself. It's really hard to get the balance right, because often explaining the (sometimes several) reasons why a post doesn't pass muster is likely to appear hostile rather than supportive.

• I think this is an excellent answer. And the penultimate paragraph and final paragraph are very sensible. – Silverfish May 24 '16 at 13:26
• @Silverfish Thanks. I'd add that gung's set of already written comments strike a very good balance between gentle and firm, but I find it quicker to write something than to find his list and copy and paste the best one. There should be a smarter way to add comments that mostly come from about ten reasons why a question is not going to work in its present form. – Nick Cox May 25 '16 at 6:07
• I think @glen_b uses a browser extension that lets him add predefined comments without having to go and look them up then do a copy-and-paste job. – Silverfish May 25 '16 at 8:09
• @Silverfish +1. NickCox is definitely my favourite answerer on Meta. – amoeba May 25 '16 at 9:50
• I'm the Darth Vader of CV? – Nick Cox May 25 '16 at 10:39
• @Silverfish I do indeed. I find it has dramatically sped up my ability to leave comments -- it offers about half a dozen slots of question-comments and about half a dozen slots for answer comments. The default ones aren't bad but I've edited or replaced several of them based on a answers to a question here on meta plus my own thoughts. What happens is once it's installed in your browser, ...ctd – Glen_b Jun 2 '16 at 1:58

We shouldn't be chary of using the mechanism we've got to deal with questions that we think can't be or oughtn't to be answered before they're improved:

• People tend to assume their questions are good ones & to discount somewhat comments to the contrary as quibbling. Closing a question kills false hopes of getting a useful answer without improving it.

• Giving specific guidance on how to improve a question in comments is often a great help, & never does any harm. But voting to close a question doesn't at all preclude this.

• Voting to close a question brings in other people, through the review queues, who can also help improve it. It sets the wheels in motion & doesn't rely on a single user to keep coming back to a question.

• If we're right that a question can't be or oughtn't to be answered before it's improved, then any answers it gets won't be good ones, or of the kind we want. Sometimes precipitate answers can lead to confusion, for the OP and for readers, especially if the question's edited later.

• When closed questions get re-opened it's usually within a few hours of their being edited, & rarely more than a day. (I think—it'd be interesting if there's any data available.) A faster response would be better, & I expect we'll get faster as the site grows, but that's not bad.

• Questions that remain unanswered, un-upvoted, & closed are automatically deleted after nine days, de-cluttering the site. (If they're not closed they hang around for a year; & with two or more comments, for ever).

Now I'm not saying we always draw the line in the right place (I agree with some of the points you make on whether some kinds of question ought to be closed at all), but it is a matter of drawing a line, & I can't see the advantage of shilly-shallying—at best, if someone edits quickly in response to a comment alone, an answer might appear a little sooner—rather than acting promptly.

• I find your fourth bullet point most important: closing unclear questions prevents users from posting answers that may be worse than no answer at all. – Stephan Kolassa May 22 '16 at 18:45
• @StephanKolassa: Perhaps the third gives it a run for its money. – Scortchi - Reinstate Monica May 22 '16 at 19:32

I believe my vote to leave open / vote to reopen rates are rather higher than the norm on CV. I have also raised several majority "close"/"leave closed" votes on Meta where I disagreed strongly enough with the result. So in that sense I naturally sympathise with @gung's position.

Nevertheless, I do prefer a quick close to a drawn-out one. The cost of a slow close is that answers begin to appear before the question is sorted out. The process of editing and reopening a question is slow and laborious. The process of sorting out a question once answers have been posted, though, tends to be even more laborious and the threads that result are often poorer in quality (messier and harder for later readers to work through).

I think some of the substantive points of this question are, ironically, too broad! I have intended to raise several of them as standalone items in Meta threads of my own, if I get the chance. Let me just add some thoughts, currently rather unstructured.

I agree that self-study questions are often closed too early - if the OP has clearly made an honest attempt at the Q, it's silly and premature to close it just because the [self-study] is missing (but it happens). On the other hand if the OP has only made a de minimis attempt - or claims to have made an attempt but puts no details in the Q - I'd rather close it and give the OP time and space to edit it. This is partly because self-study Qs seem to attract Math SE style "fastest fingers in the West" answers, and once one of these has arrived the OP loses incentive to improve the Q. Moreover, if the OP shows their attempt, it often illustrates a mistake or misconception that is worth addressing in its own right. This is often how self-study questions add the most educational value. If the OP doesn't show their attempt, this is lost.

A related and possibly more serious (IMO) issue with self-study questions is we are very prone to closing as duplicates when two questions have similar mechanics to solve - but I think we often do it in a rather broad way, so that someone with the mathematical or statistical sophistication of the OP may be unable to translate the application of one question to another.

Incidentally, I think this is more serious because the close-edit-reopen cycle seems to be even less agile for duplicates. So closing as duplicate requires particularly careful consideration, both on self-study questions and others. The page notice on questions closed as dupes doesn't seem to encourage OPs to edit if they feel what they really were trying to get at is distinguishable from the target.

There are quite a few close reasons that I think should usually (unless the question is beyond redemption and the OP does not seem to be acting in good faith) be accompanied with an explanation of what the close voter thinks needs to be changed for the question to be left open (and I'd extend this to reopen votes). I'd particularly say this for too broad, unclear what you're asking and self-study. In the review process I am sometimes saddened at lack of engagement with OPs, who must find the process rather mystifying and disappointing. It would help if more read the help pages and "how to ask a good question" guides first, of course, but some aspects of this site's culture are not immediately obvious from the help pages. This is especially true when standards have changed over time but OPs have seen grandfathered Qs and presumed a question in similar vein would still be acceptable.

I agree that some questions on programming are closed despite a statistical component. In some cases it is worthwhile doing so so that the OP has the chance to separate out the statistical and programming aspects of the question - for instance to make the coding part work on a more minimal reproducible example, which brings the Q more within the scope of our site and renders it more useful to future readers. On balance I think we are probably too aggressive closing questions that include code, or which discuss statistical programming practices but I can sympathise with the view that "there's nothing more boring than reading questions about software you don't know". Personally I think this is something that is worth a rehash of "what software issues should we consider on-topic here", which is a vexed question! (For my money - algorithm questions should almost always be on-topic though sometimes they are currently closed; I might stretch so far as to say package recommendations particularly for more complex tasks should be more welcome here, but I think I'd be a minority voice; I wouldn't want the site to become a repository of SPSS how-to tutorials. I think this is going beyond the scope of the current question though.)

I'd also add something about dataset request questions - for specific datasets they should clearly be off-topic but often people ask for datasets that are exemplars of certain statistical properties (but not caring so much what practical context the data are from), and yet their questions are often also being closed and they are being redirected to Open Data SE, which does not seem appropriate in my view (and has been discussed on Meta before).

• I was thinking about an answer but I agree with almost all of this one. The only thing I disagree with is the inclusion of self study among close reasons that usually benefit from being accompanied by specific guidance in comments - surely the automatically generated text is clear enough. Trying to cajole someone into making more than a half-hearted attempt to do their own homework is tedious & usually unsuccessful, whereas helping someone to clarify a real question is useful & rewarding. – Scortchi - Reinstate Monica May 21 '16 at 13:41
• @Scortchi Yes I can understand that there is a cost/benefit thing to posting a comment beneath a question of any kind (one often ends up, over the day or so, engaged in a comment ping-pong that one might have been better without) and if the question is likely to be closed, and particularly if the automatically generated text is clear once closing is complete, one might be better off without it. – Silverfish May 21 '16 at 15:07
• On balance I still prefer engagement in this case (though, for the reasons you give, this isn't so clear-cut as for unclear/too broad) because (a) I like gung's suggested comment text (the one he links to), (b) I have found - as gung does - that quite a few OPs are quite receptive and responsive to it, and this avoids the close-edit-review cycle (which, as gung says in the comments above, is not very agile), (c) if the OP is confused about the requirements they now have a chance of human interaction via the comments thread. – Silverfish May 21 '16 at 15:10
• I like @gung's text for self study better than the close reason text. Perhaps we could change the latter. I don't know if we can edit the duplicate close reason text - gung's text is better for that too, as it does encourage editing when needed. – Scortchi - Reinstate Monica May 21 '16 at 18:24
• About the close as duplicate notice, if my memory does not fail there is a message which will appear only to OP with guidance about what to do next. – Andre Silva May 22 '16 at 13:54
• @AndreSilva: I didn't know that. Anyone know what it is? – Scortchi - Reinstate Monica May 22 '16 at 19:33
• – Andre Silva May 22 '16 at 19:59
• @AndreSilva: Thanks. That seems good enough. – Scortchi - Reinstate Monica May 22 '16 at 21:54

I personally do not see the fail-fast policy that we implicitly adopted as something bad per se. I think that the problem lies in slow re-opening.

Often when I see off-topic-like questions (e.g., too broad, programming focused) I comment on them asking OP's for clarification and at the same time vote to close them. I do so because - from my experience - in the vast majority of cases either the OP responds or edits the question immediately (which leads me to retracting the vote), or he/she does not respond at all. The problem is when the question was edited, so that it is no longer unclear, or software-focused, but it still does not get re-opened because it did not receive enough votes. In my opinion the problem lies in us focusing more on closing than on re-opening questions.

We do not gain anything by leaving the off-topic questions opened. In most cases off-topic questions lead to off-topic answers, unclear questions lead to unclear answers, and low-quality questions lead to low-quality answers. In many cases such questions get one or two random upvotes and sometimes even get answered what makes them part of our large collection of low-quality questions. When such questions stay on the site they clutter the search results, which in effect leads to more duplicated questions since people do not find relevant search results among hundreds of off-topic questions.

I think we should close off-topic questions fast, but re-open them even faster if they got edited.

• Part of the problem I have with the re-open interface is that the comments discussion is not visible -- so on a post on some obscure topic, precisely 0% of the question is clear to me, but if the comments say "please clarify these three issues," I can usually figure out whether those points have been clarified. Absent those hints, I usually just Skip. – Sycorax May 23 '16 at 12:58
• @GeneralAbrial agree that it is a problem with flawed un-close mechanism -- it was rised multiple times on meta-meta (e.g. meta.stackexchange.com/questions/125/…) – Tim May 23 '16 at 13:12
• @GeneralAbrial, if you click the tab in the upper left corner, you can switch the view to see the thread as displayed on the main site. Then you can view the comments. I always read through them (including clicking the 'see more comments' link) to ensure I understand the issues before I vote on reopening. – gung - Reinstate Monica May 23 '16 at 13:34
• (+1) I "skip" an awful lot of reopen discussions, sadly, partly because of how clunky the interface is but more often because I'd rather the people who have voted to close have a look at whether the points they raised (or felt, without explicitly writing down!) had been sufficiently addressed. Sometimes I feel perfectly capable of judging that for myself, but other times I get the feeling there's something more subtle going on. – Silverfish May 24 '16 at 21:02