It seems to me that we have been closing more and more questions, and that we are moving them into the close votes review queue very quickly. This seems to have evolved naturally, but I wonder if we have moved past the optimum point without anyone stopping to think about whether we are moving in the right direction. I hope this thread will give us a chance to do so now.
Whereas I used to think that most treads coming through the close vote queue should be closed, I am increasingly voting to leave open or just not voting at all on threads there (perhaps leaving a comment to the OP instead). I am somewhat reluctant to list specific threads here, because people have voted on them and I don't want anyone to feel defensive. However, I can make some general comments and people may be able to recognize cases that fall into that pattern:
duplicate: (Actually, I have no problem with how this is working.)
off topic: I see two issues in this category:
self-study questions: My impression is that any question that seems like it might be an assignment is immediately voted to be closed as self-study. I do occasionally vote to close such questions immediately, but mostly I just leave a comment for the OP to add the tag and read the wiki. (There is suggested text that people can use here: How best to use the review queue?)
Many self-study questions come with 'I have this question: ... I thought it might be ... but I can't figure it out'. Such a case is 90% of the way to meeting our requirements, in my opinion. True, it doesn't have the tag, but new users are very unlikely to know our policies and may well be trying to get help in a completely honorable way. Even if they have less than that, it may still be viable to leave a comment and let them (potentially) edit.
Having been notified to add the tag, if they don't add it after a reasonable period of time (maybe a few hours), it seems appropriate to vote to close, but not before and not without any comment (or so it seems to me).
... focuses on programming...: It seems to me that any question that even mentions
R(e.g.) is often summarily voted to close. I do continue to believe that we should focus on questions about statistics, machine learning, etc., and not about how to use software or tech support, despite the fact that you have to use software to do statistics, machine learning, etc. Nevertheless, I often suspect the situation is more ambiguous than just whether code is listed with the question.
For example, a post may have a mix of statistical and coding questions, in which case, I believe the post could stay open to answer the statistical questions, and we could leave a comment explaining that the coding questions may not get an answer. I have seen other cases where there was a coding error, but I suspect the reason the OP made a coding error is not because they misunderstood the code, but because they don't properly understand the underlying statistical concepts. In that case, I believe the Q should stay open so we can address the 'real' issue. Likewise, there are other cases where people are explicitly asking about code, but presumably wouldn't if they knew the statistical issues better. I often leave the "Ambiguous code / statistical question" comment from the meta.CV thread linked above in situations like that. Etc.
unclear what you're asking: A lot of threads have been voted to close as unclear without a comment explaining what is unclear and what the OP could do about it to fix their question. I recognize that there are some questions that are so thoroughly unclear you can't even state what it needs. I would estimate those are about 10% of the unclear questions, though.
Depending on how bad it is, I sometimes leave a comment, and sometimes leave a comment and vote to close. (I rarely vote to close as unclear without a comment.) In some proportion of those cases, the OP comments in reply or edits their question to clarify the points at issue. There have been such cases where I have subsequently retracted my close vote. I think more can be done here...
too broad: (I probably overuse this myself.) I suspect there are relatively few threads that really are too broad, although some clearly do exist. (I remember one question that read, more or less, 'how do you fit models, and how do you select one, and how do you interpret it?', that's too broad.) There have been a sizeable proportion (albeit probably less than half) of questions voted as too broad, that I think could have been given a brief, but broad, answer that may well have be useful for the OP and for future users. The OP may then be able to ask a clearer, more specific question, or may find that even a one paragraph gloss is enough to get them started.
I also wonder if such questions may sometimes / eventually pull out thoughtful answers that are real gems. I wonder if a question like, what basic tests can I run on each dataset to make sure that they don't have typos, might be closed as too broad today. I myself have thought about asking a question or two before, but opted not to, because I couldn't formulate the question any better than, 'how should I think about ...' or 'what do people think about ...', where a broad, preliminary answer, or some discussion in comments might have put me on the right path forward. As with unclear, I think a comment to the OP may often be a better first action than voting to close straightaway.
primarily opinion based: These are more commonly fine, in my opinion. I am aware that SE policy is that all such questions be closed, so I admit I'm on shaky ground. Nonetheless, I believe that opinion-based questions can sometimes be of real value to people and could be allowed here, if just made CW. (Unfortunately, I do not have any clear guidance on how to draw the line, though.)
Above, I have contrasted things I've seen with what I do. I don't mean to hold myself up as the ideal that others need to match. My own behavior is simply my most natural point of reference. My goal here is to raise the topic for discussion. I am open to changing my own policies as well.
My title may be a little misleading. The suggestion implied by the question isn't that a thread should spend a long time in the close votes review queue, or something like that. If a thread is to be closed, I agree we benefit from it closing as quickly as possible. Perhaps a better way to put it might have been 'Are we moving too many questions into the close votes review queue?' There is some—difficult to articulate—line between voting to close and not doing so. I wonder if that line has become too stringent. Relatedly, I wonder if a comment might not be enough in some cases, and should often be added if we are going to vote to close.
For example, @Tim notes that he always does both, which strikes me as a good policy. Below that, @GeneralAbrial makes a good point:
Part of the problem I have with the re-open interface is that the comments discussion is not visible -- so on a post on some obscure topic, precisely 0% of the question is clear to me, but if the comments say "please clarify these three issues," I can usually figure out whether those points have been clarified. Absent those hints, I usually just Skip.
It can be difficult for an OP to know what / how to edit their question, and difficult for someone to subsequently determine if the problem has been rectified, if what is unclear was left unspecified.
We have moved from a state of affairs where I used to think almost all of the threads in the close votes review queue should be closed (and those I didn't think should be closed were nonetheless understandable), to a situation where a large proportion (maybe close to half) don't necessarily need to be closed, in my opinion. Yesterday, for example, a homework question (Why is a square root not a linear transformation?) was asked. It labeled the question as an assignment, and included what the OP understood thus far. So instead of voting to close without comment, I left a comment to add the self-study tag and read its wiki. The OP added the tag in approximately 30 seconds. Despite this, someone subsequently voted to close the thread as off-topic / homework. (I had said I would avoid listing any specific examples so as not to make anyone feel attacked, but perhaps this will help make the topic clearer. I apologize for any hurt feelings.)