How can we best help to improve the questions asked on our site?

Some differences of opinion have cropped up concerning how to go about clarifying a less than perfectly formulated question. Because comment threads on our main site are not the best venue for airing these opinions, please let us discuss them here on meta. Should people just jump in to provide answers? Should we prefer first using comments to elicit clarifying information from the OPs and offer answers only when a question becomes unambiguously clear? How far should we go to help an OP reformulate a highly ambiguous or broad question?

These are important issues for our site that other sites do not confront to such a degree. They occur because statistical problems often arise out of such ambiguities; people from many disciplines use statistical procedures but do not have the terminology to talk about them; and because people often ask the wrong question at the outset and need guidance to ask the question that really matters.

In replying, please consider also offering your views about how best our community can help improve the questions and the site as a whole by employing its power to upvote, downvote, close, and delete questions.

• I get that, as a moderator, you were trying to squash a conflict, but I'm not sure how constructive it is to suggest that I was in the wrong for suggesting that Michael could be more helpful to the community by modifiying his "answer first, ask question later, if ever" approach. I've suggested that Michael consider using close votes (e.g. here: stats.stackexchange.com/questions/32000/…) before but to no avail. He's on this site as much or more than anyone else so I was just pointing out that he can aid in Jul 16 '12 at 19:55
• (cont)this vitally important service to the site. I suggested it again in a situation I (still) think is completely justified but after apparently being validated by you in the discussion today, I doubt that much progress will be seen on that front. You know as well as I do that the "close" system is rather broken on this site because almost no one uses it ... p.s. I don't think I was rude or unwelcoming to the OP in that thread - I unemotionally suggested that she provide more details and perhaps peruse the wealth of existing questions on the subject, so I'm not sure I understand that comment Jul 16 '12 at 20:00
• @Macro I'm sorry you took any of my comments (or this question on Meta) as suggesting you have been wrong about anything, because I don't think that. (I think you and Michael are both right, which is what makes this a thorny issue.) I just hope there may be some more creative and constructive ways to deal with it than have been exhibited recently. Gathering ideas about that is the purpose of the present thread. I don't think anyone wants to see 15+ comments develop beneath almost every question and answer on our site. (I exaggerate, but you get the point, I know.)
– whuber Mod
Jul 16 '12 at 20:06
• To be clear: the present question is not about you or any other user in our community specifically. It's a question of policy, practice, (and maybe etiquette) that I hope we can discuss collegially and impersonally.
– whuber Mod
Jul 16 '12 at 20:08
• related meta thread: meta.stats.stackexchange.com/questions/1067/… Jul 17 '12 at 13:07
• One note for the community: I appreciate that @PeterFlom had written this post on his personal blog, & has started linking to it a couple of times. Since noticing it, I have also linked to it in a comment & plan to use it in the future. Jul 19 '12 at 16:20

Overall, I think that more effort, as a community, needs to be put into raising the quality of the answers and not answering low quality questions. My point can be summarized as: It seems to me that only bad things can come from answering a question that two rational individuals can interpret completely differently. In situations like that, posting an answer without clarifying does more harm than good, as it can easily mislead future readers of the question. We are all statistics experts and can tell when a question matches that criteria. In that case, my suggestion is seek a minimal amount of clarification from the OP to the point where the question no longer meets that criteria (before answering) and close voting (which seldom happens, see cardinal's data below) if that is unsuccessful rather than just ignoring the question or answering one possible interpretation of it. There are a handful of high-rep users who do this quite well and, I think, lead by example (that's where I learned to do this) but there are others that take little/no role in improving this aspect of the site and their help could improve things.

Here are some other thoughts I have on the subject:

I think a major issue with this site, as opposed to other more populated sites like stackoverflow, is that the close system basically doesn't work here because we have relatively few members with 3000+ rep and even fewer who actively use their close votes. I think I can say, without hyperbole, that I've only observed maybe 10 users cast a close votes (obviously this is biased low, since I only see the votes that result in a closure) other than moderators. This propagates the existence of low-quality questions on the site since - let's face it - the mods are not omniscient.

A second issue, that I can understand happening with inexperienced users of the site but seems less excusable when we're talking about a high-rep user, is that there occasionally is an "post an answer first, ask questions later" attitude, which can result in answers being posted that actually answers a radically different question than the one posed, and, if the question is never clarified by the OP (which happens a fair amount of the time when the OP is a one-time user) can serve as a misleading direction for future readers of the question who interpreted it differently. For this reason, my opinion is that it's important to clarify first, answer second, rather than the other way around.

• Some data: (Questions migrated or deleted) There have been 998 votes to close on this site since inception. Of these, 502 of them were on 345 questions that were either migrated or later deleted. At least 342 of those votes were from a moderator. Of these 345 questions, 235 of them were handled by a single close vote, i.e., a moderator taking sole action. Jul 16 '12 at 23:31
• Some data: (Other questions receiving close votes) Of the remaining questions on this site, 330 of them have received at least one close vote. Of those, 222 received exactly 1. A total of 244 of the 330 were ultimately closed with 158 of those done with one vote (i.e., by a unilateral moderator action). There have been a total of 6 questions that have received the full five close votes on this site (3 were migrated/deleted and the other 3 were closed). We can thus infer that at least $241 + 342 = 583$ (58.4%) of the sum total close votes on this site have been made by moderators. Jul 16 '12 at 23:33
• (I have personally cast another 50 close votes, starting in November 2011.) Jul 16 '12 at 23:36
• Here are the two questions receiving the full five votes: 28062 and 29303. Questions 26784 and 26826 were deleted and questions 26154 and 30238 were migrated to SO. Jul 16 '12 at 23:47
• So do you have any suggestions for how the community should conduct itself in the future? Unfortunately just saying "more effort, as a community, needs to be put into raising the quality of the answers and not answering low quality questions" in a meta thread is unlikely to result in changes in behavior. Jul 17 '12 at 2:13
• @AndyW, my point was the bolded statement immediately after the quote you gave and, I think, almost all of the "regulars" on this site have enough expertise to know when a question falls into that category. In that case, I'm suggesting asking for clarification and perhaps subsequently close voting rather than ignoring the question or just answering one possible interpretation of the question. I tried to make that point but maybe I wasn't successful. I edited the first paragraph accordingly. Jul 17 '12 at 11:39
• So to be clear you are just suggesting what site etiquette should be in such cases? IMO asking for clarification happens in many cases (via very judicious use of comments on this site). I agree with your suggestions for etiquette for the most part, but IMO this just comes off as a rant against one person (whose behavior certainly isn't representative of all the individuals on the site). Jul 17 '12 at 12:21
• @AndyW, I took the question as asking what site etiquette should be so, yes, I guess that's the scope of my suggestion. The main example seems very on topic to me. Yes, a particular user was involved, but whuber himself indicated (in that question) he, potentially, saw merits in answering a question under the given circumstances, so it seems like a good example for discussion. Also, given that the linked question was the one that lead to this meta thread, I thought it was a good example of why, I think, clarification should be sought in that case and others like it. Jul 17 '12 at 12:31
• I do think that the lack of close-vote use is a more widespread issue so I think you've correctly pointed out that singling out that one example doesn't really add anything to my answer so I deleted it. I agree that seeking clarification does happen frequently by particular, active members of the site. But, it takes almost no time to search the site for a very low quality question that probably should've been closed or clarified months/years ago. The number of such questions could be reduced if there were more already active members who put effort into doing so - which is my entire point. Jul 17 '12 at 12:33
• I would have thought with your reputation you could close a question on your own anyway (but apparently only moderators can?) There are things we can try to do to increase close votes, some sites have community "round-ups" in which they try to deal with old un-answered questions (such as by giving new answers, upvote current answers, or delete/close because of an absentee OP). I'm just prodding for a clear exposition on your ideals and any actionable advice for the general community. Jul 17 '12 at 12:41
• @cardinal & Macro, I do agree that the close vote system doesn't work well on CV b/c there are few users >3k, & few of those are sufficiently active. However, thresholds for close votes vary, & I think comments are generally a better approach. I have often voted to close for migration purposes, but I think only twice for other reasons. Eg 29303, while pathetic (to be blunt) also elicits some sympathy from me. This person does not appear to have put much effort in, but also is clearly not a native English speaker: ... Jul 19 '12 at 16:09
• the question is somewhat mangled & they may not have been able to follow the documentation well, leading to frustration & a sense of helplessness. These can be true even for native English speakers if they really don't know much about stats or programming. OTOH, this gets under my skin. Their English is fine (albeit not native), they are a professor!, & have neither bothered to do their homework first nor accepted any answer. Jul 19 '12 at 16:09
• While having English as the primary language on CV makes a lot of sense, I wish we would make it more clear to nonnative speakers that they should feel free to post in their native language if it allows them to more freely express themselves. I suspect we have enough fairly active users that could adequately translate from several major languages to make this feasible. Maybe we should actively encourage (e.g., via comments) such posting in certain cases where it seems like language is the primary barrier, but I haven't thought that through carefully. Jul 19 '12 at 16:18
• @cardinal There is an official SE policy on non-English questions. I think it's a little Draconian, but it does make sense. For background, check out the SO meta posts.
– whuber Mod
Jul 19 '12 at 19:19
• @whuber: (+1) Thanks. At first blush, I see little reason to follow this policy to the letter. I think if we see a user struggling to express themselves and can help them get mileage out of the site, we should be flexible in how we go about it. If that means suggesting via comment that they edit to include a non-English version, I don't see much problem with that, provided the question is edited to include a full English traslation. I agree that posting a non-English question from the outset may be a little undesirable. Jul 19 '12 at 19:24

I guess that I'm mostly just repeating what others have already said, but here are my scattered thoughts on this...

There are different kinds of questions on this site. I think that questions from non-statisticians that are trying to apply statistics (ranging from "can I use the t-test for my data?!"-type questions to thorough descriptions of their application) are the ones that usually require clarification and reformulation.

When I meet a non-statistician for a consulting session, I like to start by discussing their research problem (they are usually researchers) without talking about statistics. It is important to understand the actual problem instead of letting oneself be confused by the client's attempt to describe their problem in statistical terms. As most of you undoubtably know, it is not uncommon that instead of answering "yes, you can use the t-test" the statistician finds herself saying "you're not really interested in comparing means!"

I guess that my goal when I work as a consultant boils down to the following statement: make sure that both you and the client (OP) understand the problem. On CV, that sometimes requires lots of comments. Sometimes it requires a well-written and well thought-out answer. It seldom requires a quick answer.

StackExchange sites do not lend themselves well to the kind of dialogue that is essential for statistical discussions. We leave comments. Sometimes we forget to check back on the thread, because the OP didn't know how to use the @ function. Sometimes we don't get any replies at all. Sometimes someone else writes a good answer before there is a reply to the comment. Sometimes someone writes a bad answer without waiting for the clarification, which may cause all kinds of headaches.

With the reputation/badge system being what it is, I don't see quickly written answers going away anytime soon. That being said, when we write our answers, we should at least try to be clear about how we interpreted the question. It seems obvious that we should try to avoid implicit assumptions in our answers, especially when the question is vague or perhaps even in error. Indeed, this should always be done in statistical communication - CrossValidated is no exception!

What makes clarity even more important is that months and years from now, people will still find the questions and answers posted here in Google searches. We write our answers not only to the OP, but to the non-statisticians of tomorrow as well.

Finally, I don't think that votes is a very good way of handling questions that need clarification or improvement. Downvoting questions by low-rep users can be counter-productive in that it causes the user to flee the site without clarifying their question.

• +1 Great reply. My favorites: "It seldom requires a quick answer" and "we should at least try to be clear about how we interpreted the question."
– whuber Mod
Jul 18 '12 at 12:35
• (+1) (I think my previous joke may not be so easy to decipher as such, so I removed it.) Let me just say that I agree wholeheartedly with this answer to the point of wishing I had been capable of writing such a nice one myself. Your thoughts almost perfectly match my own on this matter. :) Jul 20 '12 at 1:13
• I agree especially with "StackExchange sites do not lend themselves well to the kind of dialogue that is essential for statistical discussions", but it may persuade me that it's not worth spending my time on this site. May 26 '20 at 15:43