19
$\begingroup$

This recent question has just been converted into a community wiki (CW): When (if ever) is a frequentist approach substantively better than a Bayesian?

I understand that the idea behind converting questions into CW is that they are rather opinion-based (but not enough to be closed as opinion-based) and invite many answers as opposed to one definite answer. But I do not understand where is the border here.

E.g. these two very related questions: Who Are The Bayesians? and When are Bayesian methods preferable to Frequentist? -- are not CW (even though List of situations where a Bayesian approach is simpler... is).

I have the feeling that we are not being very consistent.

This recent question comes to mind: Approximate $e$ using Monte Carlo Simulation -- and it is not CW either, even though there are already several suggestions in the answers and one can imagine more answers appearing with various suggestions. As another example, we have plenty of non-CW How to explain something to a layperson? questions with lots of good answers.

So: What are the criteria to make questions CW according to our current policy?

Having asked that, I should say that I find CW very unhelpful because it does not allow to gain rep. I do not see why the frequentist-vs-bayesian question should be CW and I would prefer that everybody would gain reputation from answers there. (Personally, I have started to draft an answer to this question earlier today but now I feel much less inclined to continue.)

Should we consider relaxing our criteria?

I would also like to point out that on some other SE websites (e.g. on academia.SE) having lots of opinion-based answers is considered okay, and such questions are not converted to CW. So it seems that our policy is not a general SE-wide rule, but a result of our own choice.

$\endgroup$
15
  • 3
    $\begingroup$ I believe I made mine CW from the start. It clearly seems like a CW question to me. This is a good question, though. $\endgroup$
    – gung - Reinstate Monica
    Commented Feb 5, 2016 at 17:21
  • 3
    $\begingroup$ (+1) I think this is a good issue to discuss; thank you for bringing it up. Please note that I have been tracking the Frequentist-is-better question since its inception and converted it to CW only when the answers posted there compelled me to make the change. This is the kind of question that very well could be suitable as non-CW, depending on how people reply. At this point, the alternatives appear worse to me: close the thread as off-topic or selectively delete the more opinionated answers. BTW, I'm not going to pay any attention to what they do on Academia: that's irrelevant. $\endgroup$
    – whuber Mod
    Commented Feb 5, 2016 at 17:21
  • 1
    $\begingroup$ @gung I see. By the way I have just found out that it is not possible anymore to make questions CW when asking them (this has been changed some time ago). Currently only moderators can turn questions into CW. $\endgroup$
    – amoeba
    Commented Feb 5, 2016 at 17:24
  • $\begingroup$ I either marked it from the start or flagged it to be converted at the moment I posted it. $\endgroup$
    – gung - Reinstate Monica
    Commented Feb 5, 2016 at 17:26
  • $\begingroup$ @whuber: Frankly, I don't quite see how the decision about whether a question should be CW or not can depend on the answers. You mention closing as off-topic too; but surely being on-topic or off-topic is a property of the question, not the answers? I am confused. If there are bad (very opinionated) answers they should either be deleted or, in my opinion, rather downvoted. But the presence of bad answers should not take the reputation points away from the good answers. $\endgroup$
    – amoeba
    Commented Feb 5, 2016 at 21:46
  • 1
    $\begingroup$ In general I would agree, but there are a small number of questions whose nature is unclear until answers appear. Have you noticed that a crucial part of the definition of CW is in terms of the answers? Specifically, a thread definitely should become CW when it has multiple, clearly distinct, objectively valid answers. In this particular case I (perhaps over-generously) initially imagined that it could be answered in a unique, dispositive, best way. After two days and many answers it has become clear that will not be possible. $\endgroup$
    – whuber Mod
    Commented Feb 5, 2016 at 23:45
  • 1
    $\begingroup$ @whuber: I would encourage you to write an answer here elaborating on the policy you are using (I cannot find any guidelines on that! what is a definition of CW?). If the policy is a question that cannot be (or is not being?) answered "in a unique, dispositive, best way" then I don't quite understand why e.g. the question about Monte Carlo approximations of $e$ as well as the other examples that I gave in my question are not CW (just to be clear: I would be hugely disappointed if the result of this discussion is that they are going to become CW). Can you help me appreciate the difference? $\endgroup$
    – amoeba
    Commented Feb 5, 2016 at 23:59
  • $\begingroup$ @whuber: [cont.] I carefully read What are “Community Wiki” posts? FAQ on meta.SE but it does not say anything about when moderators should convert threads to CW. In a related post on meta.SE a moderator says that "there's a lot of different idealogies across the network about this". So what is ours? $\endgroup$
    – amoeba
    Commented Feb 6, 2016 at 0:02
  • $\begingroup$ You're right--the approximating $e$ question really needs to be CW. $\endgroup$
    – whuber Mod
    Commented Feb 6, 2016 at 0:02
  • 2
    $\begingroup$ The definition, criteria, and uses of CW have been evolving. Nevertheless, much of the previous dialog here on Meta is applicable. The best posts seem to be at meta.stats.stackexchange.com/questions/290, meta.stats.stackexchange.com/questions/409, meta.stats.stackexchange.com/questions/931, meta.stats.stackexchange.com/questions/1298, meta.stats.stackexchange.com/questions/1474, meta.stats.stackexchange.com/questions/1593, and meta.stats.stackexchange.com/questions/2300 (in chronological order). $\endgroup$
    – whuber Mod
    Commented Feb 6, 2016 at 0:08
  • 3
    $\begingroup$ @whuber: This is awful :-( That people stop getting reputation points after a thread is converted to CW looks like an enormous disadvantage to me and I cannot see what advantages could possibly outweigh it. What are the advantages, actually? I can understand CW status on a post about jokes or quotes (why should people get 100s of rep points for posting an xkcd cartoon), but e.g. in the case of approximating $e$ question we are talking about carefully crafted and elaborate answers. $\endgroup$
    – amoeba
    Commented Feb 6, 2016 at 0:12
  • 2
    $\begingroup$ I agree. I think it's worth discussing. $\endgroup$
    – whuber Mod
    Commented Feb 6, 2016 at 0:13
  • 2
    $\begingroup$ As the person who asked the "When (if ever) is a frequentist approach substantively better than a Bayesian?" question, I'll volunteer that I am torn. On the one hand, I appreciate that it is/was difficult to tease some answers apart from others on the basis of quality, both at the low and high-quality end of the spectrum. And when it came down to it, there were two answers that I felt were really good. But losing out on the rep for a 40+ up-voted question was a major bummer and I don't understand why my question was considered CW when the similar Bayesian-focused questions were not. $\endgroup$
    – jsakaluk
    Commented Feb 8, 2016 at 23:07
  • 1
    $\begingroup$ @jsakaluk: Thanks for joining in. In my experience, it is often very difficult to choose the best answer even for the most concrete and mundane non-CW questions imaginable. $\endgroup$
    – amoeba
    Commented Feb 8, 2016 at 23:18
  • 1
    $\begingroup$ Yea, it was especially difficult because there were a few really high-quality responses (I felt). But yeah, even though I acknowledge there is some subjectivity in terms of what constitutes a question that can be definitely answered, and questions that cannot, I feel as though my frequentist-focused one had very many (dis)confirmable answers, so I'm left unclear what the identifiable parameters for a non-CW-worthy question are. $\endgroup$
    – jsakaluk
    Commented Feb 8, 2016 at 23:21

2 Answers 2

Reset to default
9
$\begingroup$

[Posting an answer myself to enable people to vote.]

By now Approximate $e$ using Monte Carlo Simulation became CW as well. I think that making such questions CW is an unfair, unreasonable, and harmful policy.

  • This policy is unfair. If @Aksakal posted his (by now accepted) answer first, it could have been upvoted and accepted by the OP without many other answers appearing. Most likely (see comments above) the thread would not have been converted to CW in this case and @Aksakal would get his well-earned reputation. As he posted it after a flurry of other answers, the thread has become CW and he will not get his rep points. This is unfair.

    Also, the amount of reputation depends on when the thread will end up being converted to CW, because the points earned before that moment are not retracted. There are answers with 100+ upvotes in popular non-CW threads that should be CW according to the strict interpretation. Converting them now leaves this 1k+ rep points with the answerers. Other similar threads were converted to CW much earlier. Equally good answers did not earn any rep there. This is arbitrary and unfair too.

  • This policy is harmful. We have reputation system for a reason. Making the thread CW takes away large part of the incentive to put effort into providing new or into improving existing answers. This is a huge disadvantage.

  • This policy is unreasonable. There is no reason to turn this question into CW. There is no StackExchange policy that prescribes doing it. The whole CW concept is outdated and "largely deprecated": it is not possible since 2010 to post questions as CW; automatic conversion of answers to CW was discontinued in 2014; What are “Community Wiki” posts? meta.SE FAQ does not say anything to this effect; Putting the Community back in Wiki 2014 SE blog post only says that

    a question attracting very large numbers of partial answers can be a sign of a topic that wants to be a wiki

    with "very" being in italics in the original. The approximating-$e$ question has 5 answers; that is not a "very large number".

I think we are being pedantic in following an outdated and detrimental policy and should seriously re-think it. And we are not even following it consistently.

My suggestion is to use CW status only for "big list" questions like the ones about jokes or quotes -- the ones that are essentially off-topic and are only here due to their "historical importance". Note that the answers there usually do not require much effort and so it is not too bad if they do not count towards reputation. In contrast, new on-topic questions that invite substantial answers requiring effort to compose should not become CW.

Update to clarify

  1. Anybody who thinks that Approximate $e$ using Monte Carlo Simulation should be CW please explain why Why does the number of continuous uniform variables on (0,1) needed for their sum to exceed one have mean $e$? should not. [@Silverfish makes an attempt in the comments.]

  2. Anybody who thinks that How would you explain Markov Chain Monte Carlo (MCMC) to a layperson? should be CW please explain why How would you explain covariance to someone who understands only the mean? should not.

  3. Anybody who thinks that Is normality testing 'essentially useless'? should be CW please explain why PCA on correlation or covariance? should not.

  4. Anybody who thinks that When (if ever) is a frequentist approach substantively better than a Bayesian? should be CW please explain why Is there any reason to prefer the AIC or BIC over the other? should not.

  5. Anybody who thinks that What is the difference between data mining, statistics, machine learning and AI? should be CW please explain why What's the difference between probability and statistics? should not.

  6. Anybody who thinks that Python as a statistics workbench should be CW please explain why Excel as a statistics workbench should not.

The border to me appears so vague and so arbitrary that the only consistent solution that I see is to remove it. Substantial & answerable questions should not be CW. My suggestion is that all the above listed questions should not be closed and not be CW.

In addition, CW status is obviously not being used consistently (most glaring are examples 2 and 6), which creates massive unfairness. It is not some peanuts of reputation that we are talking about. These are often threads with 100+ upvoted answers, i.e. we are talking about ~1k reputation being given or taken. The only way to make the situation fair is to remove the CW status (it does not work the other way round).

Having said that, I can see some argument for making the Frequentist-vs-Bayesian question CW because of its holy war topic. I disagree with this argument but I can see it. If this is the community's standpoint then I propose an intermediate solution that at least all non-holy-war questions listed above should not be CW.

(I would encourage anybody who posts an answer to defend the current CW practice to comment on all the examples above.)

$\endgroup$
27
  • 4
    $\begingroup$ This is a well-written and well-constructed argument, I downvoted only because I happen to disagree with it. I know it can be disheartening to "miss out" on reputation for CW posts (it happens to me too) but it doesn't stop me posting on CW posts either - there is more to incentives than the rep system on here, although that is a powerful tool. However, I don't think rep is appropriate on opinion-driven threads, because they tend to have opinion-driven upvotes. I think that serves to stop people "gaming" such threads, and so the alternatives are arguably worse. $\endgroup$
    – Silverfish
    Commented Feb 6, 2016 at 21:38
  • 1
    $\begingroup$ @Silverfish: I can see your point about "opinion-driven" threads (and I encourage you to post an answer elaborating on it). But what makes the approximate-$e$ question opinion-driven?! $\endgroup$
    – amoeba
    Commented Feb 6, 2016 at 21:40
  • 1
    $\begingroup$ I do think there is a benefit to CW, which goes to the most important people on our site - the audience. CW threads are clearly marked as being different to our normal ones, and for good reason. A large part of the point of ticks and upvotes was to serve as communication to our readers of what (hopefully) educated and informed folk thought the "best" answer, and let these "float to the top". We mark out on CW threads that here we find rankings subjective and the order should be taken with a much greater pinch of salt than usual. $\endgroup$
    – Silverfish
    Commented Feb 6, 2016 at 21:42
  • 1
    $\begingroup$ Roughly speaking, I felt the e-simulation Q invited different approaches to reach different end-points, which it would be hard to describe as objectively better/worse. Most probability questions, by contrast, could have answerers taking quite different approaches but to reach a common end-goal: there's clearly some subjectivity in deciding if a certain approach was "neater" or "avoided unnecessary algebra" or "showed great insight into a wider range of problems" but those are still useful criteria that answers could be ranked on. $\endgroup$
    – Silverfish
    Commented Feb 6, 2016 at 22:17
  • 3
    $\begingroup$ It's my understanding that CW's used to keep open questions that aren't a good fit for our site's Q&A format - "every answer is equally valid" - , can't be made to fit, & therefore ought really to be closed; but which we like for some reason & want to keep open despite our rules. So not necessarily just "rather opinion-based". Are you proposing we should be stricter or laxer about closing? Or decide one way or the other without resorting to this CW business? $\endgroup$
    – Scortchi Mod
    Commented Feb 7, 2016 at 20:15
  • 1
    $\begingroup$ I agree with you about the approximate-$\mathrm{e}$ question; probably because, though many answers are possible, there are clear criteria for what an answer should be. I'm less happy with saying When (if ever) is a frequentist approach substantively better than a Bayesian? is fine because it's hard to see the difference from "Bayesian vs frequentist methods - what do you think?". On the other hand it doesn't seem right to close such a popular question - especially as after several opinionated answers other people ... $\endgroup$
    – Scortchi Mod
    Commented Feb 8, 2016 at 11:05
  • 3
    $\begingroup$ ... might well think it unfair that they shouldn't be allowed to add their own opinions. So CW is a kind of compromise: a signal that it's all right to get on your soap box & ride your hobby horses here, but it's not really what the site's for. $\endgroup$
    – Scortchi Mod
    Commented Feb 8, 2016 at 11:07
  • 1
    $\begingroup$ I lean towards thinking neither of those need be CW - that is I can't see the benefit, though perhaps I'm over-looking something. $\endgroup$
    – Scortchi Mod
    Commented Feb 8, 2016 at 11:39
  • 1
    $\begingroup$ Agree. CW should be used only for old popular off-topic questions. Nowadays, big lists, book recommendation, asking for data, subjective questions should be closed; including that one first cited in the question. $\endgroup$
    – Andre Silva
    Commented Feb 8, 2016 at 17:06
  • 1
    $\begingroup$ So I'm obviously biased because I wrote the Frequentist question, but I have a hard time understanding why you would want to close a question that has generated 44 upvotes, >3000 views, and 11 thoughtful answers in less than 5 days. I'm new to the meta aspect of the site, so such a question could technically end up being "inappropriate" for CV, but I would seriously question the motivation underlying a mandate for a stats Q&A site to shut down a question generating such broad interest. I legitimately learned several things--some which seem not a simple matter of opinion. Isn't that the point? $\endgroup$
    – jsakaluk
    Commented Feb 9, 2016 at 4:05
  • 2
    $\begingroup$ My inquiry here is clearly motivated by my question being under discussion, but I mean that for other questions that might fall under the purview of this discussion too. If people are asking thoughtful questions, the community is providing thoughtful answers, and people are learning, isn't that kind of the reason for this site existing? Sometimes there isn't a clear-cut unambiguous answer. But as long as people are providing references for the positions they espouse--and maybe that's' what we need to encourage more--then it's on the reader to decide what evidence is more convincing for them. $\endgroup$
    – jsakaluk
    Commented Feb 9, 2016 at 4:10
  • 5
    $\begingroup$ @jsakaluk: But mightn't it be that among the reasons CV's become what it is - a place where you can ask a question like that & get thoughtful answers rather than ill-informed rants - is that we have rather strict rules on what sort of questions we deem appropriate for the site? Whenever the success of the Stack Overflow model is discussed, the contribution of insisting on specific, objectively answerable questions is emphasized. $\endgroup$
    – Scortchi Mod
    Commented Feb 9, 2016 at 17:08
  • 2
    $\begingroup$ @Scortchi: Sure--I'm not proposing abandoning all rule and order on CV. But if the intent of CW-related-rules is to prevent thoughtless, ill-informed rants/questions/answers, then I submit that the technical details of the rules need to be reviewed when posts containing thoughtful questions/answers are being mistakenly and/or inconsistently categorized as CW. And while I am wholly supportive of the need for specific, objectively answerable questions, I'm not onboard with a definition of "objectively answerable" that leaves no room for different interpretations of empirical evidence. $\endgroup$
    – jsakaluk
    Commented Feb 9, 2016 at 17:19
  • 1
    $\begingroup$ I can't see why "how do we prove X" or "how do we get this result" or "how do we solve this question" should be CW, if they were CW then virtually ever self-study Q here would be CW and that's clearly not what CW is designed for. For me, that's quite distinct from "what's a cool example of X", "where can I see phenomenon X in real life", "what's a surprising fact about X" which would all be strong candidates for CW (though far from the only kind of CW question). $\endgroup$
    – Silverfish
    Commented Feb 10, 2016 at 12:54
  • 2
    $\begingroup$ Apologies for the verbosity. For the TLDR, perhaps if we are rating answers by their "coolness" or measuring how well they enter the "spirit of the question" then there's a case for considering CW ... think that roughly expresses my intent $\endgroup$
    – Silverfish
    Commented Feb 10, 2016 at 13:10
-1
$\begingroup$

I am posting a possible solution that doesn't address all of the points but may be a solution to mitigating the effects of losing reputation score for having a question become CW.

Would it be possible for the moderators to let those whose reputation would be affected by having the original post become CW know beforehand so that they may decide to use those points as a possible bounty to highlight some other question of interest to them? I am not sure why you are not allowed to keep the reputation in the first place, but this the reputation would still be taken away and at least used in some "productive" fashion by that user.

I pose this solution knowing that my own post: What is a data scientist? could potentially go CW and now I wonder should I chance losing all that reputation and so instead place a large bounty, or should I roll the dice and gamble on the fact that it may not go CW. So I think a heads up (moderators if you are listening) would be nice :)

$\endgroup$
4
  • 7
    $\begingroup$ There is some confusion here. Your question about data scientists has currently 51 upvotes and is a very likely candidate to be converted to CW (in fact, I am surprised that it did not happen yet; I would of course prefer that it remains non-CW, see my answer). If this happens, then your 51*5=255 reputation points will not be taken away from you. But you will stop receiving any reputation points for any further upvotes. $\endgroup$
    – amoeba
    Commented Feb 12, 2016 at 17:15
  • $\begingroup$ @amoeba Awwwwwwww that makes perfect sense. I thought everything was revoked. In that case, should I delete this answer? $\endgroup$
    – RustyStatistician
    Commented Feb 12, 2016 at 17:16
  • 1
    $\begingroup$ I think you should keep this answer up. Regardless of whether all total, or all future rep is lost, I agree with you; if nothing else, a heads-up to the effect of, "so, by the way, your post is going to be turned into CW...", would be nice. Or perhaps even better: could moderators provide a comment on what would generally need to change in a post for it to be kept non-CW? $\endgroup$
    – jsakaluk
    Commented Feb 13, 2016 at 6:07
  • 3
    $\begingroup$ The spirit is right but you could make a general case that all actions by moderators should be directly flagged to the individuals most concerned. Even if this were automated, I am unhappy about the principle and unclear where you should draw the line. Do/should spammers, posters of rude or intemperate comments, posters of questions put on hold, etc. get direct notification? While individuals' rights and sensitivity deserve complete respect, ultimately all moderators' actions are on behalf of the community. (For myself, I really don't mind any loss of reputation contributing to CW threads.) $\endgroup$
    – Nick Cox
    Commented Feb 13, 2016 at 11:34

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .