20
$\begingroup$

There has been a lot of discussion about software-related questions on CV (see here or here for examples...). And yet there is still no clear idea and guidance on what CV is, what it strives to be and who it is for.

I've been a user of and contributor to CV for a long time (intially during the beta phase) and I find it unclear what is and isn't allowed here. But more so I find the creep towards a more statistical per se remit for the site, to wit, @Chl recently said

It also worth noting that our site policy has evolved since its inception, and we now try to focus on questions that require statistical expertise rather than questions asking about tools, or polls involving shopping-list or extended discussion.

Which alarms me immensely (well, not bit about the polls, discussion of shopping lists). That is not what I signed up to with CV originally, and not what I have been contributing to. I also feel that such a move would be to the detriment of CV as a whole. I don't mean to single out @Chl here, but this is as best an admission I could find of what I have grown to believe has been happening here over the past year or so.

Now, I'll admit to not following all the meta discussions (here or on Stack Overflow where I am somewhat more active) so I may not have been privy to some of the discussions regarding this shift, but at the very least there seems to be disquiet or a lack of clarity about what CV is and is not.

In a recent thread on a similar topic @NickCox quoted the What topics can I ask about here? page in support of his argument that many R-related questions are off topic here. I quote the relevant section again.

There are certain subjects that will probably get better responses on our sister sites. If your question is about

  • Programming, ask on Stack Overflow. If the language is statistically oriented (such as R, SAS, Stata, SPSS, etc.), then decide based on the nature of your question: if it needs statistical expertise to understand or answer, ask it here; if it's about an algorithm, routine data processing, or details of the language, then please refer to the collection of links to resources we maintain.

The problem with this "evidence" is that it refers explicitly to "Programming". Just because R is a programming language, it doesn't mean that using R is an act of programming. I would wager that the majority of R Users are not involved in any programming (as most people would recognise the term) but are using it as they would Minitab, Excel, SAS, SPSS etc to run statistical analyses.

If the above is now representative of a broader CV policy, the advice above needs to be updated to removal the Programming qualifier.

So my question or perhaps point really, is that the current policy regarding what is on or off topic here needs clarification. But I would like to stimulate a wider debate about what CV aims to be. As I expressed elsewhere, a site devoid of software-related questions is not something I want to see for CV but many users seem to think that anything with "...do this in R?" is OT here.

I reject the argument that if a question is ostensibly related to or about R that it is OT for Cross Validated. If that question is motivated from primarily a statistical point of view then I don't care if it has an R or SAS or SPSS focus. Such questions should be on-topic here and I would like to encourage that CV embrace such questions.

For example, this question, though it is not the best example it is pertinent as the comment by @BabakP about it being OT precipitated my particular question, would (or should) be a useful and relevant contribution here.

P.S. Throughout I have referred to R but you could replace "R" in the above question with Your Favourite Stats Software and my point would remain the same.

$\endgroup$
  • $\begingroup$ Apologies; it is now late where I am & I post the above just as I am off to bed. I will respond to any questions/comments etc in (hopefully, kids allowing) 7 or 8 hours! $\endgroup$ – Reinstate Monica - G. Simpson Sep 17 '13 at 5:07
  • 9
    $\begingroup$ I agree with you. But it seems that we are a minority. Right now, all R or SAS or SPSS etc questions (however tangentially related to those as languages) are being sent to CV. Data analysis is done by packages like this; not all statistics is theory. $\endgroup$ – Peter Flom - Reinstate Monica Sep 17 '13 at 9:03
  • 4
    $\begingroup$ To borrow from a recent post gung made, the basic logic re: this type of OT for me is to ask myself: Can this question be reasonably answered by saying something analogous to "type ______"? If the answer is yes, I close vote. If not, I don't. I strongly disagree with any suggestion that all questions that relate to software, no matter how tangentially are being sent to SO (this is how I parsed Peter's comment, assuming 'CV' was a typo. I could be wrong). Look no further than our most popular tag: R. $\endgroup$ – Macro Sep 17 '13 at 18:07
  • $\begingroup$ By way of follow-up, I have posted a new Meta question to curate clearer or more explicit house rules on what is on/off topic for CV. This text could replace the user-visible What topics can I ask about here? page in the Help Centre. $\endgroup$ – Reinstate Monica - G. Simpson Sep 18 '13 at 4:47
  • 1
    $\begingroup$ One worthwhile thing to note is the kinds of questions that SO fields and those it migrates here. What we see migrated to here is almost always far more statistical than what tends to be migrated to there from here (with almost no overlap), and the kind of question we'd be debating over being on topic here is - if posted there instead, almost always simply answered there. That is, in practice SO appears on average to be drawing its line about what's on topic there (at least for things initially posted there) quite a bit further into the statistical side than we're discussing here. $\endgroup$ – Glen_b -Reinstate Monica Sep 19 '13 at 23:51
  • $\begingroup$ possible duplicate of What is on topic on Cross Validated? $\endgroup$ – russellpierce May 20 '14 at 16:20
14
$\begingroup$

I don't see any real disagreements here. The key is in the last line: if your interest in any question were to change radically upon replacing R by BMDP or Systat or whatever, then that is likely because the question is not asking anything about data analysis or statistics: it is asking about how to say the magic words to make the software behave. That's what SO is for.

Of course that does not rule out fielding software-related questions here on CV. As Peter Flom points out in a comment, software is a necessary medium for doing data analysis. (I have argued elsewhere that it's not a good medium for conveying ideas about data analysis, though, because there is no software platform that a majority of people doing data analysis are familiar with.) We have always recognized here that many ostensibly software-related questions are, when closely examined, really questions about analyzing data, even though they manifest themselves as software-related problems. We want to answer the deeper question about how to analyze the data rather than the immediate, shallow question about what to change in the code. This, to me, is one of the keys to writing great questions and providing great answers: a great question provides enough information to identify the root cause of an issue and an awesome answer provides a revelation about that cause, increasing our understanding regardless of what software platform we may choose to use ourselves.

I don't think anyone anywhere here has claimed that "using R is an act of programming." That's why programming is specifically mentioned in our FAQ. When you are in fact using R only for programming, you are employing it in its capacity to do a job that C++ or Python or VBA or whatever could do, and you should prefer to address your question to a community of programming experts. That ain't us: that's SO. When you are using R to carry out statistical analysis or data visualization, your questions about that are squarely on topic here and always have been.

Recently we have been migrating an average of 2 questions per day (out of 50 per day total) to SO. They are not all R-related: in the list I see a JMP question, a Stata question, and a Java question in the first ten. What are the R-related questions about?

  • How to compute a transition matrix from time series data. I disagree with this migration: this question invites answers that explain how to estimate Markov transition matrices and then illustrate the procedure with R.

  • How to reproduce a paper's results. Although it could be handled in a way that is informative, there seems little prospect that such an approach would appeal to the OP: he just wants the R equivalents of Stata commands.

  • This question concerns a package installation problem. That should be of no interest on this site: send it to those who love to diagnose such software problems.

  • Clustering time series by correlation. This vague question really needed guidance from our community and IMHO is squarely on topic here. I disagree with its migration. However, the OP did get a useful answer on SO and her comments there suggest that maybe our community was right in its decision: the discussion is now focusing on details of R syntax and not the deeper issue of why this form of analysis is being attempted or how best to do it. But maybe that's a self-fulfilling result: when a question stays here we discuss the data analysis implications and when it resides on SO, that community naturally addresses syntax, performance, and other computing issues.

I could go on analyzing the migrations, but already some generalities seem to be emerging:

  • If we adopt the approach of addressing every software-specific question as being (potentially) a question about data analysis, in almost all cases we will be right and the question need not be migrated (although it might need considerable improvement to deflect its original focus from R to the application of R).

  • If we pause to write a comment asking the OP about the nature of their question, we can be confident that we are directing them towards the community best suited to help them.

  • We need to be a little slower to close questions. When I review the statistics, I find that many of our community members who are voting to close are spending remarkably little time to make their decisions and some of them are voting far too often for closure.

In brief, when you are in doubt about a question, don't migrate it, improve it!

$\endgroup$
  • 3
    $\begingroup$ Apart from that, I agree with most if not all of the remainder of your answer. That I was motivated to post my semi-rant is evidence that this common sense approach is not being followed. Your analysis of recent migrations bears this out too. There is something wrong with the site's advice on what is on-topic if several prominent CV people can use it to argue their own, opposite sides in discussions about what is & is not on topic. Also, there doesn't seem to be consistency in this regard from our moderators. $\endgroup$ – Reinstate Monica - G. Simpson Sep 17 '13 at 16:20
  • 3
    $\begingroup$ Finally, I think we need to send a very clear signal that comments along the lines of "this is about R, this if OT, post on Stack Overflow" are not welcome as the sole, substantive point of the comment. Pointing out how to improve the Q should be at the very least required by the commenter. In general though we (the CV community) just need to be far more consistent & stop requiring that every question be software agnostic or supported by statistical theory in order for it to be on topic. $\endgroup$ – Reinstate Monica - G. Simpson Sep 17 '13 at 16:24
  • 3
    $\begingroup$ The first one should maybe just have been marked as a duplicate. Here's one, but I'm virtually certain there are others as well. Especially for questions like this, the active community could probably do more to help locate duplicates. $\endgroup$ – cardinal Sep 17 '13 at 16:35
  • 2
    $\begingroup$ Gavin, I think you have gone so far in stretching a point that you exaggerate my response beyond recognition. I have never advocated sending "trash" to SO nor have I ever knowingly done such an act. I have strongly agreed that using R is not necessarily "programming," but I am still advocating a more nuanced characterization that allows for some questions about R to be more appropriate for SO than they are here and others to be more appropriate here. It's never a matter of whether I want a question here, but on which site is the better fit (for the OP and the SO & CV communities). $\endgroup$ – whuber Sep 17 '13 at 17:41
  • 4
    $\begingroup$ I detect no inconsistency among moderators, BTW, although occasionally we have differences of opinion--and each one of those helps us refine our subsequent actions. Your main point seems to be that you feel confused by our statements about what this site is about. I would like to invite you, then, to turn from ranting--which is useful and appreciated but will soon run its natural course--and instead suggest constructive changes that will clarify for you and others what is considered on topic here. What should our FAQ say? $\endgroup$ – whuber Sep 17 '13 at 17:43
  • 3
    $\begingroup$ @Gavin I appreciate your engaging with this in such a considerate and thoughtful manner. My belief is that almost any approach you use that (a) gets the ball rolling with a specific suggestion and (b) allows for community discussion is going to be fine. A Meta post would work. I will plead ignorance concerning the concerns within the SO community and therefore wish to thank you in advance for anything you can write that will help me--and perhaps many others here--understand what the issues are on that side, so that we can evolve our policy in a way that respects the needs of CV and SO. $\endgroup$ – whuber Sep 17 '13 at 18:23
  • 4
    $\begingroup$ (+1) After having read this, I'm afraid of being guilty of often migrating software-related questions too quickly. I will definitely pay more attention in the future and try to improve the question rather than migrating it immediately. $\endgroup$ – COOLSerdash Sep 17 '13 at 18:31
  • 5
    $\begingroup$ And no, I am not attempting to impugn honest differences. I just happen to believe that some voters may be rather too eager to close many questions that the community and its moderators find acceptable. It's rather a matter of suggesting that we all need to calibrate our voting and take a little time with our considerations. To all: please consider the (oft neglected) options of editing to improve questions and posting comments encouraging the OPs to improve them. $\endgroup$ – whuber Sep 17 '13 at 20:45
  • 5
    $\begingroup$ @whuber, I don't know who you're talking about but suspect I may be implicated. I can think of several instances where I've been on the "close" side of a split vote and am not embarrassed to own up to it. The simple fact is: I think this site has a lot of low-quality questions that should be closed/on hold. When I was more active on the site, I tried as much as anyone to encourage the OP to improve the post rather than voting to close but grew discouraged after other high rep users would undermine this effort either by answering the low-quality post or giving it their "OK" in the review queue $\endgroup$ – Macro Sep 17 '13 at 22:16
  • 3
    $\begingroup$ (cont.) as a result, when I do review, I tend to err on the side of closure because I think answering "bad questions" is should be discouraged and putting them on hold is one way to make that happen. During the "hold time" the issues can be sorted out (e.g. by questioning the OP, etc.). BTW, I'm curious how you could "time" the reviews. I often close vote on questions that are not in the review queue, or see a question, think about voting to close, then see it later in the review queue. If your calculation accounts for this then moderators have way more power than I thought :-) $\endgroup$ – Macro Sep 17 '13 at 22:20
  • 8
    $\begingroup$ @Gavin, if you see the purpose of the site as creating a repository of good questions and answers that will be useful to future visitors (and not a site where you serve the OP as though they are a customer) then maybe you can see why I think this behavior defeats the purpose of the site (and why I'm on record railing against it). The main thing is that "bad" questions are often unclear. An exchange involving an unclear question and someone's random guess at what the question was asking is unlikely to be useful to future readers. What's the rush? Why not clarify the question before answering? $\endgroup$ – Macro Sep 17 '13 at 22:26
  • 2
    $\begingroup$ To get back to the original issue, here is a good example of what I think belongs to SO: stats.stackexchange.com/questions/70315/… Although the context of the question was statistical, the question itself is not. I have no strong feelings one way or the other, but from what I can tell questions like these get better answers at SO. On the other hand, questions asking for the usage of statistical software I find on topic, see Gael's posts. $\endgroup$ – Momo Sep 18 '13 at 8:02
  • 2
    $\begingroup$ @gavin, I think we're talking passed each other here so I'm going to stop. I'll just point out a) that putting a question on hold is temporary and does not preclude anything you're saying, b) II never vote to close a question that doesn't clearly go against site policy, and c) pondering how something can be "against SE ethos regardless of what other community members think" makes my head hurt. Also, please call me Macro. I have no idea who Marco is. $\endgroup$ – Macro Sep 18 '13 at 12:08
  • 4
    $\begingroup$ @GavinSimpson, I want to 2nd Macro's points here. In several comments it seems to me you believe putting a question [on-hold] prevents the OP from clarifying their Q & users from commenting to get more info. If so, these are not true. Users can comment & OPs can edit their Q after having been put [on-hold]. My impression is that OPs are more likely to edit & provide necessary info when their Q was put [on-hold] than when only a comment requesting such info was made. Moreover, the change from [closed] to [on-hold] was intended to encourage this & make it seem friendlier. $\endgroup$ – gung - Reinstate Monica Sep 19 '13 at 16:29
  • 3
    $\begingroup$ @Gavin, I'd like to co-sign whuber's most recent comment but I should clarify: No, I don't prefer to use closure to stop bad answers. After making countless good faith attempts to remedy behavior that I perceive as - to use your words - "against the ethos of SE" by using comments, I see close voting as my only recourse. This is not specific to software related questions; it also refers to users answering blatant duplicates of questions they themselves have answered before (possibly multiple times) and answering questions where no one, including the answerer, knows exactly what's being asked. $\endgroup$ – Macro Sep 19 '13 at 21:24
11
$\begingroup$

It seems to me that the line between on-topic and off-topic has drifted a little bit lately, and it'd be nice if we could (collectively) re-establish it.

I wanted to second @whuber's point that most software questions that might be off-topic have underlying data analysis questions with two examples:

  1. I once asked about why the distance values differed between SPSS's and R's dendrograms. It turned out to be something fairly mundane (an artefact from when SPSS drew ASCII diagrams), but at the time I was wondering whether the rescaling had some potentially-useful statistical interpretation.

  2. I also recall a question about adjusting the priors on a Naive Bayes classifier in R. This could be answered in several different ways. The "StackOverflow-y" answer is probably either

    • "Add these arguments to the function call, like so...."
    • "You can't in $X$; use package $Y$ instead."

    However, armed with (a very little bit of) statistical knowledge, one could suggest multiplying the posterior probabilities of each class $P(\textrm{Class}=C|\textrm{Data})$ by $\frac{P_{new}(C)}{P_{old}(C)}$ to manually update the priors instead.

While I don't want to wade through tons of "how do I import my Excel file into R?" or "why does my NumPy matrix look weird?", I'd personally like the line to be drawn such that these sorts of questions are firmly on the on-topic side.

Even some of the "How can I do this, but in another language" questions may have merit, as some techniques have far too many names (there are probably about a dozen ways of saying "PCA"), and sometimes a term is overloaded to mean multiple things.

Maybe the rule of thumb should be something like "How often would this happen in a non-statistical context?" Everybody has programming problems with things like I/O and flow control, but I would venture to guess that there are very few off-topic situations where someone would care about potential differences between Matlab's eig function and Octave's qz

$\endgroup$
  • 2
    $\begingroup$ +1 (but only now having a chance to comment). I tried to work @whuber's point into the proposal I posted in the other Meta thread. I could add another example from your Answer here too. Thanks. $\endgroup$ – Reinstate Monica - G. Simpson Sep 19 '13 at 15:04
11
$\begingroup$

I don't really know where to post this but there is an additional point that I feel should be made. When looking at questions like

You need quite a lot of statistics knowledge to even simply understand the question. The answer might ultimately be as simple as “type XXX” but finding the right function, being able to understand the relevant documentation and to cut through inconsistent and confusing terminology, or to understand what parameters you need to use and check that the function is really doing what you want is far from trivial.

The wording of the questions might emphasize the “how-do-I-carry-out-test-X” aspect and ostensibly suggest that the relevant statistical decisions have already been made but they still have no place on StackOverflow. Those are simply not questions a programmer could answer.

Obviously, we might still want to close them for other reasons but we should at least agree that the only site where they could possibly be on-topic and stand a good chance of reaching knowledgeable contributors is CrossValidated.

$\endgroup$
  • 7
    $\begingroup$ +1: "Not questions a programmer could [be expected to] answer" sounds like a useful criterion to help us decide about migrations. $\endgroup$ – whuber Sep 18 '13 at 13:29
  • 2
    $\begingroup$ There's a lot of good ideas here. However, I've heard versions of this argument before & I don't really find it compelling. Take a look at some of the people who are answering R questions on SO; I don't think they'd be incapable of understanding the statistical content of these questions & answering appropriately. Moreover, although R is a programming language, other languages are typically more appropriate for most programming tasks. The reason people use R is for statistics. Anyone who knows enough R to answer Q's on SO probably knows stats. $\endgroup$ – gung - Reinstate Monica Sep 19 '13 at 13:09
  • 3
    $\begingroup$ @gung A few of these users are clearly experts in statistics and could certainly understand the questions I listed but it's because they are statisticians or users of statistics, not programmers. Some SO users could perhaps also answer most questions asked here on CV if they chose to but that's not a good reason to ask statistics question there. $\endgroup$ – Gala Sep 19 '13 at 14:18
  • 2
    $\begingroup$ You're right. I don't suggest people should ask statistics questions there, but I do think it's fine to ask R programming questions there that require "statistical reasoning" to understand. $\endgroup$ – gung - Reinstate Monica Sep 19 '13 at 14:20
  • 1
    $\begingroup$ For the type of questions I have in mind, there is no algorithm to speak of, no branching, no loops, just a function call. In SPSS the solution could also be to use a specific menu item or change some parameters in a dialog box. In this case, nobody would suggest it's appropriate to ask on SO. But it seems that because R's user interface is primarily text-based, it somehow becomes a “programming” question. As someone who studied computer science before becoming interested in statistics, I still don't see it. $\endgroup$ – Gala Sep 19 '13 at 14:23
  • $\begingroup$ OK, but then what's the point of moving such questions to a website intended for programmers since answers only require statistics expertise? $\endgroup$ – Gala Sep 19 '13 at 14:28
  • 2
    $\begingroup$ @gung Perhaps if this were phrased "Not questions the average programmer could [be expected to] answer", or something to that effect, would that be more suitable? I agree that there are many people on Stack Overflow who could answer statistical questions. I see this point as more about making the person asking the question think about the best venue for that question. $\endgroup$ – Reinstate Monica - G. Simpson Sep 19 '13 at 15:03
  • $\begingroup$ What would you think of amending that to "Not questions the average R programmer could [be expected to] answer", @GavinSimpson? (Are there R-based statistical questions R programmers can't understand?) There is a fundamental ambiguity here which is part & parcel to the trouble we're having defining where the line should be drawn. $\endgroup$ – gung - Reinstate Monica Sep 19 '13 at 15:09
  • $\begingroup$ @gung If you question is "Are there any R-based statistical questions some R programmers can't understand?" the answer to that is clearly yes (I would be a good case in point). (I would s\understand\answer\ tho.) I'm not sure why this is being pulled back to R and not stats software in general? I take your point that we don't want to suggest Stack Overflow users don't know statistical or aren't capable of answering statistical question. My suggestion was to clarify that if you don't think the average programmer would be able to answer your question Cross Validated is probably the better venue. $\endgroup$ – Reinstate Monica - G. Simpson Sep 19 '13 at 15:23
  • 1
    $\begingroup$ By "R", I mean any such statistical software (eg, Stata, SAS, etc) @GavinSimpson. I am not singling out R for discrimination. I am very fond of R, & use it daily as my software of choice. I am familiar w/ SAS & SPSS, but have mixed feelings about them; I've never used Stata. My point about your version of the quote is that the "average programmer" probably develops software in C++ for commercial clients w/o any knowledge of even what a t-test is. But if you think of the average R <Stata, possibly SAS, etc> programmer understands about stats it's a whole different ball game. $\endgroup$ – gung - Reinstate Monica Sep 19 '13 at 15:36
  • 1
    $\begingroup$ @GaëlLaurans, you are raising a good point. I am becoming much clearer, as a result of the ongoing discussion, that just because a question is only about how to use statistical software doesn't mean it's a programming question, & that such questions are off-topic on SO under a fairly strict definition of "programming" (that essentially mirrors my somewhat strict definition of "stats"). The result of this is that such questions would be off-topic for both CV & SO. Those questions are essentially tech support for software & probably belong on the r-help listserve (statalist, etc). $\endgroup$ – gung - Reinstate Monica Sep 19 '13 at 15:41
  • 1
    $\begingroup$ @gung Good point, but Stack Overflow primarily is filled with C, C++, JAVA, etc programmers - the R tag is relative small there. Hence my point was that Cross Validated would probably be a better place if the average programmer might not understand or be able to answer the Q. There may well be R programmers on Stack Overflow who could answer, but when deciding on where to post, Cross Validated may be better as you might get different eyes on the problem that you wouldn't get if you just focus on the R programming side of one's question. $\endgroup$ – Reinstate Monica - G. Simpson Sep 19 '13 at 17:10
  • 1
    $\begingroup$ Do the C++ programmers on SO answer the R questions, @GavinSimpson? When I'm on SO (which is relatively little), my impression is that SO is an archipelago of relatively isolated islands. Ie, the C++ programmers never answer the R questions & R users (or at least I) never read the C++ Qs. $\endgroup$ – gung - Reinstate Monica Sep 19 '13 at 17:15
  • 2
    $\begingroup$ Just to throw in some numbers: The easiest things to count are tags. For R, CV tags are 4556, SO are 37354; for SAS, CV tags are 234, SO are 1550; for Stata, CV tags are 283 and SO tags are 466; for SPSS, CV tags are 557 and SO tags are 296. Clearly these numbers say nothing about (0) the detail of any posts (1) which site was more appropriate for posts (2) fraction of any or good answers (3) anything else, but the variation in ratio is quite striking. $\endgroup$ – Nick Cox Sep 19 '13 at 17:43
  • 2
    $\begingroup$ @gung Well, I think we could, to some extent, accommodate these usage questions here but I understand I am probably more lenient than most in this respect. In any case, some are indeed essentially tech support questions (didn't think of that term before but that's really what it is). We should decide for ourselves whether we want to close, welcome or improve them but not believe they should be asked on SO merely because software or code is involved. $\endgroup$ – Gala Sep 19 '13 at 18:00

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .