What does Cross Validated aim to be, who is it for?

Question

There has been a lot of discussion about software-related questions on CV (see here or here for examples...). And yet there is still no clear idea and guidance on what CV is, what it strives to be and who it is for.

I've been a user of and contributor to CV for a long time (intially during the beta phase) and I find it unclear what is and isn't allowed here. But more so I find the creep towards a more statistical per se remit for the site, to wit, @Chl recently said

It also worth noting that our site policy has evolved since its inception, and we now try to focus on questions that require statistical expertise rather than questions asking about tools, or polls involving shopping-list or extended discussion.

Which alarms me immensely (well, not bit about the polls, discussion of shopping lists). That is not what I signed up to with CV originally, and not what I have been contributing to. I also feel that such a move would be to the detriment of CV as a whole. I don't mean to single out @Chl here, but this is as best an admission I could find of what I have grown to believe has been happening here over the past year or so.

Now, I'll admit to not following all the meta discussions (here or on Stack Overflow where I am somewhat more active) so I may not have been privy to some of the discussions regarding this shift, but at the very least there seems to be disquiet or a lack of clarity about what CV is and is not.

In a recent thread on a similar topic @NickCox quoted the What topics can I ask about here? page in support of his argument that many R-related questions are off topic here. I quote the relevant section again.

There are certain subjects that will probably get better responses on our sister sites. If your question is about

• Programming, ask on Stack Overflow. If the language is statistically oriented (such as R, SAS, Stata, SPSS, etc.), then decide based on the nature of your question: if it needs statistical expertise to understand or answer, ask it here; if it's about an algorithm, routine data processing, or details of the language, then please refer to the collection of links to resources we maintain.

The problem with this "evidence" is that it refers explicitly to "Programming". Just because R is a programming language, it doesn't mean that using R is an act of programming. I would wager that the majority of R Users are not involved in any programming (as most people would recognise the term) but are using it as they would Minitab, Excel, SAS, SPSS etc to run statistical analyses.

If the above is now representative of a broader CV policy, the advice above needs to be updated to removal the Programming qualifier.

So my question or perhaps point really, is that the current policy regarding what is on or off topic here needs clarification. But I would like to stimulate a wider debate about what CV aims to be. As I expressed elsewhere, a site devoid of software-related questions is not something I want to see for CV but many users seem to think that anything with "...do this in R?" is OT here.

I reject the argument that if a question is ostensibly related to or about R that it is OT for Cross Validated. If that question is motivated from primarily a statistical point of view then I don't care if it has an R or SAS or SPSS focus. Such questions should be on-topic here and I would like to encourage that CV embrace such questions.

For example, this question, though it is not the best example it is pertinent as the comment by @BabakP about it being OT precipitated my particular question, would (or should) be a useful and relevant contribution here.

P.S. Throughout I have referred to R but you could replace "R" in the above question with Your Favourite Stats Software and my point would remain the same.

Answer 1

I don't see any real disagreements here. The key is in the last line: if your interest in any question were to change radically upon replacing R by BMDP or Systat or whatever, then that is likely because the question is not asking anything about data analysis or statistics: it is asking about how to say the magic words to make the software behave. That's what SO is for.

Of course that does not rule out fielding software-related questions here on CV. As Peter Flom points out in a comment, software is a necessary medium for doing data analysis. (I have argued elsewhere that it's not a good medium for conveying ideas about data analysis, though, because there is no software platform that a majority of people doing data analysis are familiar with.) We have always recognized here that many ostensibly software-related questions are, when closely examined, really questions about analyzing data, even though they manifest themselves as software-related problems. We want to answer the deeper question about how to analyze the data rather than the immediate, shallow question about what to change in the code. This, to me, is one of the keys to writing great questions and providing great answers: a great question provides enough information to identify the root cause of an issue and an awesome answer provides a revelation about that cause, increasing our understanding regardless of what software platform we may choose to use ourselves.

I don't think anyone anywhere here has claimed that "using R is an act of programming." That's why programming is specifically mentioned in our FAQ. When you are in fact using R only for programming, you are employing it in its capacity to do a job that C++ or Python or VBA or whatever could do, and you should prefer to address your question to a community of programming experts. That ain't us: that's SO. When you are using R to carry out statistical analysis or data visualization, your questions about that are squarely on topic here and always have been.

Recently we have been migrating an average of 2 questions per day (out of 50 per day total) to SO. They are not all R-related: in the list I see a JMP question, a Stata question, and a Java question in the first ten. What are the R-related questions about?

• How to compute a transition matrix from time series data. I disagree with this migration: this question invites answers that explain how to estimate Markov transition matrices and then illustrate the procedure with R.

• How to reproduce a paper's results. Although it could be handled in a way that is informative, there seems little prospect that such an approach would appeal to the OP: he just wants the R equivalents of Stata commands.

• This question concerns a package installation problem. That should be of no interest on this site: send it to those who love to diagnose such software problems.

• Clustering time series by correlation. This vague question really needed guidance from our community and IMHO is squarely on topic here. I disagree with its migration. However, the OP did get a useful answer on SO and her comments there suggest that maybe our community was right in its decision: the discussion is now focusing on details of R syntax and not the deeper issue of why this form of analysis is being attempted or how best to do it. But maybe that's a self-fulfilling result: when a question stays here we discuss the data analysis implications and when it resides on SO, that community naturally addresses syntax, performance, and other computing issues.

I could go on analyzing the migrations, but already some generalities seem to be emerging:

• If we adopt the approach of addressing every software-specific question as being (potentially) a question about data analysis, in almost all cases we will be right and the question need not be migrated (although it might need considerable improvement to deflect its original focus from R to the application of R).

• If we pause to write a comment asking the OP about the nature of their question, we can be confident that we are directing them towards the community best suited to help them.

• We need to be a little slower to close questions. When I review the statistics, I find that many of our community members who are voting to close are spending remarkably little time to make their decisions and some of them are voting far too often for closure.

In brief, when you are in doubt about a question, don't migrate it, improve it!

Answer 2

It seems to me that the line between on-topic and off-topic has drifted a little bit lately, and it'd be nice if we could (collectively) re-establish it.

I wanted to second @whuber's point that most software questions that might be off-topic have underlying data analysis questions with two examples:

1. I once asked about why the distance values differed between SPSS's and R's dendrograms. It turned out to be something fairly mundane (an artefact from when SPSS drew ASCII diagrams), but at the time I was wondering whether the rescaling had some potentially-useful statistical interpretation.

2. I also recall a question about adjusting the priors on a Naive Bayes classifier in R. This could be answered in several different ways. The "StackOverflow-y" answer is probably either

   • "Add these arguments to the function call, like so...."
   • "You can't in $X$; use package $Y$ instead."

   However, armed with (a very little bit of) statistical knowledge, one could suggest multiplying the posterior probabilities of each class $P(\textrm{Class}=C|\textrm{Data})$ by $\frac{P_{new}(C)}{P_{old}(C)}$ to manually update the priors instead.

While I don't want to wade through tons of "how do I import my Excel file into R?" or "why does my NumPy matrix look weird?", I'd personally like the line to be drawn such that these sorts of questions are firmly on the on-topic side.

Even some of the "How can I do this, but in another language" questions may have merit, as some techniques have far too many names (there are probably about a dozen ways of saying "PCA"), and sometimes a term is overloaded to mean multiple things.

Maybe the rule of thumb should be something like "How often would this happen in a non-statistical context?" Everybody has programming problems with things like I/O and flow control, but I would venture to guess that there are very few off-topic situations where someone would care about potential differences between Matlab's eig function and Octave's qz

Answer 3

I don't really know where to post this but there is an additional point that I feel should be made. When looking at questions like

• https://stats.stackexchange.com/questions/70310/how-to-fix-mean-and-variance-of-latent-variables-with-the-mirt-package
• How to perform Chi square test for trend in SPSS v 19?
• https://stackoverflow.com/questions/18788051/3-level-logistic-regression
• https://stackoverflow.com/questions/18774985/bootstrap-on-correlation-coefficients-in-r

You need quite a lot of statistics knowledge to even simply understand the question. The answer might ultimately be as simple as “type XXX” but finding the right function, being able to understand the relevant documentation and to cut through inconsistent and confusing terminology, or to understand what parameters you need to use and check that the function is really doing what you want is far from trivial.

The wording of the questions might emphasize the “how-do-I-carry-out-test-X” aspect and ostensibly suggest that the relevant statistical decisions have already been made but they still have no place on StackOverflow. Those are simply not questions a programmer could answer.

Obviously, we might still want to close them for other reasons but we should at least agree that the only site where they could possibly be on-topic and stand a good chance of reaching knowledgeable contributors is CrossValidated.