Data Science SE

Question

I noticed the ad on the main site today for the data science proposal, which evidently is 77% committed. Some of the people committed to the proposal are users on CV. Most of the questions listed look like they would be on-topic on CV, or even might be duplicates of existing questions. A minority of the questions are more programming oriented, but I would guess they could fit on one or another of the various SE programming sites. Of course, people can form a new site if they want, but I'm wondering what the value-add is? Is there something we're lacking, or should we be more welcoming of something? Moreover, what do people think of the proposal?

Answer 1

Someone raised this issue in Area 51, Overlap with existing sites, with only a little discussion.

I would think Cross Validated would be suitable for most Data Science topics, but apparently there is a perception that CV is for theoretical stats questions. The CV description is "a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization" which seems aligned with Data Science.

As far as the question of what can be done to avoid the overlap goes, we can

• participate in the Area 51 discussion
• start some more organized effort of "try CV first"
• add "data science" to the CV description
• be accepting of less statistical data science questions (data cleaning, storage, ...)

As a point of comparison, there are two math sites:

• Mathematics "for people studying math at any level and professionals in related fields"
• MathOverflow "for professional mathematicians"

Answer 2

Spoiler: a highly opinionated answer follows.

I am here with Peter Flom. I think questions on how to handle stuff in statistical packages deserve a site that is separate from CV and separate from SO. I would be the first to admit that I am behind on any of the Data Science issues involving Hadoop or Mahout -- I don't get to see the limits of my existing tools in the work that I have, and however much I would love to learn Python and such, I don't have the need for the applications that I am an expert in, and thus have no pressure to learn them.

Here's how it settles in my head (which is the sample of size 1, and there are 4000 heads on CV meta, most of which are undoubtedly better than mine). To me, programming is figuring out how compilers work, and what they allow you to do, almost regardless of the input data (and a good program will protect itself from poor data, anyway). Most statistical packages (excluding probably R) lack the syntax features to express these levels of abstraction from the data. On the contrary, in data analysis tasks, we usually have to take data for granted, and dance around it, may be with some reshaping or contracting of the data or construction of some new variables to better explicate the existing data structures (and the vast majority of the real life data sets represent a single rectangle -- medical patients and their vitals, say; sometimes two or three rectangles -- patients plus hospitals plus insurance companies). (Going back to my ignorance comment in the first paragraph -- this is the kind of data that I deal with, not the jagged, multiple sites twitter threads that many people fancy to mine.) So R or SAS or Stata questions on SO have looked out of place to me all the way along.

There has been a very recent discussion about this on Statalist, the support forum for Stata software. Due to some planned hardware and personnel retirement, this whole list is being transplanted from a text-only email version to a forum version. My wishful thinking was, though, that the mass of Stata users (several hundred, I am guessing) could move to StackExchange, for the mutual benefit. The Statalist users will get a functional platform that works well in improving answers and collecting knowledge. CV will get a group of users with high technical expertise (although some of the most active ones are already here; you can easily identify them in the appropriate tag activity).

As far as duplication of the content goes, there has been discussion of how to split mathematical statistics questions between CV and math, and what to do with education questions. Duplication is probably unavoidable; however in case of the overlap with math, at least math people are aware that theoretical statistics is present on CV. Whether the data science people are aware that A/B tests have been invented some hundred years ago is an open question to me; some are, but many probably aren't. What to do with the data science movement is an open question for professional statistical societies, but that's a wave that statisticians cannot steer even the slightest. Extrapolating this experience here, I would expect that the CV users won't be able to stop the Data Science Area 51 from whatever trajectory it is on (although comparisons with Big Data proposal that never took off the ground are very obvious), especially given a much greater mass of programmers-proper in the StackExchange system as a whole compared to the mass of statisticians-proper on CV, and a much lower reputation of whuber compared to Jon Skeet. If anybody will have the patience to sift through the Data Science questions and point to the appropriate duplicates on CV, I guess we should give them a platinum badge of a saint.

Answer 3

Update September 18, 2014 (data are for public Beta)

\begin{array}{| r | r | r |} \hline & \text {CV} & \text{Data Science} & \text{Data Science}\\ \hline \hline \text {Metric} & \text {end of Beta} & \text{80 days in Beta} & \text{128 days in Beta}\\ \hline \text {Q per day} & 7.6 & 2.6 & 2\\ \text{% answered} & \text {96%} & \text {90%} & \text {82%}\\ \text{avid users} & 138 & 47 & 55\\ \text {total users} & 1,763 & 1,773 & 2,311\\ \text {answer ratio} & 3 & 2.5 & 2.4\\ \text {Visits/day} & 1,159 & 230 & 179\\ \hline \end{array}

Looks like a decline. The interesting fact is that the only metric doing really good is "total users" (at $+550$ in $48$ days, this is $11$-$12$ new accounts per day). This means that there is an interest for the site "out there", but those interested are passively waiting to get knowledge by "other peoples' questions and answers"... But sometimes everybody is "other people", as indicated by the decline of the "Q per day" metric, which in turns makes the site poor and leads to lower visits by the (increasing number of) total users. Also, one could conjecture that the interested users do not feel knowledgeable or confident enough to answer questions, as indicated by the falling "% answered" and "answer ratio" metrics (which sounds a reasonable conjecture given that the field is essentially nascent).

Feels a bit sad.

---

Update August 2, 2014 (data are for public Beta)

\begin{array}{| r | r | r |} \hline & \text {CV} & \text{Data Science} \\ \hline \hline \text {Metric} & \text {end of Beta} & \text{80 days in Beta} \\ \hline \text {Q per day} & 7.6 & 2.6 \\ \text{% answered} & \text {96%} & \text {90%} \\ \text{avid users} & 138 & 47 \\ \text {total users} & 1,763 & 1,773 \\ \text {answer ratio} & 3 & 2.5 \\ \text {Visits/day} & 1,159 & 230 \\ \hline \end{array}

Answer 4

Edit:

The site has now launched.

Old post:

I lurked around there and want to share my preliminary, biased impression:

Judging from the first few questions there is a lot of overlap and redundancy with SO and CV. Case in point: the five most recent questions were R vs Python, sentiment analysis, calculating AUC, extract rows in R, number of layers in neural networks. All these questions have been answered on SO or on CV. Voting behavior tends to be a bit better and currently the fraction of unanswered questions is extremely low. I am not yet comfortable with giving a qualified judgement of the quality of the answers and questions, but they are missing the "knights in shiny armor" that CV undoubtedly has. Preliminarily it appears to me to not be too high, a lot of opinion and hand-waving, less SE like and more discussion forum-ish. But perhaps the "CV-way" is too engrained for me to be fair.

Authentic "data science" questions (i.e. not suitable for CV or SO) seem to be rare and the site seems to tie in mostly with what data science has come to mean nowadays: Data analysis for engineers who have a very narrow (and imo misinformed*) definition of "statistics". In that it is telling that the highest voted question is The difference between DS and Stats and the top answer pretty much sums that (imo misguided view) up (we older people would call it "computational statistics" and what is listed as specific DS tasks are what we did in grad school since the early 90ies in our statistics curriculum).

*This is certainly also statistics' own fault, especially with respect to how it is traditionally taught.

Answer 5

I am not sure what about "data science" would not be covered either here or at StackOverflow, but one thing I can think of is programming questions about R and SAS and the like.