Improvement Drive - Cross Validated

Question

Greetings from Stack Exchange HQ! We have been assigned to evaluate Cross Validated and recommend some methods for making it even better. Please note that this is a more hands-on evaluation, and is not related to the Community Quality Evaluations already done on sites. This is a distinct evaluation project designed to quantify the community and health of the site and recommend ways for you, the core users, to improve it! Yay quantifying and evaluating!

Let’s get right to it--your site is doing really well in a lot of key metrics! Your sites gets good traffic and has a good base of users. It also appears that Cross Validated has a lot of solid content. You have engaged users who care about the site and the quality of content on it. That is awesome. These are qualities and behaviors common to good sites, and we’re pleased to see your site doing well at them.

However, according to our metrics, these areas of the site need improvement.

1. Answer ratio: Cross Validated is hovering at a 78% answer rate, which is the lowest of the sites we evaluated. We understand that ‘statistics’ is an incredibly broad topic covering many fields of study, but we also believe that in order for this site to flourish on our model we need to find a way to increase the answer ratio.
2. Voting Ratio: this is tied to the previous item, but it is still an issue and one we have addressed before.
3. Chat: Activity in your chat rooms is almost non-existent. Chat is a feature we believe strengthens and reinforces community and fosters positive and helpful communication between users.
4. Lots of viewers: Unusually, perhaps, Cross Validated isn’t suffering from lack of views, but we believe it is suffering from lack of "quality of viewers". There is probably a better way to phrase that, but the upshot is that we need to get some more experienced eyes on the site.

Here are resources and ideas for improving the site and community:

1. Reach out to your real life networks. Share the site with your co-workers, students, TAs, professors, etc. Share the site and invite your smartest, most knowledgeable stats-crazed friends. Referral Badges are awesome (just saying!).
2. Chat Events! Whether they are weekly topical events or daily check-in times, increased chat usage is an easy way to increase the appeal of your site to new and experienced users.
3. Social Media Networking: Cross Validated doesn’t really suffer from lack of views, but it seems like most of those views are primarily from the ‘homework’ crowd and aren’t going to do much to improve that answer rate or improve the breadth of knowledge in our sites. Use your personal social networks (twitter, tumblr, FB, stats messageboards, etc.) to promote the heck out of this site to the people you know would enjoy CV and be helpful resources to the community. We are not advocating spamming everyone you know. However, you, the current passionate community of Cross Validated probably have some contact in some network with the next generations of users, if you know anyone else in the field.

I want to make it clear that none of these suggestions are required. They are merely suggestions for actions that we believe help to grow and sustain your community--things we think will work and help.

We are relying on you, the core user community to implement and spearhead these efforts. Our position and expertise allows us to evaluate our sites, but YOU are the users. You know the site and we want to enable you to improve it.

If you, or other core users of Cross Validated, have any other ideas of what can be tried to improve on these metrics, or any other element of the site you see as problematic, please leave an answer to this meta post. Our team will be working with a selection of sites for the next several weeks and you can feel free to reach out to us directly via email for support.

This improvement push has grown out of a larger effort to create a model for judging the health and potential of each of our sites taking into account all the relevant data we have about your site. Please let us know if there is a metric you think we are missing.

Finally, we know you are excited about the potential for our sites but please keep this meta post focused on things you, as the collective community, can do as opposed to feature requests or larger branding campaigns. We want your ideas, so please make them separate meta posts so we can keep all these discussions focused!

Answer 1

Regarding the "answer ratio" - I think that is partially a function of the nature of statistics and the questions we get. Not so much that statistics is incredibly broad - because the people who answer the questions here have a remarkably broad range of expertise - but some questions either have no answer or are so poorly worded that they cannot be answered (although, if they were better worded, they could be answered).

As for "chat" - well, what is it for? I've been very active here over the past year, but I'm not sure what chat is for, except for the case where a conversation in one of the threads gets a bit astray.

Answer 2

1. There are some topics with a lot of overlap with stackoverflow (e.g. data mining questions) and I guess also math.stackexchange and mathoverflow (yes, there are even two sites for math!)

   It's not always clear which site is the more appropriate place, and when to suggest people to use the other site. This of course also leads to cross-posts; these tend to get answered more quickly on the larger site (i.e. main SO).

   IMHO it would be more helpful if this were more clearly defined, and if the "migrate" function was easier to use. The "close" vote does not include migrate to stats. IMHO, it should offer at least all sites the user is also active on, not just some random 5 subset.

2. My impression is that the easy questions are more likely to end up on main SO, while all the really tough statistics questions do end up here. To me this explains the lower answer and upvote ratio.

   This makes me wonder: should we actually care about these metrics?

3. I tried chat on main SO, and found it pretty much worthless. Why would I use it here, then? You don't need "live" math, you need some time to think and write down your ideas in a well-formulated way.

Answer 3

This answer focuses entirely on the repeated encouragement to use chat.

I have glanced at chat and (more often) been encouraged by automatic prompts to move a conversation to chat, but I've always resisted.

I naturally have no objection to anyone using chat if they find it useful (or even entertaining) but chat to my mind has various disadvantages, at least for users like myself:

1. What can be most valuable about threads is the diversity of suggestions, including frank disagreements about how to see or to tackle the problem. The comments system is already excellent and well suited to making smaller points, including those expressed a little more informally.

2. I have noticed that although chat is very visible if you seek it out, many people lose inhibitions they have in the forum, become digressive, personal, trivial and even offensive. (A recent poster made obnoxious remarks in chat about people who had tried to help him, but gave up frustrated.) They also often become more inarticulate and write more poorly. The value of chat is thus very limited except for the people concerned at the time, so I really don't see that anyone should be encouraged to chat if they don't feel so inclined. (If you want to read this in reverse, and say "But that is what chat is for!" I agree, but it is not to my taste.)

3. Invitations to chat often come from inexperienced users, intent on one or two things only, getting an individual answer to a specific question (and often getting blessing for something they have already decided they want to do, usually in the form "Is this OK?"). Chat just distracts and detracts from the bigger goal of CV, building up an archive of good answers of use to a large number of people.

(Using comments heavily does carry an obligation to clean up once a conversation is done, removing stuff made redundant by later edits, and so forth.)

Answer 4

I disagree with one of your metrics - I hope that I am allowed to do that if I have a good reason. I think the zeroth metric to return is value. Value can hard to measure because it can mean different things to different people. One person's cost is another person's benefit, sometimes. Metrics should be built to measure and allow value to be maximized.

First off, CV is one my favorite sites. It really is. I can learn. I can be challenged. There is a community of excellent and admirable folks who treat each other well and if you treat them well and with respect you can ask good questions and get good (and sometimes truly amazing) answers. I am on here at least daily.

I'm going to tell you something personal, but brief. I had a neighbor that I tried to build any sort of connection with for years. One day I was making cookies, and had a thought. Even though I had sugar, I went and asked if I could borrow a cup of sugar. They loaned it to me, and in return I brought them some cookies. The next day there was a knock asking if they could borrow some butter. One day! By giving my neighbor a chance to give, I got the chance to have a good friend. It was a good win, and a win-win.

There are a number of different demographics that use the various Stack Exchange sites, and I think that CV as-it-is offers things that other, adjacent technical ones (Stack Overflow, mathematics, computational science) do not - an entrance to the community for "entry level folks" or even "highly capable but new-to-statistics" folks. For me that "entrance" or "accessible participation" has been through the older unanswered questions. I have earned "necromancer" by looking through unanswered questions under my favorite tags and seeing a "gem" that I know just the right answer for. There is a lot (whole honking lot) that I don't know and so having the right (non-zero) questions down my (atypical) alley has been good. If all the questions were answered then I would not have had that opportunity. This community would have been a lot more closed to me. Being able to answer questions, even if they are sometimes less technical, is like CV giving me the chance to share a cup of sugar.

One thing that I have oblique (pun intended) view into here is that this site, I think, returns very great value to the technical community. I will see someone who acts "newbish" asking several questions on a topic, and a few months later will see a published article on the topic. The person who published ins't always the "newb", but I suspect they were encouraged to go down the road they did because of the newb's question. I would bet that between 10% and 25% of tough questions here result in things like masters theses, PhD dissertations, and other peer-reviewed/published work and software packages that result in substantial improvement in the field. Seriously - I think you should apply for an NSF grant on the topic. I think that CV could be responsible for several hundreds to thousands of meaningful steps forward in statistics, the field that is most applied outside of directly engineering a product.

The "Nobel Prize" syndrome causes folks who found buried treasure in their backyard to go try to climb mount impossible. I think that all treasure is buried in backyards. I think that one of the things that CV does is help folks figure out how to look around where they are and find the unknown and fun, and that is where buried treasure is always found.

I have never successfully used chat here. I think that it missed its window. Around 10 years ago it would have been the only way to go. Now, unless it is a longer-term conversation, perhaps like on the back-side of Wikipedia pages, I don't know if it is ever going to gain traction. Is it really such a bad thing if it doesn't catch on?