22
$\begingroup$

I think that if there is an awareness of the statisticians of the professional societies of the purpose and success of StackExchange CV many would want to help out. In the ASA quantitative literacy has been our goal to improve. Incoming ASA President Marie Davidian gave a presentation yesterday on a grant program she has to teach basic statistics for the aim of getting more American born students interested in pursuing a career in statistics or a related field. There is a recognition of the ever-growing need for statistics to handle big data problems and the Obama administration commitment to find research in these quantitative areas.

The great thing that CV does is to get the experts to teach the novices some basic statistics in the context of problems they are working on. The accumulation from many examples improves everyone's knowledge of good statistical practice including the experts'. If the ASA leadership could see how effective we are and have some assurance that the final accepted answers are sound I am sure they would feel an obligation to participate or at least encourage others to do so.

Who should we approach (ASA president, Presidents Elect, Past Presidents, other society leaders)? Would there be any changes to our system that would increase the participation of ASA members?

$\endgroup$
  • 11
    $\begingroup$ +1 These are interesting questions, although I, being European, think that "ASA members" could be replaced by "statisticians" ;) $\endgroup$ – MånsT Jul 31 '12 at 12:57
  • 6
    $\begingroup$ What you describe sounds an awful lot like the peer-review system, @Michael... ;) $\endgroup$ – MånsT Aug 1 '12 at 7:51
  • 12
    $\begingroup$ @MichaelChernick As usual, with all the due respect, it seems like you are more worried about the credentials of the members than about the actual quality of the answers. Recall the famous saying "Change Yourself First to Change the World". The moderators repeatedly promote elaborated answers and "They lead by example". $\endgroup$ – user10525 Aug 1 '12 at 9:19
  • 8
    $\begingroup$ A system that is based on reputation points...: rep points don't confer any real power and answers are deemed "correct" based on community opinion, not the pre-existing rep of the answerer. voters are encouraged to downvote on the basis of style rather than substance.: No one encourages that, at all. The point is that if the poor style obscures the true message of the answer and makes what is written is hard to understand, then one shouldn't be surprised at a poor response in terms of net upvotes. This is no different than poor writing in a paper or a textbook. $\endgroup$ – Macro Aug 1 '12 at 12:28
  • 12
    $\begingroup$ I am not sure the the benefits outweigh the negatives: any perceived negatives probably arise from expecting the site to be something it's not. Taken for what it is - an informal Q&A site contributed to by volunteers - the site is quite good at helping a lot of people and providing helpful answers. From someone who uses the site and answers questions at a record breaking rate that far surpasses any of his predecessors, it's surprising to hear that you honestly think the site could possibly be a net loss for the statistical community. $\endgroup$ – Macro Aug 1 '12 at 12:41
  • 5
    $\begingroup$ A link to the article mentioned by @RobHyndman (Interesting info!). $\endgroup$ – user10525 Aug 1 '12 at 13:12
  • 15
    $\begingroup$ Michael, I don't know what these "higher purposes" you refer to are but, as I said before, I suspect you're expecting the site to be something it isn't. It's not a peer-reviewed publication and it's not a PhDs-only (or consultants-only) club. People should judge the answers for what they are, using critical thought and possible additional research after receiving an answer, not based on whether someone with various titles and letters after their name "certified" them to be correct. Giant +1 to @Procrastinator's comment above. $\endgroup$ – Macro Aug 1 '12 at 14:47
  • 7
    $\begingroup$ The statement "I am addressing an issue that has to do with whether or not more professional statisticians should get involved with StackExchange in order to spread good statistical methodology to the widespread practice of data analysis" seems contradictory to "MY COMMENTS ARE NOT ADDRESSING THE SITE OR WHAT IT SHOULD BE". $\endgroup$ – Matt Parker Aug 1 '12 at 18:31
  • 11
    $\begingroup$ @Michael, As a moderator I'm investigating one of your more worrisome claims: "I have seen correct answers that are counter to the OPs faulty intuition be downvoted and incorrect answers be upvoted because they agree with faulty intuition." Could you please provide links to a few examples so I can ascertain the extent of the problem and consider what might be done to improve things? Also, I'm unsure how to address the charge of "no real quality control or consistency," because that goes directly to the activities of the community and its mods. Could you share some of the supporting evidence? $\endgroup$ – whuber Aug 1 '12 at 21:43
  • 6
    $\begingroup$ Let's avoid some possible misconceptions, @Michael. (1) Moderators are permanent, as long as we remain active. The election process by which we are selected has tended to work well. (2) You fail to mention more prominent goals, as others have explained. Please re-read the site's "About" page. (3) The examples set within our best threads offer an extraordinary and unprecedented look at actual statistical practice. If you think there are better practices, then please encourage the best people you know to join us and show us how it's done. (4) Exactly what "major changes" would be needed? $\endgroup$ – whuber Aug 2 '12 at 19:28
  • 9
    $\begingroup$ As far as comparisons to Wikipedia go, I respectfully suggest you are laboring under outmoded misimpressions: the world has long passed you by. Now the onus is on you to establish that an openly debated freely editable platform--one of the four pillars of SE--is not less credible or trustworthy than an encyclopedia edited by experts. This is what has most astonished me about CV: anonymous people, laboring out of love, frequently provide corrections to and better answers than many of the recognized experts who have participated here. $\endgroup$ – whuber Aug 2 '12 at 19:37
  • 7
    $\begingroup$ All three (not two!) moderators, on this site, have a link to their homepage, as many users, although SE policy does not require to state private information or qualifications. But that's not really the issue here. Novices are welcome here, as are professional statisticians, because we all aim at providing users with a repository of knowledgeable and long-lasting answers to specific and well-thought questions, in the spirit of Stack Exchange community-driven sites. (...) $\endgroup$ – chl Aug 2 '12 at 22:04
  • 7
    $\begingroup$ (Con't) I believe the issue we should focus on, and the one which was stated in your opening question, is how professional statisticians might get involved in this site, what they can expect from this site (compared to ASA listserver, for example) and what benefit they can derive for themselves and for the community. $\endgroup$ – chl Aug 2 '12 at 22:04
  • 7
    $\begingroup$ The problem is that novices judge and can be fooled: A lot of the voting on this site is done by a rather small group of people. I've interacted with most of these people and I'd trust their votes. Novices don't vote much and so the influence of uninformed voting seems negligible - we're still patiently waiting for the apparently widespread examples of this "problem". In any case, to (roughly) quote MansT: "Reputation on the site, in the usual sense of the word, is determined the same way it is in the real world, not by upvotes". $\endgroup$ – Macro Aug 3 '12 at 12:42
  • 13
    $\begingroup$ As someone who asks questions on this site and has gained a lot of benefit from those who are willing to take the time to respond to those questions - I think the idea of credentials and all this concern about being down voted is over blown. I can tell you that responses from those interested in listing all their publication or membership credentials are almost always the least helpful. Citing 10 scholarly journal articles has a place, but typically those that have real experience and passion for the topics they respond to - provide FANTASTIC value through well explained and crafted answers. $\endgroup$ – B_Miner Aug 3 '12 at 19:25
13
$\begingroup$

I think the opposite to Michael in one respect: it's important that no assurance should be given that the accepted answers are sound.

Many of the questions on CV concern real-life problems. A site for learners of Statistics to help each other learn is unequivocally a boon to the discipline; a site where professionally certified statisticians purport to provide solutions to non-statisticians on the basis of reading a couple of paragraphs about their problem would be a travesty of the consulting process. As it stands the onus is on askers of questions to make them clear & include all relevant information, & to critically evaluate the answers; anything that tended to give a contrary impression should be, & probably would be, of concern to professional bodies or learned societies considering lending their support to CV.

Statistical societies worldwide impose obligations on their members to keep on learning, help others learn, & promote public understanding of the discipline: CV is already well suited to help them fulfil these obligations.

$\endgroup$
  • 7
    $\begingroup$ Agreed. I'd also underline a widespread comment on the original post that orienting discussion to one national association (for all its international links and roles) and to the situation in one country is quite inappropriate, if not insensitive. CV is an international forum. $\endgroup$ – Nick Cox Feb 7 '14 at 13:46
  • 2
    $\begingroup$ @Nick: Absolutely. In fact quite a lot of things in this question & the comments perhaps side-tracked responses from the main issue, which is quite a worthwhile one to think about. $\endgroup$ – Scortchi - Reinstate Monica Feb 7 '14 at 15:02
  • 3
    $\begingroup$ And I wasn't thinking about approaching presidents to get some sort of seal of approval, but sending in an article for the newsletter, suggesting a link to CV in the website, stuff like that. $\endgroup$ – Scortchi - Reinstate Monica Feb 7 '14 at 15:17
  • 1
    $\begingroup$ @Horst: Assuming that opening sentence is followed by a statement of the goals of the analysis & the particular difficulty encountered - otherwise it would just get closed - I don't see any general grounds for concern. Many of us - again, at all levels of statistical experience & education - learn new stuff through first bumping up against issues that arise when trying to analyse a particular data-set. Where the OP is obviously out of his depth people often do suggest consulting a statistician or reading up on the subject. $\endgroup$ – Scortchi - Reinstate Monica Oct 15 '14 at 13:19
24
$\begingroup$

As we are close to exhausting the comment thread, I will add my voice here. This is a personal take as a two-year member on this site.

I thought the original idea was to do some brainstorming on how to bring the attention of ASA members, and probably statisticians from other societies, to this site, Cross Validated.

The great thing that CV does is to get the experts to teach the novices some basic statistics in the context of problems they are working on.

Yes, it is true that we see a lot of good answers from working statisticians or consultants to working problems, but there are also many interesting exchanges between experts themselves. I don't think 'teaching novices' may really motivate professional statisticians to visit this site. Instead we need to clearly define what they can expect from this site (compared to ASA listserver, for example) and what benefit they can derive for themselves and for the community. In my view, an open platform for exchanges like this is an unique opportunity to give his voice to anyone interested in statistics, and share his opinion with others, whatever their level of expertise. It is close to the idea of Open Science, and in any case it offers new perspectives for experts and casual users to become acquainted with specific problems and engage in fruitful discussions. Who are the experts on this site? Everyone that provided the right answer to a nice question is a candidate. Everyone who consistently provides good answers to specific questions associated to a specific tag is certainly an expert, and he is probably perceived as such by the community. Other professional statisticians might enjoy exchanging with him.

We are a community of benevolent users. The purpose of this site, and more generally any sites in the Stack Exchange network, is to provide users with a repository of knowledgeable and long-lasting answers to specific and well-thought questions. Nothing more. We do not ask users who they are: they are free to add some personal information in their public profile. We do not ask users to be online everyday, but we expect they will be kind enough to follow questions and answers in which they are involved, and contribute to the community by voting, flagging, and helping the community as a whole (and writing blog posts if they have some free time :-). People involved in QAs get reputation and badges, as recognition of their participation. With higher reputation comes higher power to boost this site. No rating system is perfect, but the one which was chosen is generally doing its job: good answers get higher votes, active members get higher reputation. This reflects the opinion of the community, which is composed of experts and students in applied and theoretical statistics, casual users of statistics or "anyone else doing data analysis or interested in it as a discipline."

We ask questions and propose answers. New users can vote up when they have 15 rep. Starting with 125 rep, they can vote down. They are always free to accept an answer: this is their very own decision to mark an answer as useful with respect to the question they asked. Finally, askers can always change their mind if a new and better answer is offered. Again, this decision is that of the user. When the OP vote up and accept an answer, that makes up a total of 25 points to the respondent. Other users can cast their votes, in either direction, which offers an obvious counterbalance to this individual decision. Most importantly, votes help to sort out good questions and promote good answers.

We need votes. Voting is one of the bases of this site. However, as has been said on tex.SE, "content should always be the key." Everybody on this site can participate, and with growing reputation users get more privileges (retagging, voting to close, voting to reopen, wiki edits, etc.) to help to maintain this site in good health. Earning reputation in recognition of exemplary participation on SE has no other goal than allowing users to weigh in site management and expression of the user community. In the end, they become trusted users, and, certainly, they have always been willing to share their knowledge and expertise.

We elect moderators. Moderators are not elected to dictate anything on SE sites, but to help the community of users with specific tasks, in particular they "look at every flagged post, and take action if necessary," as described in A Theory of Moderation. As high-rep users often do, they also remind users with SE policy, provide guidelines for a better interaction on this wiki. It is important to remember that they are part of the community.

This site is really a good opportunity for us. I believe everyone here would agree that we try our best to promote best practices in statistical data analysis by providing authoritative and meticulous answers. Users do their best, taking into account the purpose of this site. Moreover, the SE system offers great facilities for live edits, updates, and add support for rich formatting of text. This is something that is clearly impossible on listserver. Professional statisticians might see this as a great opportunity to have more interactive and perennial exchanges.

To meet a higher purposes the judges of answers should be certified experts.

Who will decide on that? Does that mean that we have to exhibit a blue/white card attesting our level of knowledge, thereby allowing a weighted voting process? I think this reflects a misunderstanding of the purpose of Stack Exchange sites. "Professional statisticians" can decide to participate or not, as they do when interacting on dedicated mailing-lists, maintaining software, or providing up to date material on their own websites. We won't ask them what's their level of expertise is; we just ask them to provide the best possible answer, in the spirit of Stack Exchange community-driven sites.

I am sympathetic with any suggestion to extend this network of professional statisticians and people interested in statistics as a discipline. Surely, everybody will be happy with accredited statisticians visiting our site from time to time, but to challenge the actual quality of this site, as the following comment of yours might suggest

the system needs structure and consistency it seems to me to lack. A system that is based on reputation points assigned inconsistently by members who are novices and have no expertise to judge the correctness or incorrectness of answers.

runs counter to the motivations of Stack Exchange sites where everyone can contribute. Interested professionals are invited to participate, to the extent of their availability and their own interest.

To sum up,

CrossValidated is for statisticians, data miners, and anyone else doing data analysis or interested in it as a discipline.

  • Anyone interested in contributing to this site is welcome.
  • Contributions made by professional statisticians are greatly appreciated.
  • This site promotes well-crafted and useful answers to specific problems, and it is driven by a community of benevolent users who all aspire to high quality.
$\endgroup$
  • 4
    $\begingroup$ (+1) This is a fantastically well-composed, thoughtful answer. Thank you for taking what must have been a nontrivial amount of time out of your schedule to write such a nice answer! $\endgroup$ – cardinal Aug 3 '12 at 21:14
  • 6
    $\begingroup$ Very nice. I just want to emphasize a point that you touched upon. Credentials shouldn't matter; good content matters. An answer by "John Smith, Ph.D." shouldn't be given more weight than a better answer by "BaseballGuy42" who posts anonymously and may or may not have the same credentials. Good content should be able to speak for itself without merely appealing to authority. To the extent the other stuff matters, it's to better inform your Bayesian prior estimate of the quality of the content :) $\endgroup$ – Michael McGowan Aug 3 '12 at 21:50
  • $\begingroup$ @chl +1 and Thanks chl: I think this is a very good argument for what the site stands for and why professional society members such as the ASA membership might want to join. Could you turn this into an invitation letter that any of us could use to invite colleagues? $\endgroup$ – Michael Chernick Aug 3 '12 at 23:55
  • $\begingroup$ On the certification issue you and others continue to take my comment out of context. I think that if there is to be a site not necessarily StackExchange that can be trusted as a source for novices to go to to have their questions answered confidence can only be obtained if there is some form of certification. Certification is not a big problem. The ASA has it, Statistical Society of Canada has it, and the Royal Statistical Society has it. I happen to have ASA PSTAT certification so I know how it works. ASA has a committee that reviews applications. $\endgroup$ – Michael Chernick Aug 4 '12 at 0:02
  • $\begingroup$ Applications include CVs and three recommendations. There is a serivce fee to the ASA for processing certification and certified members remain certified for five years and can be renewed at these 5 year intervals. The judgement is based on experience and competency. No specific level of education is required. You do not have to have a masters degree or a PhD to be a certified professional statistician but you do need to satisfy the committee that you are competent and sufficiently trained. $\endgroup$ – Michael Chernick Aug 4 '12 at 0:04
  • $\begingroup$ StackExchange will not make such changes and therefore I think as good as it may be it will attract some but others may be reluctant to participate. I am proposing that the ASA may want to move further to meet general goals it has that such a forum could produce. $\endgroup$ – Michael Chernick Aug 4 '12 at 0:10
  • $\begingroup$ So please try to see that I am raising two separate issues. 1. Find a method to encourage ASA members and other professional statisticians to join StackExchange to continue to raise the quality of discussion and answers to questions on the CV site. 2. Try to convince ASA leadership that the question and answer approach of StackExchange could be taken up in an ASA website where the ASA could provide a similar service with the goal of improvement quantitative literacy and advancing the quality of statistic practice in society. $\endgroup$ – Michael Chernick Aug 4 '12 at 0:15
  • 11
    $\begingroup$ @MichaelC: Regarding "...can be trusted as a source for novices to go to to have their questions answered confidence can only be obtained if there is some form of certification." I don't agree with this sentiment. First of all, good CV answers often contain links and references to other sources. Second, the voting system usually works very well for pointing out bad answers. Third, that someone is a certified statisticians is no guarantee that they will write a correct answer. You yourself have had to edit mistakes in your answers a few times, because the community pointed out mistakes. $\endgroup$ – MånsT Aug 4 '12 at 7:54
  • 8
    $\begingroup$ I think that the fact that errors in answers by a very experienced ASA PSTAT certified statistican who has authored several books are caught by the community is evidence that our system can be trusted to result in trustworthy answers... $\endgroup$ – MånsT Aug 4 '12 at 8:07
  • 3
    $\begingroup$ @MichaelChernick I understand the two issues, and I answered the first one because this was the original question: How to encourage ASA members to join given how this system works. The second issue sounds like a call to fork a site like this on ASA servers and open it to certified users only, because you see a need to change two major 'drawbacks' of our system: user anonymity and vote reliability, that credentials from external judges are supposed to correct. This is less relevant to CV itself, IMHO, and it has to be demonstrated that this won't turn into a "shell game" between experts. $\endgroup$ – chl Aug 4 '12 at 9:36
  • $\begingroup$ @chl I really appreciate your thoughtful answers and comments. PSTAT certification is just one possibility for the credential. The site itself would not be restricted to experts. That would defeat the purpose. But judging questions and answers should be done by experts to ensure relevance and correctness. $\endgroup$ – Michael Chernick Aug 4 '12 at 16:11
  • $\begingroup$ @MansT Everyone makes mistakes including experts and that is why many experts are needed to check and correct the answers of others. Right now I agree that the system works well. But it is not the kind of system that I feel is guaranteed to work consistently. I have seen individual examples where things go wrong. $\endgroup$ – Michael Chernick Aug 4 '12 at 16:16
  • 2
    $\begingroup$ Very interesting thread. As somebody who only recently came across this site I have been very impressed with the quality of the answers (and moderatorship) and the obvious commitment and enthusiasm displayed by members. I would caution that any move towards credentials etc could be a move towards exclusivity which might discourage new users. I am not formally trained in statistics but hold a PhD in a related field, does this mean that my answers are therefore always less valuable than a qualified statistician? $\endgroup$ – BGreene Aug 14 '12 at 9:58
13
$\begingroup$

My forecast is pessimistic.

The community of academic statisticians (aka ASA members) produces fantastic initiatives that would greatly enhance the discipline as a whole. Unfortunately, most of them are one-man uphill battles.

When Jim Pitman was the President of IMS, he actively promoted the idea of having more publications in open access. There were many good points in his presentations and articles on this (of which the most tangilbe outcomes is posting the Annals contents in arxiv.org). An obvious one is that it is cheaper for the end user -- why should one pay the commercial publisher for essentially zero added value? We submit a properly formatted, camera-ready paper (although I've had a number of examples of publishers who retyped my LaTeX into Word... it is difficult to desribe how furious I was fixing their typoes and furmatting errars); why paying somebody $1000/year to access five papers from this journal? Another idea had to do with incentives in evaluation of faculty: OK, we all understand that you need to publish in JASA and Annals to get tenured. But once you are there, why don't give more consideration to free online journals? Let the young faculty play in these high impact journals. People know who Peter Hall or David Dunson is if they've opened more than three issues of JASA or Annals over the last five years, and you don't need to continue proving yourself anymore to get promoted any further. Make your work accessible to non-academic researchers and the third world. Wonderful ideas; how much of that has changed over the past five years? None.

David Banks visited what then was my department of statistics, and gave a talk that analyzed whether Wikipedia is complete, in the sense of having every topic covered. In the end of the presentation, he urged the members of the audience to contribute to wikipedia and help improving it. Wonderful ideas; how many statisticians have contributed to Wikipedia over the past five years? A handful, I guess; and we probably have all of them on this site of benevolent statisticians, as chl put it. (I do list my contributions in my curriculum vitae, just in case I were to run across another benevolent statistician at the evaluating end.)

Finally, just looking at our website, of the big names in statistics (defined as people who would get an immediate recognition from about half of the attendees at the JSM... or whatever the analogue is in Europe -- sorry, my view is totally obstructed by the US borders), we have Michael Chernick and Frank Harrell, who does not seem to be very active recently. That's a very low hit rate -- I guess I could put another hundred or so names into this hat of the authors of the important books and papers who would fit the above definition.

Of the existing statistics cultures, only consulting (represented wonderfully by whuber) may have any incentives to contribute actively to CV. Academic statisticians need to publish their papers in as highly ranked journals as possible, and CV does not have an impact factor. Industry statisticians need to report their hours, and CV is not a paying client. Consultants may seek to extend their business, so CV may indeed be an engine. Once the criteria for promotion IN ALL DEPARTMENTS would include maintaing a 1000 rep/year on CV and maintaining at least three wikipedia pages in your expertise area, making at least one edit a year in each of them, the situation will change, and we'll all be crowded out by people who are better statisticians than StasK is. (I've heard of people who made contributing to CV a requirement for their consulting classes, though.)

Here's a little bit of the silver lining. The faculty search committees would benefit from CV in the parts of evaluating the communication abilities of their candidates: if a person can explain things well on CV to statistically un- (or under-) prepared audience, chances are they will succeed in the classroom or in the consulting function, too. This is the only realistic selling point I see as a way of explaining how CV can be useful to the statistical community, the way it exists and functions right now. If young faculty were to work on CV and to bring this kind of culture into their classes, we might see the paradigm shift in the nexts twenty or so years.

$\endgroup$
  • 2
    $\begingroup$ I agree that even with good intentions to help academic statisticians may find that they have very little time to contribute to a site like CV. But I have notice large participation of statisticians ( mostly industrial consulting statisticians) on the ASA eGroups. It was on the consulting statisticians eGroup that I learned about CV from Bill Huber. I think there can be some gain here. My idea of aN ASA site to do something like this may be a pipe dream but the goal of improving quantitative literacy and statistical practice is are clear goals consistently raised by ASA presidents. $\endgroup$ – Michael Chernick Aug 5 '12 at 19:03
  • $\begingroup$ Michael, you are right, ASA groups and section mailing lists are quite active, although among these, the Section on Consulting is by far the most active -- again, see the reasons I outlined above. Other groups and sections may have lower activity levels; SRMS, the second or the third largest sections, has its own mailing list outside amstat.org, and it's kinda active with ~5-10 messages a week. $\endgroup$ – StasK Aug 6 '12 at 14:41

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .