13
$\begingroup$

I'm seeing a steady worsening of statistical preparation on the part of those posing questions to the site. If I wanted to start an engineering project, I think I would talk to an engineer before doing anything, or at least read an intro engineering book.

I see people routinely asking statistical questions in a way that demonstrates that they want to use the site as a kind of Google quick-fix pill to address important statistical issues, without bothering to do any background statistical studying of any kind. Everyone seems to feel they should understand statistics without studying, but I don't think that people feel the same way about engineering, chemistry, or neurosurgery.

What can we do better to get people to at least pretend to do background study before posting questions?

$\endgroup$
  • $\begingroup$ Is this question an example of what you are thinking of? stats.stackexchange.com/q/308097/1352 $\endgroup$ – S. Kolassa - Reinstate Monica Oct 15 '17 at 18:44
  • $\begingroup$ That's an example though not one of the worst. $\endgroup$ – Frank Harrell Oct 15 '17 at 23:46
  • 6
    $\begingroup$ I don't have data on this, but if it's true it's unfortunately a reflection of our success. Apart from Wikipedia, which other site would seem as likely as CV to be a source of free authoritative advice for naive statistical users? (No interest or intention to start a debate on the merits of Wikipedia, which at a minimum we can agree to have different aims.) Also, I don't think many students are obliged to study some engineering when their main aims and talents lie elsewhere, but being obliged to study some statistics is common across many fields. $\endgroup$ – Nick Cox Oct 17 '17 at 0:40
  • 3
    $\begingroup$ I agree with this assessment and feel a growing irritation at OP not even checking Wikipedia pages that would address their issues, not to mention opening a textbook... $\endgroup$ – Xi'an Oct 23 '17 at 8:25
9
$\begingroup$

Summary: The following does not excuse questions that do not show any effort of research and thinking: we certainly should request that.

However, there are reasons why I think we have to expect questions at a very low level and unless this site is relabeled as "for statisticians" I'd expect this to stay this way.


Long version:

If I wanted to start an engineering project, I'd hire an engineer and not pretend to be one.

While I'd have been very happy to have statisticians in the projects, my academic experience is that this is often not the case, and for a student it is not possible to get the statistician (unless statistical consulting was planned for in the project proposal). Impossible in the sense that the lack of proper statistical advise is caused by blind spots in project organization. (BTW: I do see a link here e.g. with widely lamented code quality issues).

I've encountered a number of projects (in natural sciences) where during proposal writing and starting the project, noone thought of getting statisticians into the project, which at some point leads to PhD students doing/learning statistics in a swim-or-drown fashion. From what I know in my field, many such students can get little if any guidance in that respect from their supervisors (or, maybe even worse, are in a situation where some kind of statistical test is basically thrown at them with possibly the only justification being that someone heard of that test - I'm thinking of a very difficult discussion I had with a medical doctor who absolutely wanted me to calculate variable significance for a random forest [because they had seen such beautiful stars] in a situation where the data generation process meant that random forest variable importance may be very low for truly important variables. If I hadn't been a stubborn postdoc with 10+ years of experience in both the data generation and the data analysis meaningless numbers would have been calculated and most likely published).
Note that in this scenario, even very naive questions (from a statistical point of view) may be a sign of a very bright student: they found out that they need to ask a statistician.


Disclaimer: I'm no statistician. I'm chemist, and my formal training included very little statistics, basically enough to realise that there is a whole field (chemometrics) that applies statistics to chemistry. After 15 years specializing in this in a mostly autodidactic way, I feel my statistics knowledge to be OK in what I do every day (chemometric analysis of spectroscopic data), but very sketchy in other respects. However, I'm also able to translate between statisticians and chemometricians and chemists - and I find I'm asked to do exactly this. And I still find that when consulting statisticians, I have much trouble due to the different language.

I've also personally encountered a number of statistics-related research questions where my gut feeling was that they are probably solved already but was not able to search effectively as my description of the problem did not lead to good search terms. (For some of them, I've since found the name they got in other fields - totally different from all I did think of.) But for the question at hand the conclusion certainly is that bad wording should be expected.

There are a lot of questions here in the topics I follow where I fell the very same thing has already been asked twice this week (specially with the cross validation questions). But I also think that the usual terminology encourages some misconceptions, and the other questions I refer to typically share the same answer, but not necesarily similar sounding questions.


Everyone seems to feel they should understand statistics without studying

I certainly feel that everyone should understand enough statistics to realize that (when) a statistician should be consulted. But again, in this context that applies possibly more to supervisors than to the students we encounter here.

However, I'd suspect that one reason for this feeling is that e.g. a chemist or physicist developing some kind of measuring instrument typically knows how to calculate "the result". They feel that they did actually study as they work in their very own field.

However, they are often unaware of uncertainty issues (exception: analytical chemists and AFAIK also technical physicists learn about measurement uncertainty in their studies in a somewhat deeper manner than 1st semester labwork practica cover). And even worse, while that may be OK in pure physics or chemistry experiments (where they may have enough expert knowledge to make sure that uncertainty is not an issue in answering the question of the study), those disciplines move towards looking at biological systems now, and the "expert feeling" of their domain doesn't give any clue about the variability that biological systems typically have.
Which I think is also a cause of the organisational statistics disaster I discussed above.


I don't think that people feel the same way about engineering, chemistry, or neurosurgery

I don't know whether this is any kind of consolation, but I don't think those unhappy attempts at statistics are worse than some attempts at working in a chemical lab I saw from (non-chemical) engineers and physicists...

$\endgroup$
10
$\begingroup$

The help on How do I ask a good question? has a an opening section on "Search and Research".

The page relating to questions autoclosed for "not meeting quality standards" also makes reference to showing your research.

As Tim pointed out, votes (and comments) are a good way to make it clear that questions need work. Sometimes that's not sufficient and we have some other tools --

We can certainly close a question that doesn't display any research if the post fits one of the closure reasons (many will fit under unclear or too broad) and extremely poor questions can be flagged as "very low quality".

$\endgroup$
  • 5
    $\begingroup$ Agree, however in many cases giving -1 is a way to show that you consider a question is low quality. In extreme cases I'd be inclined to close them as suggested by Glen_b. $\endgroup$ – Tim Oct 15 '17 at 13:04
  • $\begingroup$ Thanks Tim; I should have mentioned votes explicitly $\endgroup$ – Glen_b -Reinstate Monica Oct 15 '17 at 20:11
4
$\begingroup$

I disagree with Frank on whether the level of question is declining. I haven't noticed such a change. Indeed, compared to some other sites, I find a lot of the questions are quite high level.

However, I agree with him about "everyone feels they should understand statistics" and "if I wanted to start an engineering project ..." but I'd go further: If I wanted to start an engineering project, I'd hire an engineer and not pretend to be one.

If that was the case, then most of the questions here would be either very high level questions from other statisticians or low level questions from people who are just curious about something.

$\endgroup$
  • 7
    $\begingroup$ Peter to clarify, I don't want to imply that there is a decline of high-level questions. But I'm noticing an increase in the statistical naivete on the other end. $\endgroup$ – Frank Harrell Oct 15 '17 at 12:33
  • 7
    $\begingroup$ The site has been continually growing over the years. It seems to me the number of high-level questions has remained fairly constant; the growth has been largely due to increasing numbers of poor-quality questions, which makes the proportion of high-level Qs shrink. $\endgroup$ – gung - Reinstate Monica Oct 15 '17 at 14:09
  • 6
    $\begingroup$ The problem seems to occur often with first time users. They generally don't tour the site before asking questions. Sometime a member will suggest that they take the tour and learn how to ask a question. I wonder if there can be another way to deal with this. $\endgroup$ – Michael R. Chernick Oct 15 '17 at 21:50

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .