Are questions about ethical use of statistics on-topic?

Question

A strong majority of the questions I have seen on this site relate to technical aspects of statistics such as mathematics, algorithms, and sample design. I am curious if questions about ethics are on-topic here.

Here is an example of a question I am interested in.

In healthcare we are often concerned with studying the quality of care and clinical outcomes. Patients come from a diverse range of backgrounds when they arrive in a clinical setting, and some of these backgrounds can be accounted for in the analysis. For minorities or under-represented populations there can be relatively few individuals per group, motivating the use of techniques such as mixed effects models rather than doing without pooling. Such techniques exhibit shrinkage in the sense of pulling the random effects sizes toward the fixed effect, which may or may not be biasing in an estimator sense. This shrinkage might result in underestimating how different some of these populations are, leading to decisions that decrease quality of care or clinical outcomes. Is there an ethical way to use mixed effects in an analysis of such populations?

And yet, I doubt this question is suitable for this site. Questions about ethics require systems of values which can differ in important ways between people, making potential answers subjective. It is also somewhat vague or open-ended. I think such a question is better answered outside of this site by each health authority.

What do you think of the on-topic-ness of questions about the ethics of applying statistics?

Answer 1

I don't believe there's any precedent for ruling such questions out of scope. Their requiring statistical expertise to answer would be the general principle under which they're in scope.

The kind I don't think we want include questions that

• are too specific—say pertaining to the standards of an ethics board at a particular institution

• ask us to resolve a conflict between statistical & extra-statistical ethical imperatives. "If I don't do the p-hacking my boss insists on, I may lose my job & not be able to pay for my son's violin lessons"—write to the agony aunt at your newspaper of choice.

Answer 2

I wouldn't recommend closing that question or migrating it to, say, the Philosophy SE site. The ethical concerns are grounded in the properties of the analyses, which we can discuss here. For example, the question is motivated by the assumption that using a mixed model will yield a biased estimate. What methods are biased, how much, and in what manner, are things we can explore. In the end, there will be different pros and cons for different choices; I don't think we should tell you that you are obligated to chose pro A over pro B given con B vs. con A, but by the time all of that is clear you will be equipped to make the appropriate decision yourself.

Answer 3

I would add a simple point that questions can be ethical and technical at the same time.

Is it a good or at least defensible idea to omit outliers? is a frequent example, where one simplified answer might start with

• Yes possibly, if it is clear that an outlier is wrong and can't be corrected, or turns out to be irrelevant to your goals, but be clear about what you are doing

• No almost certainly, if the motive is that such outliers are awkward or inconvenient for your actual or intended analysis

-- and what underlies the second point is a concern for being honest and realistic about the data.

Other examples could be

• Binning variables where one motive is to reduce the possibility of tracing data points back to individuals (often desirable or even mandatory), but otherwise the usual statistical reaction is that such practice is throwing away information

• Blurring variables by adding noise where similar concerns arise

• Transforming variables where one concern (usually misplaced, but it exists) is that this amounts to changing the data, which would be wrong

and perhaps above all

• The tension between exploratory data analysis (advised or presented positively as looking at the data carefully to spot problems and possibilities not necessarily evident in advance) and data snooping (advised against on several grounds)

• Shady or dubious practices with multiple significance tests, where one dimension of concern is how that is statistically or scientifically honest, where honesty is ethical as well as technical

Leo Breiman has this advice on p.1 of his (excellent, but seemingly little known) text Statistics: With a View Toward Applications (Boston: Houghton Mifflin, 1973)

The main thing to learn about statistics is what is sensible and honest and possible.

Flagging honest is a little unusual in such a text -- not that texts advocate the opposite! -- but it is totally in order.

Answer 4

There are questions where the main point is ethics but to answer requires statistical background. For instance if you serve on the data monitoring committee for a clinical trial you may be faced with a situation where it seems that the trial may need to be stopped early for safety (not for futility or for benefit). This is an ethical issue balancing the interests of various parties but in order to give a good answer you need to understand the issues involved in early stopping which I believe are statistical.

Answer 5

The example that you mention is interesting, but it might be better to frame the question in a way such that it doesn't become a question about ethics of medicine and deals instead with the different aspects (among which one is the statistical aspect) separately.

A discussion about ethics is not on-topic here. For the specific example that you mention the discussion about ethics would also only muddle the technical approaches that are relevant in the discussion. The technical approaches are ontopic, but the discussion is not.

In principle questions that involve ethics are not off-topic, anything that is about statistics is on-topic, but the questions should not become broad, unclear, opinion, etc. It are those aspects that make questions about ethics in practice off-topic, but it is not the ethics that is in principle off-topic.