8
$\begingroup$

One of the less precisely-defined (mathematically or otherwise) aspects of performing Bayesian inference is assignment of priors based on domain knowledge, and there are plenty of questions on choosing piors on this site. Advice that I have read (from Andrew Gelman, Richard McElreath, and others) has ranged from choosing weakly-informative priors to choosing priors that balance being uninformative with regularization to leaning into use as much prior information as possible. This meta-post is not about choosing priors per se, but a closely-related topic: soliciting domain knowledge from experts to help me design priors.

It a further step removed from selecting density functions for (proper) priors to solicit information from domain experts, which may happen before any mathematical ideas are even put on the table. We're not always the domain knowledge expert, and it is important to learn what we can from them. I recently added a random effect to a model due to a conversation I had with an expert that I would not have thought to include otherwise. I believe these solicitations are part of the workflow of statistics, and cannot be safely ignored in practice.

The trouble, for me, is that I can't seem to find the words to ask good questions about this. I have developed my own ways of doing these things based on reading and practice, and some of what I want is to compare notes with others. This doesn't fit the format here, so I don't ask.

Broadly "comparing notes" is off-topic on the main site, I believe, but I wonder if there are questions about soliciting information from domain experts that could be asked. I find that I run into a labyrinth of problems around candidate questions either not being broadly relevant to others (i.e. too specific) or subjective.

  • What do you think of asking about questions about how to solicit information from domain experts to use later in selecting priors?
  • If you think it can/should be done here, can you provide an example or describe requirements?
  • If you don't think it can be done, are there problems beyond being too specific or subjective that you would like to share?
$\endgroup$
5
  • 10
    $\begingroup$ Prior elicitation is what it's called, it has an extensive literature, & I can't see any reason why questions on it oughtn't to be within the scope of our site. $\endgroup$
    – Scortchi - Reinstate Monica Mod
    Commented Feb 18, 2023 at 19:28
  • 3
    $\begingroup$ Thank you for challenging the premise of the question, and correcting the terminology. I had the right idea, but the wrong wording came to mind. I see that Hosack et al 2017, Mikkola et al 2021, Stefan et al 2022, and Azzolina et al 2021 appear to be relevant papers. $\endgroup$
    – Galen
    Commented Feb 18, 2023 at 19:33
  • 1
    $\begingroup$ The works of J.B. Kadane too. $\endgroup$
    – Scortchi - Reinstate Monica Mod
    Commented Feb 18, 2023 at 19:51
  • 3
    $\begingroup$ Thank you for the recommendation. It looks like Experiences in Elicitation is among Kadane's popular works. $\endgroup$
    – Galen
    Commented Feb 18, 2023 at 19:56
  • 4
    $\begingroup$ If you edit this to be in the form of a question about prior elicitation, then it would be suitable for migration to the main site. No need for the preamble, etc. Just write it like a normal question. $\endgroup$
    – Sycorax Mod
    Commented Feb 19, 2023 at 14:03

0

Reset to default

You must log in to answer this question.

Browse other questions tagged .