The "mixture" vs. the "compound-distributions" tags

Question

TL;DR:

• I have added a tag wiki and an excerpt to compound-distributions. The excerpt notes that these are also known as "mixtures", but that that term also has other uses.

• I have included a warning in the tag wiki excerpt for mixture. It now reads:

  A mixture distribution is one that is written as a convex combination of other distributions. Use the "compound-distributions" tag for "concatenations" of distributions (where a parameter of a distribution is itself a random variable).

  I will link to this meta question in the tag wiki.

• I will go through threads with the mixture tag in the next days and retag them as compound-distributions as warranted.

---

Explanation:

The term "mixture" has two distinct meanings in our context:

• A mixture distribution is a convex combination of two or more underlying distributions with associated probabilities $p_i$ that sum to one. A realization of the mixture distribution comes with probability $p_i$ from the $i$-th underlying distribution. Examples include:
  • The mixture of two or more normal distributions with different means and/or variances: gaussian-mixtures.
  • The mixture of a point mass at zero with another (discrete) distribution: zero-inflation.
• A mixture may also be a "concatenation" of two distributions (compound-distributions), where our realizations are distributed according to some parameterized distribution $X\sim F_\theta$ where the parameter $\theta$ itself is a random variable, $\theta\sim G$. So, to draw an $x$, you first draw a $\theta\sim G$ and then, with this $\theta$, draw $x\sim F_\theta$. $G$ is called the mixing distribution. Examples include:
  • The negative-binomial distribution arises naturally as a Poisson distribution whose parameter is itself gamma distributed: it is a "Poisson-gamma mixture".

It does not help that even when people discuss compound distributions as such, they almost invariably call the resulting compound distribution an "$F$-$G$ mixture", as in the negbin case (not, e.g., an "$F$-$G$ compound"). Ah well.

---

Our mixture tag refers exclusively to the first meaning above. Here is its tag wiki excerpt:

A mixture distribution is one that is written as a convex combination of other distributions.

The overloaded nomenclature results in people tagging questions as mixture when they are actually asking about mixtures in the second sense above, i.e., about compound-distributions.

The "disambiguation" section of the mixture tag wiki only disambiguates it from mixed models.

---

Any thoughts, comments or improvements on the tag wikis would be more than welcome.

Answer 1

The two meanings don't seem all that distinct. If, in the definition of the 1st sense of "mixture", you take probabilities as probability densities & sums as integrals, you get a mixture in the 2nd sense; if, in the definition of the 2nd sense, you allow $\theta$ to index a countable set of distributions, you get a mixture in the first sense. (And there seems to be a gap—what do you call it when $\theta$ indexes an uncountable set of distributions that don't constitute a parametric family?^†) At any rate the crux of the distinction needs to be made more explicit.

Even if compound distributions are more properly viewed as an extension of mixture distributions rather than as a sub-class, I wonder whether it mightn't be better to let mixture cover the general idea, & compound distribution the particular case—practically, to refrain from removing the mixture tag when adding the compound distribution tag.

† Wikipedia doesn't help: the article on mixture distributions defines compound distributions as those where $\theta$ indexes a set of uncountable distributions; while the article on compound distributions defines them as you do.