5
$\begingroup$

We have a ×254 tag but no tag for hurdle models. I have just created ×1. Now I am wondering if it should better be made a synonym of [zero-inflation] or left alone as a separate tag.

According to What is the difference between zero-inflated and hurdle distributions (models)?, these two models are meaningfully distinct. However, most of the time they are supposed to deal with the same issue. It seems that the two models should better be assigned to a common "umbrella" tag. The question is: can [zero-inflation] surve as such an umbrella tag DESPITE "zero-inflated model" and "hurdle model" being two different things? Can we perhaps view "zero inflation" as an umbrella name for the underlying issue?

The current wiki excerpt for [zero-inflation] reads:

Variables that are counts (non-negative integers) often have an excess of zeroes compared to a certain count distribution. Zero-inflated regression models (e.g. zero inflated Poisson, zero inflated negative binomial) are designed to deal with this. Less commonly, continuous data can have this issue, and there is zero-inflated normal regression to deal with that situation.

If we make [hurdle-model] a synonym, we could modify it e.g. like this:

Excess of zeroes in the data compared to a certain distribution that otherwise describes the data well. Regression approaches for count data include zero-inflated Poisson or negative binomial GLMs, and hurdle models.

Additional question: there also is a ×14 tag without an excerpt, that is probably supposed to be used for semi-continuous distributions with point mass at zero. I could write such an excerpt, but shouldn't we perhaps make it a synonym of [zero-inflation] too? After all, a point mass at zero does yield "an excess of zeros" compared to any continuous distribution. And the current excerpt of [zero-inflation] even mentions zero-inflated continuous models.

| |
$\endgroup$
  • 5
    $\begingroup$ I'd be fine with unbrella tag. I'd argue that the main pourpose of the tags is to be convenient rather then precise. $\endgroup$ – Tim♦ Nov 10 '17 at 11:46
  • 2
    $\begingroup$ I don't think they are synonyms. Zero-inflation is a characteristic of a variable. Hurdle models are one attempt to deal with that. And hurdle models could (at least in theory) be used for other kinds of inflation. And I agree with you about point mass at zero - I think that ought to refer to nearly continuous variables that are never negative and can have a mass at 0. $\endgroup$ – Peter Flom Nov 10 '17 at 12:36
  • 2
    $\begingroup$ @PeterFlom OK, but we use [zero-inflation] tag e.g. for zero-inflated Poisson regression, don't we? Even though it's also one model that attempts to deal with "zero inflation"? $\endgroup$ – amoeba Nov 10 '17 at 12:38
  • $\begingroup$ I guess it gets into a question of how many tags we want to have. ZIP is not a synonym of "zero-inflated" in the way synonym is usually used in English. But synonym seems to be used a little differently here. What are the arguments for "lots of tags" vs "only a few"? (Also, are there guidelines to this sort of stuff somewhere?). $\endgroup$ – Peter Flom Nov 10 '17 at 12:43
  • 4
    $\begingroup$ The generic term seems to be clumping but I suspect many questioners will not recognise it. I think it is fine to keep zero-inflated to refer to any sort of clumping at zero and not try to have too many more precise tags. $\endgroup$ – mdewey Nov 10 '17 at 13:08
  • 1
    $\begingroup$ @PeterFlom Yes, "tag synonym" is not necessarily a real synonym. Sometimes we decide to combine several related tags into one "umbrella tag". For example we have [combining-p-values] tag, and [fishers-method] is a tag synonym; even though it's only one of the methods to combine p-values. There are other examples as well. I don't think we really have guidelines for that written anywhere; but I feel that there is some vague consensus, at least among the people who are usually discussing tag-related issues on Meta. If you want, you could start a Meta thread to discuss this point. $\endgroup$ – amoeba Nov 10 '17 at 14:05
4
$\begingroup$

Thank you for the good discussion so far, and thanks to @amoeba for pointing me to it! I agree with many points raised so far and just want to share a few personal experiences with this topic.

General scope: I always interpreted the tag [zero-inflation] as being about the phenomenon along with various models for it. Hence, I didn't expect only questions about so-called zero-inflated models but also so-called hurdle models. Due to the close connections between properties of the data and properties of the different models, I prefer a broad tag over several narrower tags.

Count vs. continuous: While most questions are about count data with zero-inflation, I'm personally quite happy to also have questions about continuous data with zero inflation (point mass), e.g., the tobit model (censored normal) or the Cragg model (hurdle or two-part model: probit + truncated normal). Again, there are many parallels between these continuous and the discrete models, so that we could get useful spillover effects from discussing these topics under the same tag.

Jargon: Distinguishing "zero-inflation" and "hurdle" into different tags, is not only difficult because the corresponding count regression models are so similar. It is also difficult because jargon is not unified. For example, large parts of the relevant literature talks about "zero-inflated beta regression". However, this is not entirely consistent: The beta distribution does not have any zeros and hence these cannot be inflated. Instead calling it a "two-part" or "zero hurdle" beta regression model would be better. But this is not what the literature does. Hence, we would have difficulties cleanly separating it here.

Recommendation: My personal preference would be to

  1. Use an extended description of the [zero-inflation] tag to say that it is chiefly about count data but also continuous data. And provide references to related tags such as [tobit-regression].
  2. Additionally create a tag [count-regression] in addition to the existing [count-data] tag. Or alternatively extend the description of the latter tag to include count regression models.
  3. Either omit the [hurdle-model] tag or improve it to explicitly include the term "two-part model" which is more popular for certain kinds of hurdle models.

But not all of these preferences are very strong. I could live very well with some of the other solutions posted previously!

| |
$\endgroup$
  • $\begingroup$ Thanks for answering! Very good points in the beginning. Re your suggestions: (1) Based on gung's draft and some of your points, I would suggest the following excerpt for the [zero-inflation] tag: Excess of zeroes in the data compared to a specified distribution. Regression approaches include zero-inflated & hurdle aka two-part models. For count data, zero-inflated Poisson (ZIP) or negative binomial (ZINB) are common. Improvements welcome. We could then put links to related tags in the full tag wiki. $\endgroup$ – amoeba Nov 17 '17 at 9:05
  • $\begingroup$ (2) I am against [count-regression]. We already have [poisson-regression] and people have been using [negative-binomial] for NB regression, so creating [count-regression] now will leave a huge number of existing threads without this tag. I don't see a point. But we could extend the excerpt of [count-data], yes. $\endgroup$ – amoeba Nov 17 '17 at 9:07
  • $\begingroup$ (3) If we settle on the "broad" scope of [zero-inflation] tag as per your answer, then I think [hurdle-model] should be made a tag synonym. $\endgroup$ – amoeba Nov 17 '17 at 9:07
  • $\begingroup$ Good point re [poisson-regression] and [negative-binomial]. Personally, I'm not fond of distinguishing the two because they are so very closely related. But I think it would help to point out that [count-data] can be coupled with either [poisson-regression] or [negative-binomial]. This wouldn't involve major changes but help to create spillover effects between the tags. $\endgroup$ – Achim Zeileis Nov 17 '17 at 10:59
  • 3
    $\begingroup$ Proposed adaptation of tag description for [zero-inflation]: Excess of zeros in a variable compared to a specified reference distribution. Regression approaches include zero-inflated models and hurdle aka two-part models. For count data, zero-inflated and hurdle models based on Poisson or negative binomial distributions are common (ZIP/ZINB and HP/HNB). The HP/HNB labels are far less common than ZIP/ZINB but for symmetry reasons I would still include them. $\endgroup$ – Achim Zeileis Nov 17 '17 at 11:06
  • $\begingroup$ Gung updated the tag description using your wording (slightly shortened), and made the hurdle-models the synonym of zero-inflation. I mark your answer as accepted! Thanks. $\endgroup$ – amoeba Nov 18 '17 at 12:32
  • $\begingroup$ Thanks for this - and for going through my contributions on CV - very much appreciated. I'm glad I could contribute a little bit back... $\endgroup$ – Achim Zeileis Nov 19 '17 at 0:07
4
$\begingroup$

I usually agree with @gung on tag issues, but this time I am not convinced by his answer. I want to make two points.

I don't see how we can meaningfully separate tags for zero-inflated count models, hurdle count models, and zero-inflated-aka-hurdle continuous models. Because: (A) Zero-inflated and hurdle count models are clearly distinct but very similar and very related. I think they should be grouped together in one tag. (B) For continuous models, there is no difference between zero-inflated and hurdle. I googled a bit, and I see that people talk about "zero-inflated gamma" and "hurdle gamma" models (and the same for lognormal). That's the same thing.

Hence, my point #1: we need one tag for [zero-inflated-and-hurdle-count-and-continous-models]. Let's say we call it .

@Gung suggests to have separately from the model tag(s). I am not sure I see the benefit. 100% of questions in the former tag would fit to the latter tag, and I think that most (90%?) of the questions in the latter would fit to the former.

Hence my point #2: let's just use for all of that.

Yes, there are other models to deal with zero-inflated data. E.g. or . These tags should clearly stay separate. I admit that there is some conceptual blurriness here, but this currently seems to me to be the most reasonable approach.

(On reflection, I would get rid of altogether because it seems to be used for other things as well, such as priors with point mass at zero.)

| |
$\endgroup$
  • 2
    $\begingroup$ I mostly agree w/ this. I'm fine w/ [zero-inflated-and-hurdle-models], eg, although it seems wordy to me. We could eliminate p-m-a-z & fold that into z-i. We can also stay w/ z-i if you like. The suggestion of z-i-d was just about making it clearer / more distinct that the tag refers to the phenomenon of zero-inflation, & not one particular (albeit popular) strategy to address that phenomenon. Eg, I don't think we should make weighted-regression a synonym of heteroscedasticity, likewise I don't think we should make instrumental-variables a synonym of endogeneity, etc. $\endgroup$ – gung - Reinstate Monica Nov 11 '17 at 3:15
  • $\begingroup$ [zero-inflation] is fine but [zero-inflated-and-hurdle-models] seems way too cumbersome... $\endgroup$ – usεr11852 Nov 11 '17 at 13:26
  • $\begingroup$ @gung I see your point. How about we rename a tag into something that makes it clear that it's about ZI/hurdle models and not about the phenomenon (and get rid of the phenomenon tag at all)? [zero-inflated-and-hurdle-models] is clearly an awful name. Maybe just [zero-inflated-model]? And we write in the excerpt that this tag should be used for hurdle models too (and make the corresponding synonym). I've been looking at [zero-inflated] Qs to get some empirical evidence about how this tag is currently used and most of the threads are about ZI/hurdle models (both count and continuous). ctd $\endgroup$ – amoeba Nov 12 '17 at 0:21
  • $\begingroup$ [cont.] There is indeed a small but non-trivial fraction of threads that arguably have nothing to do with any ZI-models (e.g. "What is effect on PCA of having too many zeros in the data?" or "Explanatory variables with many zeros"). Then we could remove this tag from such Qs. $\endgroup$ – amoeba Nov 12 '17 at 0:21
  • $\begingroup$ Could we do [models-for-0-inflation] (or something like that), & then have [zero-inflated-data]? We could eliminate the current tag, & create additional tags, [zero-inflated-model] & [hurdle-model], that become synonyms of m-f-0-i. $\endgroup$ – gung - Reinstate Monica Nov 12 '17 at 1:35
  • $\begingroup$ @gung Well, I am concerned about the tag overlap. I understand the difference between models and data that you want to preserve, but if the tags (one for models and one for data) overlap more than X% then it doesn't make sense to have both of them. I don't know what is the threshold value of X, but in this case I estimate overlap of at least 90% and that should IMHO be above any threshold. (I could post some statistics based on searches that I did if you want.) That's why I am now thinking that maybe we don't need any tag for the zi-data. $\endgroup$ – amoeba Nov 12 '17 at 10:03
  • $\begingroup$ @gung An alternative that is perhaps also fine is not to do anything really. Then I would remove [hurdle-model] tag that I created and would manually get rid of [point-mass-at-zero]. [Zero-inflation] stays the tag about the phenomenon and we use it for all the zi/hurdle models which is the status quo. It's just weird that under this scheme we don't have any tag or tag synonym about hurdle-models. So if somebody types "hurdle", nothing will pop up. $\endgroup$ – amoeba Nov 12 '17 at 10:06
  • $\begingroup$ I need to think about the overlap issue. We could change the excerpt to mention hurdle models. $\endgroup$ – gung - Reinstate Monica Nov 13 '17 at 1:57
  • $\begingroup$ What if we made the excerpt read: Count variables often have excess zeroes compared to a specified count distribution. Less commonly, continuous data can have this occur. Use this tag for this issue & for approaches to deal w/ it (eg, ZIP & hurdle models).? $\endgroup$ – gung - Reinstate Monica Nov 16 '17 at 2:33
  • 1
    $\begingroup$ @gung See Achim's answer here, he makes some good points (I pinged him to share his opinion because he has by far the most answers in the [zero-inflation] tag). Taking some of his points into account, I modified your suggested excerpt as follows: Excess of zeroes in the data compared to a specified distribution. Regression approaches include zero-inflated & hurdle aka two-part models. For count data, zero-inflated Poisson (ZIP) and negative binomial (ZINB) are common. What do you think? $\endgroup$ – amoeba Nov 17 '17 at 9:02
  • $\begingroup$ @gung See also Achim's edit in the comments under his answer. $\endgroup$ – amoeba Nov 17 '17 at 11:16
  • $\begingroup$ I'm happy w/ his proposed excerpt as the solution, if that's fine w/ everyone else. $\endgroup$ – gung - Reinstate Monica Nov 17 '17 at 14:15
  • $\begingroup$ Sounds good to me @gung. Should we (you) make the hurdle-models the synonym of zero-inflation then? I could manually take care of the several point-mass-at-zero threads later. $\endgroup$ – amoeba Nov 17 '17 at 16:55
  • 1
    $\begingroup$ I changed the excerpt to @Achim's version (I shortened it slightly). h-m & p-m-a-z have both been made into synonyms of z-i & merged. I deleted p-m-a-z. So it's all done now. There is no full wiki for z-i, however. We could work up something comprehensive to put there. We could also work on adding the tag to the appropriate threads where it's missing. $\endgroup$ – gung - Reinstate Monica Nov 17 '17 at 17:21
  • 1
    $\begingroup$ I think you had asked me to weigh in on this, but you said everything I'd have wanted to say, but better. I'm generally in favor of having fewer tags, so what you settled on with Achim's answer works great for me. $\endgroup$ – shadowtalker Nov 18 '17 at 17:13
1
$\begingroup$

There is "zero-inflation" the phenomenon, and there are zero-inflated-(Poisson / negative binomial / etc.) models as one strategy to deal with that phenomenon (in addition to hurdle models and possibly other strategies). I interpret the tag to be about the phenomenon. The excerpt should be clarified to eliminate the conflation of phenomenon with one possible remedy. Here is a possible edit:

Count variables often have excess zeroes compared to a specified count distribution. Less commonly, continuous data can have this issue, use [point-mass-at-zero] for that.

We could further create tags for various existing approaches to dealing with the phenomenon, such as [zero-inflated-poisson], [zero-inflated-neg-bin], [hurdle-model]. I do wonder how well so many specific tags will fare in practice, though. My guess is that a question could be tagged with [zero-inflated] and [poisson-regression], e.g., and do just fine. That is somewhat inconsistent with having [hurdle-model] as a stand-alone tag, but I think it may be a workable compromise until there are sufficient threads to merit revisiting the issue. Or not, I'm about 50-50 on this...

Another possibility is to change the existing tag from [zero-inflation] to [zero-inflated-data], and then have a generic [zero-inflated-model]. That would be less open to error. My basic suggestion here is to add / clarify the excerpts and otherwise leave it alone as much as possible.

To answer your specific questions:

  1. I wouldn't make [hurdle-model] a synonym of [zero-inflation]

    But we should clarify the excerpt for [zero-inflation] as pertaining to the phenomenon.

  2. I wouldn't make [point-mass-at-zero] a synonym of [zero-inflation]

    But we should create a good excerpt for it.

| |
$\endgroup$
  • 2
    $\begingroup$ Hmm. Having [zero-inflation] and [zero-inflated-poisson]/[zero-inflated-neg-bin]/etc appears to me to be definitely a bad solution. To be fully consistent we would either need to get rid of [zero-inflation] altogether and sort it into more specific tags (you don't suggest that and I agree that it's a too fine splitting) or clump everything together into one umbrella tag (my suggestion)... Of course we don't need to be fully consistent. $\endgroup$ – amoeba Nov 10 '17 at 17:03
  • $\begingroup$ Another thing is that one can have zero-inflated positive-continuous data (e.g. zero-inflated gamma or something), in which case as far as I understand there is no difference anymore between hurdle+gamma or zero-inflated-gamma... No idea if this is used in practice or not. $\endgroup$ – amoeba Nov 10 '17 at 17:04
  • $\begingroup$ [cont.] and gamma+point-mass-on-zero would also be the same... $\endgroup$ – amoeba Nov 10 '17 at 17:14
  • 1
    $\begingroup$ @amoeba, another possibility is to change the existing tag to [z-i-data] & then have a generic [z-i-model]. That would be less open to error. My basic suggestion here is to add / clarify the excerpts & otherwise leave it alone. $\endgroup$ – gung - Reinstate Monica Nov 10 '17 at 17:49
  • $\begingroup$ I am not sure I understand what we would gain by separating z-i-data and z-i-model. Whoever has to deal with z-i-data will necessarily be looking for z-i-model. I think they go hand in hand. Could you explain a bit more what exactly you don't like about merging this all together into one umbrella tag? Do you think it would be too unspecific? I mean, we did not even have the hurdle-model tag until today, so it's not like it's a flourishing tag that should be left alone... $\endgroup$ – amoeba Nov 10 '17 at 20:32
  • $\begingroup$ @amoeba, if you want to ditch the [hurdle-model] tag, that's fine. I think a tag for the phenomenon of having too many 0's in your (count) data, & a tag for the phenomenon w/ continuous data, both w/ clear excerpts, is what's needed. If you want tags to address things like Qs about zi-models vs hurdle models, we would need additional & more fine-grained tags. But I could live w/o them just as easily. $\endgroup$ – gung - Reinstate Monica Nov 10 '17 at 20:45
  • $\begingroup$ (cont.) I don't think it's helpful to have the phenomenon of z-i conflated w/ the potential remedy of zi-models. When someone "has to deal with z-i-data" they aren't necessarily "looking for z-i-model". They might be looking for a hurdle model, or an ordinal regression, or something else. $\endgroup$ – gung - Reinstate Monica Nov 10 '17 at 20:46
  • $\begingroup$ When someone "has to deal with z-i-data" they aren't necessarily "looking for z-i-model". They might be looking for a hurdle model - that's why I wanted to make it a synonym :-) But of course if there are even more possible remedies (you mentioned ordinal regression) then it stops making sense to merge everything... One reason to create hurdle-model tag was that I found it strange that we don't have it, so I thought I would create it and we'd make it a synonym of something. More as a quick fix to the current situation. $\endgroup$ – amoeba Nov 10 '17 at 20:54
  • $\begingroup$ Regarding continuous data: I searched for "zero inflated gamma" (stats.stackexchange.com/search?q=zero+inflated+gamma) and I find quite some questions (some talk about "gamma hurdle"). Most of them have [zero-inflation] tag. But that's about continuous data. What would you do about that? $\endgroup$ – amoeba Nov 10 '17 at 20:58
  • $\begingroup$ @amoeba, it's potentially OK w/ me to have h-m as a synonym for z-i, & then to also create z-i-m & make it a synonym as well. But I'm a little leery of conflating a phenomenon w/ a potential method to address it. I think that's conceptually confused. Alternatively, just have z-i & p-m-a-z (delete h-m). Re gamma, I think Q's about continuous variables would be better retagged as p-m-a-z. $\endgroup$ – gung - Reinstate Monica Nov 10 '17 at 21:03
  • $\begingroup$ I am just worried that people thinking about "hurdle gamma" or "zero-inflated gamma" will not think about using p-m-a-z tag. Let's see: zero inflated gamma is:question yields 19 results, hurdle gamma is:question yields 9 results, [point-mass-at-zero] gamma only 4. Actually, now that I look at it, [point-mass-at-zero] is a diverse tag, people use it e.g. for continuous priors with point mass at zero, which is a different context I would say... $\endgroup$ – amoeba Nov 10 '17 at 21:08
  • 1
    $\begingroup$ I tried to summarize my current point of view in an answer that I just posted. Thanks a lot for this discussion. $\endgroup$ – amoeba Nov 10 '17 at 23:41
0
$\begingroup$

If the distribution is continuous you might add a 0 point mass but this is hard to model in a regression framework. One alternative is to use a Tweedie distribution with $1 \leq p \leq 2$ which is continuous and has a point mass at 0. If the tag is going to mention anything about continuous distributions I'd recommend mentioning the Tweedie as an option.

| |
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .