13
$\begingroup$

I don't know much about boosting, but I accidentally noticed that we have the following boosting-related tags:

The problem is that boosting (https://en.wikipedia.org/wiki/Gradient_boosting) is almost always used with decision trees, so [boosting] is almost exclusively about [gradient-boosted-trees]. Also, "GBM" stands for "gradient boosted machine" which was in the title of Friedman's famous 1999 paper... where he introduced boosted trees! So it's all the same thing.

Therefore, I think [gradient-boosted-trees] and [gbm] should become synonyms of [boosting]. This combined tag would have 486 threads. This seems to me to be a pretty clear-cut case.

In addition, we could consider merging the following tags:

into [boosting] as well. These tags are not that large, are clear subsets of [boosting], and whoever is using boosting in practice is most likely (?) using either xgboost or adaboost anyway. If we merge all the above-listed tags into [boosting], then it would have 613 threads. I am making this second suggestion because I generally prefer larger tags, but I don't have much of an informed opinion here.

$\endgroup$
  • 14
    $\begingroup$ Upvote if you agree with combining [gradient-boosted-trees], [gbm], and [boosting]. $\endgroup$ – amoeba says Reinstate Monica Oct 23 '17 at 13:40
  • 3
    $\begingroup$ Upvote if you disagree with combining [gradient-boosted-trees], [gbm], and [boosting]. $\endgroup$ – amoeba says Reinstate Monica Oct 23 '17 at 13:40
  • 3
    $\begingroup$ Upvote if you agree with merging xgboost/adaboost/etc into boosting as well. $\endgroup$ – amoeba says Reinstate Monica Oct 23 '17 at 13:40
  • 5
    $\begingroup$ Upvote if you disagree with merging xgboost/adaboost/etc into boosting. $\endgroup$ – amoeba says Reinstate Monica Oct 23 '17 at 13:41
  • 3
    $\begingroup$ (If you don't have an opinion then do not vote above.) $\endgroup$ – amoeba says Reinstate Monica Oct 23 '17 at 13:41
  • 3
    $\begingroup$ Maybe it would be better if those comment options were answers? $\endgroup$ – gung - Reinstate Monica♦ Oct 23 '17 at 15:04
  • 2
    $\begingroup$ I'm not sure it's right to say that, in practice, most people doing boosting would be using xgboost or adaboost. Certainly many people routinely use xgboost. But many people also use the GradientBoostingClassifier/GradientBoostingRegressor implementations in scikit-learn, which are not xgboost. And my impression is that while adaboost is still learned about a lot because of its historical significance, it may not be used that much in practice anymore. $\endgroup$ – Jake Westfall Oct 24 '17 at 9:09
  • $\begingroup$ Thanks @Jake, that's important. Looking at how people voted so far, I guess we can leave [xgboost] and [adaboost] alone. It's fine with me. $\endgroup$ – amoeba says Reinstate Monica Oct 24 '17 at 9:27
  • $\begingroup$ @gung I am not sure what's the most convenient way but I think I prefer comments for multiple-choice voting: it's much more compact. I would rather somebody more knowledgeable (than me) posted an answer with a specific unified suggestion; alternatively I can post an answer with a (hopefully) consensus suggestion after a couple of days of discussion and voting in the comments. $\endgroup$ – amoeba says Reinstate Monica Oct 24 '17 at 9:45
  • 1
    $\begingroup$ It's certainly sensible to prefer the most knowledgeable person to provide the answer, but in this case they seem to be just, "I agree" & "I disagree", which is less of an issue. Votes on answers can be examined & changed better, & it allows for comments / discussion about options more conveniently than this. That's why I suggested answers. $\endgroup$ – gung - Reinstate Monica♦ Oct 24 '17 at 11:35
  • 1
    $\begingroup$ @JakeWestfall I've stopped teaching AdaBoost. I prefer to just teach gradient boosting, it makes more sense and is still exceedingly useful. $\endgroup$ – Matthew Drury Oct 26 '17 at 2:42
16
$\begingroup$

After some discussion in the comments above, the emerging consensus seems to be:

  1. Map and to as synonyms.
  2. Leave and alone.

This plan looks good to me. Upvote if agree, downvote (and explain in the comments!) if disagree.

Update (Nov 5): This has been implemented.

$\endgroup$
  • 2
    $\begingroup$ If there is no substantial disagreement w/i the next, say, week, I will make the synonyms. $\endgroup$ – gung - Reinstate Monica♦ Oct 25 '17 at 11:58
  • 2
    $\begingroup$ I'm a pretty big boosting fanboy. I think this is correct. $\endgroup$ – Matthew Drury Oct 26 '17 at 4:15
  • $\begingroup$ Strong disagree. Gradient boosted trees and GBM are popular implementations of boosting, but are not synonymous with it. Even within xgboost, perhaps the most popular implementation of tree-based GBM, there are options for boosting with GLMs as the base learner instead of decision trees. $\endgroup$ – Paul Oct 30 '17 at 18:32
  • $\begingroup$ @Paul Thanks. Can you clarify: are you saying that all three tags ([g-b-t], [gbm], and [boosting]) should stay distinct, or is there any pair that you think can be combined? Apart from that, let me stress that for two tags to be made synonymous it is not necessary that the respective terms are exactly synonymous. We have to consider how strongly the topics overlap, and how useful it is to have tags separate, i.e. how useful it is for the site users to navigate these tags separately. You are saying that g-b-t and gbm are subsets of "boosting". [cont.] $\endgroup$ – amoeba says Reinstate Monica Oct 30 '17 at 20:42
  • $\begingroup$ [cont.] However, if these subsets are so larger that they cover most of the "boosting" usage, and if the usage on our site in practice is such that many questions are tagged with one tag or another without clear system that we can enforce, the it might very well make sense to merge them into the "parent" [boosting] tag. We often decide to make tags synonyms when the relationship between them is rather set/subset, but the distinction is blurry and/or subset covers most of the set. $\endgroup$ – amoeba says Reinstate Monica Oct 30 '17 at 20:44
  • $\begingroup$ @MatthewDrury FYI: See Paul's comment above. Would be good if you could explain your reasoning in more detail (I think you are our top user by rep in the boosting topic.) $\endgroup$ – amoeba says Reinstate Monica Oct 30 '17 at 20:46
  • 2
    $\begingroup$ Pretty much the same reasoning as yours above. Keeping them distinct is only meaningful if most of our users tag things correctly under this scheme. I'm not sure I was aware that gbm is used to mean a library, I always use it as a synonym for gradient-boosting. I'm a long time user of boosting, so if this is a misuse, it's probably a representative one. $\endgroup$ – Matthew Drury Oct 30 '17 at 21:14
  • $\begingroup$ Maybe I'm not understanding the user-facing implications. If this just means that users entering gbm for their tag get the tag automatically changed to boosting, then I think this is fine, since boosting is the most general term. On the other hand, equating the two would be a disservice to users IMO. It wouldn't be good to give the impression that boosting is reducible to GBM or gradient boosted trees. $\endgroup$ – Paul Oct 30 '17 at 21:47
  • $\begingroup$ @Paul Could you briefly explain to me the relationship between these terms? "Boosting" is a general term. Is all boosting "gradient boosting" or are there some other types of boosting? "Gradient boosted trees" means boosting done with trees (as opposed to other base learners), correct? And what is "gradient boosting machine" (GBM)? What relationship does it have with "gradient boosting"? I am not asking about the tags now, just clarifying the terminology in the field. $\endgroup$ – amoeba says Reinstate Monica Oct 30 '17 at 22:47
  • $\begingroup$ Gradient boosting is a particular type of boosting algorithm, probably the most widely used one. Adaboost is another, different one, older but also popular. I think gradient boosting and gradient boosting machine (GBM) are synonymous, but not 100% sure. Gradient boosted trees does indeed mean gradient boosting done with trees; other base learners could be used in gradient boosting. $\endgroup$ – Paul Oct 30 '17 at 23:37
  • $\begingroup$ @Paul Thanks. I think your concern could be addressed by supplying a carefully written tag excerpt for the [boosting] tag (assuming that the suggested synonyms are created). This excerpt should clearly say that boosting is a general term, with gradient boosting being the most widely used boosting algorithm and with trees being the most widely used base learners. The [gbm] and [gradient-boosted-trees] tags will be automatically changed to [boosting], but the very visible tag description will clarify the relationship between these terms. $\endgroup$ – amoeba says Reinstate Monica Oct 31 '17 at 20:53
  • $\begingroup$ @Paul (cont.) I edited the excerpt as follows: "A family of algorithms combining weakly predictive models into a strongly predictive model. The most common approach is called gradient boosting, and the most commonly used weak models are classification/regression trees." (stats.stackexchange.com/tags/boosting/info) Feel free to improve it. $\endgroup$ – amoeba says Reinstate Monica Nov 1 '17 at 0:23
  • 3
    $\begingroup$ @Paul I think you are a bit confused with how tag synonyms work. "x is a synonym of y" means that when someone tries to tag a post with x, the system lets them and then silently replaces the tag with y. Additionally, any time someone searches for x, instead they get results for y. We call y the "master tag" and x the "synonym tag." This differs a little from the way that the word "synonym" is used in English because it's directional. Although it's usually used for actual synonyms, such as "algorithm" and "algorithms," it is also used for the purpose in this thread. $\endgroup$ – Stella Biderman Nov 1 '17 at 13:17
  • $\begingroup$ @MatthewDrury Could you check the wiki excerpt that I wrote for [boosting] after some further discussion with Paul (see above)? $\endgroup$ – amoeba says Reinstate Monica Nov 2 '17 at 9:45
  • 1
    $\begingroup$ I think it's probably OK. I will make the synonyms, @amoeba. [gbm] & [g-b-t] are now synonyms of [boosting]. $\endgroup$ – gung - Reinstate Monica♦ Nov 5 '17 at 0:13

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .