13
$\begingroup$

I don't know much about boosting, but I accidentally noticed that we have the following boosting-related tags:

The problem is that boosting (https://en.wikipedia.org/wiki/Gradient_boosting) is almost always used with decision trees, so [boosting] is almost exclusively about [gradient-boosted-trees]. Also, "GBM" stands for "gradient boosted machine" which was in the title of Friedman's famous 1999 paper... where he introduced boosted trees! So it's all the same thing.

Therefore, I think [gradient-boosted-trees] and [gbm] should become synonyms of [boosting]. This combined tag would have 486 threads. This seems to me to be a pretty clear-cut case.

In addition, we could consider merging the following tags:

into [boosting] as well. These tags are not that large, are clear subsets of [boosting], and whoever is using boosting in practice is most likely (?) using either xgboost or adaboost anyway. If we merge all the above-listed tags into [boosting], then it would have 613 threads. I am making this second suggestion because I generally prefer larger tags, but I don't have much of an informed opinion here.

$\endgroup$
11
  • 14
    $\begingroup$ Upvote if you agree with combining [gradient-boosted-trees], [gbm], and [boosting]. $\endgroup$
    – amoeba
    Oct 23, 2017 at 13:40
  • 3
    $\begingroup$ Upvote if you disagree with combining [gradient-boosted-trees], [gbm], and [boosting]. $\endgroup$
    – amoeba
    Oct 23, 2017 at 13:40
  • 3
    $\begingroup$ Upvote if you agree with merging xgboost/adaboost/etc into boosting as well. $\endgroup$
    – amoeba
    Oct 23, 2017 at 13:40
  • 5
    $\begingroup$ Upvote if you disagree with merging xgboost/adaboost/etc into boosting. $\endgroup$
    – amoeba
    Oct 23, 2017 at 13:41
  • 3
    $\begingroup$ (If you don't have an opinion then do not vote above.) $\endgroup$
    – amoeba
    Oct 23, 2017 at 13:41
  • 3
    $\begingroup$ Maybe it would be better if those comment options were answers? $\endgroup$
    – gung - Reinstate Monica
    Oct 23, 2017 at 15:04
  • 2
    $\begingroup$ I'm not sure it's right to say that, in practice, most people doing boosting would be using xgboost or adaboost. Certainly many people routinely use xgboost. But many people also use the GradientBoostingClassifier/GradientBoostingRegressor implementations in scikit-learn, which are not xgboost. And my impression is that while adaboost is still learned about a lot because of its historical significance, it may not be used that much in practice anymore. $\endgroup$
    – Jake Westfall
    Oct 24, 2017 at 9:09
  • $\begingroup$ Thanks @Jake, that's important. Looking at how people voted so far, I guess we can leave [xgboost] and [adaboost] alone. It's fine with me. $\endgroup$
    – amoeba
    Oct 24, 2017 at 9:27
  • $\begingroup$ @gung I am not sure what's the most convenient way but I think I prefer comments for multiple-choice voting: it's much more compact. I would rather somebody more knowledgeable (than me) posted an answer with a specific unified suggestion; alternatively I can post an answer with a (hopefully) consensus suggestion after a couple of days of discussion and voting in the comments. $\endgroup$
    – amoeba
    Oct 24, 2017 at 9:45
  • 1
    $\begingroup$ It's certainly sensible to prefer the most knowledgeable person to provide the answer, but in this case they seem to be just, "I agree" & "I disagree", which is less of an issue. Votes on answers can be examined & changed better, & it allows for comments / discussion about options more conveniently than this. That's why I suggested answers. $\endgroup$
    – gung - Reinstate Monica
    Oct 24, 2017 at 11:35
  • 1
    $\begingroup$ @JakeWestfall I've stopped teaching AdaBoost. I prefer to just teach gradient boosting, it makes more sense and is still exceedingly useful. $\endgroup$
    – Matthew Drury
    Oct 26, 2017 at 2:42

1 Answer 1

Reset to default
16
$\begingroup$

After some discussion in the comments above, the emerging consensus seems to be:

  1. Map and to as synonyms.
  2. Leave and alone.

This plan looks good to me. Upvote if agree, downvote (and explain in the comments!) if disagree.

Update (Nov 5): This has been implemented.

$\endgroup$
16
  • 2
    $\begingroup$ If there is no substantial disagreement w/i the next, say, week, I will make the synonyms. $\endgroup$
    – gung - Reinstate Monica
    Oct 25, 2017 at 11:58
  • 2
    $\begingroup$ I'm a pretty big boosting fanboy. I think this is correct. $\endgroup$
    – Matthew Drury
    Oct 26, 2017 at 4:15
  • $\begingroup$ Strong disagree. Gradient boosted trees and GBM are popular implementations of boosting, but are not synonymous with it. Even within xgboost, perhaps the most popular implementation of tree-based GBM, there are options for boosting with GLMs as the base learner instead of decision trees. $\endgroup$
    – Paul
    Oct 30, 2017 at 18:32
  • $\begingroup$ @Paul Thanks. Can you clarify: are you saying that all three tags ([g-b-t], [gbm], and [boosting]) should stay distinct, or is there any pair that you think can be combined? Apart from that, let me stress that for two tags to be made synonymous it is not necessary that the respective terms are exactly synonymous. We have to consider how strongly the topics overlap, and how useful it is to have tags separate, i.e. how useful it is for the site users to navigate these tags separately. You are saying that g-b-t and gbm are subsets of "boosting". [cont.] $\endgroup$
    – amoeba
    Oct 30, 2017 at 20:42
  • $\begingroup$ [cont.] However, if these subsets are so larger that they cover most of the "boosting" usage, and if the usage on our site in practice is such that many questions are tagged with one tag or another without clear system that we can enforce, the it might very well make sense to merge them into the "parent" [boosting] tag. We often decide to make tags synonyms when the relationship between them is rather set/subset, but the distinction is blurry and/or subset covers most of the set. $\endgroup$
    – amoeba
    Oct 30, 2017 at 20:44
  • $\begingroup$ @MatthewDrury FYI: See Paul's comment above. Would be good if you could explain your reasoning in more detail (I think you are our top user by rep in the boosting topic.) $\endgroup$
    – amoeba
    Oct 30, 2017 at 20:46
  • 2
    $\begingroup$ Pretty much the same reasoning as yours above. Keeping them distinct is only meaningful if most of our users tag things correctly under this scheme. I'm not sure I was aware that gbm is used to mean a library, I always use it as a synonym for gradient-boosting. I'm a long time user of boosting, so if this is a misuse, it's probably a representative one. $\endgroup$
    – Matthew Drury
    Oct 30, 2017 at 21:14
  • $\begingroup$ Maybe I'm not understanding the user-facing implications. If this just means that users entering gbm for their tag get the tag automatically changed to boosting, then I think this is fine, since boosting is the most general term. On the other hand, equating the two would be a disservice to users IMO. It wouldn't be good to give the impression that boosting is reducible to GBM or gradient boosted trees. $\endgroup$
    – Paul
    Oct 30, 2017 at 21:47
  • $\begingroup$ @Paul Could you briefly explain to me the relationship between these terms? "Boosting" is a general term. Is all boosting "gradient boosting" or are there some other types of boosting? "Gradient boosted trees" means boosting done with trees (as opposed to other base learners), correct? And what is "gradient boosting machine" (GBM)? What relationship does it have with "gradient boosting"? I am not asking about the tags now, just clarifying the terminology in the field. $\endgroup$
    – amoeba
    Oct 30, 2017 at 22:47
  • $\begingroup$ Gradient boosting is a particular type of boosting algorithm, probably the most widely used one. Adaboost is another, different one, older but also popular. I think gradient boosting and gradient boosting machine (GBM) are synonymous, but not 100% sure. Gradient boosted trees does indeed mean gradient boosting done with trees; other base learners could be used in gradient boosting. $\endgroup$
    – Paul
    Oct 30, 2017 at 23:37
  • $\begingroup$ @Paul Thanks. I think your concern could be addressed by supplying a carefully written tag excerpt for the [boosting] tag (assuming that the suggested synonyms are created). This excerpt should clearly say that boosting is a general term, with gradient boosting being the most widely used boosting algorithm and with trees being the most widely used base learners. The [gbm] and [gradient-boosted-trees] tags will be automatically changed to [boosting], but the very visible tag description will clarify the relationship between these terms. $\endgroup$
    – amoeba
    Oct 31, 2017 at 20:53
  • $\begingroup$ @Paul (cont.) I edited the excerpt as follows: "A family of algorithms combining weakly predictive models into a strongly predictive model. The most common approach is called gradient boosting, and the most commonly used weak models are classification/regression trees." (stats.stackexchange.com/tags/boosting/info) Feel free to improve it. $\endgroup$
    – amoeba
    Nov 1, 2017 at 0:23
  • 3
    $\begingroup$ @Paul I think you are a bit confused with how tag synonyms work. "x is a synonym of y" means that when someone tries to tag a post with x, the system lets them and then silently replaces the tag with y. Additionally, any time someone searches for x, instead they get results for y. We call y the "master tag" and x the "synonym tag." This differs a little from the way that the word "synonym" is used in English because it's directional. Although it's usually used for actual synonyms, such as "algorithm" and "algorithms," it is also used for the purpose in this thread. $\endgroup$
    – Stella Biderman
    Nov 1, 2017 at 13:17
  • $\begingroup$ @MatthewDrury Could you check the wiki excerpt that I wrote for [boosting] after some further discussion with Paul (see above)? $\endgroup$
    – amoeba
    Nov 2, 2017 at 9:45
  • 1
    $\begingroup$ I think it's probably OK. I will make the synonyms, @amoeba. [gbm] & [g-b-t] are now synonyms of [boosting]. $\endgroup$
    – gung - Reinstate Monica
    Nov 5, 2017 at 0:13

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .