23
$\begingroup$

We appear to have a number of overlapping threads on imbalanced data. Here are just a few of the more highly-upvoted ones:

Are unbalanced datasets problematic, and (how) does oversampling (purport to) help?

When is unbalanced data really a problem in Machine Learning?

Does an unbalanced sample matter when doing logistic regression?

What is the root cause of the class imbalance problem?

Some of these threads themselves link to and summarise even more questions and answers on the topic (there are 950 questions tagged , many of them popular). Stephan Kolassa's heroic effort to summarise and pin down the problem is probably the most comprehensive question on the topic, but there are useful answers strewn across many of these threads.

We still keep getting new questions about this, though, and IMO they often seem to attract misleading/incorrect answers. It would be useful to have a canonical thread to point people to on this, but I don't know enough about the topic to judge which of these could be merged/closed as duplicates/deemed canonical, so I thought I'd ask here. How can we best simplify this situation?

$\endgroup$
17
  • 3
    $\begingroup$ It is possible to close a question as a duplicate of not 1, but up to 5 other questions. It might be worth starting to close existing questions as dups of the top set. That will elevate them further. Then we start closing new Qs as dups. $\endgroup$
    – gung - Reinstate Monica
    Commented Jul 21, 2022 at 11:14
  • 2
    $\begingroup$ A place to start would be to identify one (or a handful) of canonical threads and tag them with faq. That enables a quick search whenever you encounter a similar post, streamlining the process of identifying a duplicate and closing the new thread. $\endgroup$
    – whuber Mod
    Commented Jul 21, 2022 at 13:35
  • $\begingroup$ @gung-ReinstateMonica: I think that would be a great way forward. Can only moderators close as a multi-dupe? Or how do mere mortals do that? $\endgroup$
    – Stephan Kolassa
    Commented Jul 21, 2022 at 17:22
  • 1
    $\begingroup$ @whuber: great idea. I concur with mkt that the four threads above would be a great place to start. I started adding the FAQ tag to all four, they already all carry the unbalanced-classes one. Question: the "FAQ" tag says it it applied by mods, but I could add it just fine. Should I have been prevented from doing so? If not, perhaps we could revisit the tag excerpt. $\endgroup$
    – Stephan Kolassa
    Commented Jul 21, 2022 at 17:26
  • 3
    $\begingroup$ @StephanKolassa, mere mortals can only close a thread as a dup of 1 other thread. After that, you can leave a comment with links to other dups, &/or you can flag for mods to add the others. $\endgroup$
    – gung - Reinstate Monica
    Commented Jul 21, 2022 at 17:33
  • 1
    $\begingroup$ We cannot enforce the applied-only-by-mods rule. The intention is to be sparing in our application of the faq tag and to monitor it to avoid abuses (such as someone applying it to their own threads or their favorite threads). $\endgroup$
    – whuber Mod
    Commented Jul 21, 2022 at 18:02
  • 1
    $\begingroup$ whuber notes that multi-dupe closing is a lot of work for the mods, and proposes to use a single dupe target, where we link to the others. I would suggest we use When is unbalanced data really a problem in Machine Learning? as a canonical dupe, and edit links to the other threads into that question, at the very top. Thoughts, anyone? $\endgroup$
    – Stephan Kolassa
    Commented Jul 21, 2022 at 19:17
  • $\begingroup$ @StephanKolassa Part of my concern was that no single question seems to have answers that address the topic comprehensively, so the information is diffused. My hope was that those of you who understood the topic better could identify threads that could be merged so that all bases were covered (and then we could designate it canonical). If that's not feasible, I agree with your suggestion. $\endgroup$
    – mkt
    Commented Jul 22, 2022 at 9:12
  • 1
    $\begingroup$ @mkt: I see your point. I think that this would be very hard, simply because there are so many variations of the "problem". Some come from using accuracy as a KPI, some don't mention their KPI at all and treat imbalance as a prima facie problem without explaining why it isn't. The, ahem, lack of statistical sophistication does not help, either ("but I'm using precision, not accuracy, so this is not a duplicate!") I think a comprehensive treatment would be more of a journal paper than a useful CV thread. ... $\endgroup$
    – Stephan Kolassa
    Commented Jul 22, 2022 at 9:41
  • 5
    $\begingroup$ ... Does anyone want to write a paper for The American Statistician or Chance, so we can petition SO to be able to close as duplicates of a journal paper? $\endgroup$
    – Stephan Kolassa
    Commented Jul 22, 2022 at 9:42
  • 1
    $\begingroup$ @StephanKolassa Alright, then I agree with your suggested target for a canonical thread, though perhaps we can ask Tim to broaden it a little to include statistics in the title. $\endgroup$
    – mkt
    Commented Jul 22, 2022 at 12:42
  • 1
    $\begingroup$ @StephanKolassa I think you are too modest. Your effort to which the OP links approvingly seems to me a good target. $\endgroup$
    – mdewey
    Commented Jul 22, 2022 at 13:07
  • $\begingroup$ I just saw this answer for the first time, and I didn't see it referenced anywhere in here. // @StephanKolassa It's awfully tempting to write something for The American Statistician... $\endgroup$
    – Dave
    Commented Aug 11, 2022 at 5:15
  • 1
    $\begingroup$ And it’s even coming up on the economics Stack… $\endgroup$
    – Dave
    Commented Sep 13, 2022 at 3:07
  • 1
    $\begingroup$ These questions are accumulating a large number of Linked Questions, so that the ones displayed in the right pane are subsetted. It would be really nice if the ones selected for display were customizable (sorted by votes e.g.). Barring that SE team solution, I'd suggest cross-referencing them prominently in each question (or maybe linking here?). $\endgroup$
    – Ben Reiniger
    Commented Oct 1, 2022 at 16:09

7 Answers 7

Reset to default
7
$\begingroup$

In the interest of taking a decision, I will post a few answers for you to vote on. Please feel free to post more.

Option 1: Do nothing. First do no harm, etc etc.

$\endgroup$
2
  • $\begingroup$ To be clear, this option means that new questions would be closed as duplicate of one of the four (or possibly another), depending on nuances between them as perceived by the close-voters? $\endgroup$
    – Ben Reiniger
    Commented Aug 3, 2022 at 17:59
  • $\begingroup$ @BenReiniger That's probably the best-case scenario, yes. $\endgroup$
    – mkt
    Commented Aug 3, 2022 at 18:17
3
$\begingroup$

Option 7: Post a question that we answer with the usual links about this topic along with some commentary about the links (like an annotated bibliography). This then serves as the go-to duplicate when we vote to close.

This seems to have the advantage of directing askers to the multiple good resources on this topic, yet it lacks the drawbacks of multi-link duplicates (only available to the diamond moderators, and it’s an ordeal for them to do multi-link duplicates, anyway).

I would be content to start this with a question and self-answer that addresses some of my favorite links (e.g., the ones currently in my profile). If we want to make this a community wiki for those who care to add their favorite links and give some commentary, that could make sense, either as part of the self-answer I post or as separate answers.

$\endgroup$
0
$\begingroup$

In the interest of taking a decision, I will post a few answers for you to vote on. Please feel free to post more.

Option 5: Merge some subset of the threads in the question, and close others as duplicates of this new canonical thread.

If you choose this option, please specify which threads you think should be merged in a comment.

$\endgroup$
2
  • 2
    $\begingroup$ This has some appeal, but I'm not sure how possible it is to collapse all the nuances into any 1 of those Qs. $\endgroup$
    – gung - Reinstate Monica
    Commented Jul 25, 2022 at 21:20
  • $\begingroup$ @gung-ReinstateMonica Yes, I am coming around to this view. 3 of the 4 questions I mention cover very similar ground (except for the one about logistic regression) but the difference in framing and detail are quite useful. $\endgroup$
    – mkt
    Commented Jul 26, 2022 at 5:23
0
$\begingroup$

In the interest of taking a decision, I will post a few answers for you to vote on. Please feel free to post more.

Option 2: Vote to close threads as duplicate of When is unbalanced data really a problem in Machine Learning?.

*Not just new threads, but also the other threads in the question.

$\endgroup$
0
$\begingroup$

In the interest of taking a decision, I will post a few answers for you to vote on. Please feel free to post more.

Option 3: Vote to close threads as duplicate of Are unbalanced datasets problematic, and (how) does oversampling (purport to) help?.

*Not just new threads, but also the other threads in the question.

$\endgroup$
0
$\begingroup$

In the interest of taking a decision, I will post a few answers for you to vote on. Please feel free to post more.

Option 4: Vote to close threads as duplicate of What is the root cause of the class imbalance problem?.

*Not just new threads, but also the other threads in the question.

$\endgroup$
-4
$\begingroup$

In the interest of taking a decision, I will post a few answers for you to vote on. Please feel free to post more.

Option 6: Close new questions as duplicates of all 4 threads in the question

Note that this seems to be a lot more work for the mods; I think it is best avoided for that reason.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .