13
$\begingroup$

We appear to have a number of overlapping threads on imbalanced data. Here are just a few of the more highly-upvoted ones:

Are unbalanced datasets problematic, and (how) does oversampling (purport to) help?

When is unbalanced data really a problem in Machine Learning?

Does an unbalanced sample matter when doing logistic regression?

What is the root cause of the class imbalance problem?

Some of these threads themselves link to and summarise even more questions and answers on the topic (there are 950 questions tagged , many of them popular). Stephan Kolassa's heroic effort to summarise and pin down the problem is probably the most comprehensive question on the topic, but there are useful answers strewn across many of these threads.

We still keep getting new questions about this, though, and IMO they often seem to attract misleading/incorrect answers. It would be useful to have a canonical thread to point people to on this, but I don't know enough about the topic to judge which of these could be merged/closed as duplicates/deemed canonical, so I thought I'd ask here. How can we best simplify this situation?

$\endgroup$
13
  • 3
    $\begingroup$ It is possible to close a question as a duplicate of not 1, but up to 5 other questions. It might be worth starting to close existing questions as dups of the top set. That will elevate them further. Then we start closing new Qs as dups. $\endgroup$ Jul 21 at 11:14
  • 2
    $\begingroup$ A place to start would be to identify one (or a handful) of canonical threads and tag them with faq. That enables a quick search whenever you encounter a similar post, streamlining the process of identifying a duplicate and closing the new thread. $\endgroup$
    – whuber Mod
    Jul 21 at 13:35
  • $\begingroup$ @gung-ReinstateMonica: I think that would be a great way forward. Can only moderators close as a multi-dupe? Or how do mere mortals do that? $\endgroup$ Jul 21 at 17:22
  • 1
    $\begingroup$ @whuber: great idea. I concur with mkt that the four threads above would be a great place to start. I started adding the FAQ tag to all four, they already all carry the unbalanced-classes one. Question: the "FAQ" tag says it it applied by mods, but I could add it just fine. Should I have been prevented from doing so? If not, perhaps we could revisit the tag excerpt. $\endgroup$ Jul 21 at 17:26
  • 3
    $\begingroup$ @StephanKolassa, mere mortals can only close a thread as a dup of 1 other thread. After that, you can leave a comment with links to other dups, &/or you can flag for mods to add the others. $\endgroup$ Jul 21 at 17:33
  • 1
    $\begingroup$ We cannot enforce the applied-only-by-mods rule. The intention is to be sparing in our application of the faq tag and to monitor it to avoid abuses (such as someone applying it to their own threads or their favorite threads). $\endgroup$
    – whuber Mod
    Jul 21 at 18:02
  • 1
    $\begingroup$ whuber notes that multi-dupe closing is a lot of work for the mods, and proposes to use a single dupe target, where we link to the others. I would suggest we use When is unbalanced data really a problem in Machine Learning? as a canonical dupe, and edit links to the other threads into that question, at the very top. Thoughts, anyone? $\endgroup$ Jul 21 at 19:17
  • $\begingroup$ @StephanKolassa Part of my concern was that no single question seems to have answers that address the topic comprehensively, so the information is diffused. My hope was that those of you who understood the topic better could identify threads that could be merged so that all bases were covered (and then we could designate it canonical). If that's not feasible, I agree with your suggestion. $\endgroup$
    – mkt
    Jul 22 at 9:12
  • 1
    $\begingroup$ @mkt: I see your point. I think that this would be very hard, simply because there are so many variations of the "problem". Some come from using accuracy as a KPI, some don't mention their KPI at all and treat imbalance as a prima facie problem without explaining why it isn't. The, ahem, lack of statistical sophistication does not help, either ("but I'm using precision, not accuracy, so this is not a duplicate!") I think a comprehensive treatment would be more of a journal paper than a useful CV thread. ... $\endgroup$ Jul 22 at 9:41
  • 3
    $\begingroup$ ... Does anyone want to write a paper for The American Statistician or Chance, so we can petition SO to be able to close as duplicates of a journal paper? $\endgroup$ Jul 22 at 9:42
  • 1
    $\begingroup$ @StephanKolassa Alright, then I agree with your suggested target for a canonical thread, though perhaps we can ask Tim to broaden it a little to include statistics in the title. $\endgroup$
    – mkt
    Jul 22 at 12:42
  • 1
    $\begingroup$ @StephanKolassa I think you are too modest. Your effort to which the OP links approvingly seems to me a good target. $\endgroup$
    – mdewey
    Jul 22 at 13:07
  • $\begingroup$ I just saw this answer for the first time, and I didn't see it referenced anywhere in here. // @StephanKolassa It's awfully tempting to write something for The American Statistician... $\endgroup$
    – Dave
    Aug 11 at 5:15

6 Answers 6

4
$\begingroup$

In the interest of taking a decision, I will post a few answers for you to vote on. Please feel free to post more.

Option 1: Do nothing. First do no harm, etc etc.

$\endgroup$
2
  • $\begingroup$ To be clear, this option means that new questions would be closed as duplicate of one of the four (or possibly another), depending on nuances between them as perceived by the close-voters? $\endgroup$ Aug 3 at 17:59
  • $\begingroup$ @BenReiniger That's probably the best-case scenario, yes. $\endgroup$
    – mkt
    Aug 3 at 18:17
1
$\begingroup$

In the interest of taking a decision, I will post a few answers for you to vote on. Please feel free to post more.

Option 3: Vote to close threads as duplicate of Are unbalanced datasets problematic, and (how) does oversampling (purport to) help?.

*Not just new threads, but also the other threads in the question.

$\endgroup$
0
$\begingroup$

In the interest of taking a decision, I will post a few answers for you to vote on. Please feel free to post more.

Option 5: Merge some subset of the threads in the question, and close others as duplicates of this new canonical thread.

If you choose this option, please specify which threads you think should be merged in a comment.

$\endgroup$
2
  • 2
    $\begingroup$ This has some appeal, but I'm not sure how possible it is to collapse all the nuances into any 1 of those Qs. $\endgroup$ Jul 25 at 21:20
  • $\begingroup$ @gung-ReinstateMonica Yes, I am coming around to this view. 3 of the 4 questions I mention cover very similar ground (except for the one about logistic regression) but the difference in framing and detail are quite useful. $\endgroup$
    – mkt
    Jul 26 at 5:23
0
$\begingroup$

In the interest of taking a decision, I will post a few answers for you to vote on. Please feel free to post more.

Option 2: Vote to close threads as duplicate of When is unbalanced data really a problem in Machine Learning?.

*Not just new threads, but also the other threads in the question.

$\endgroup$
0
$\begingroup$

In the interest of taking a decision, I will post a few answers for you to vote on. Please feel free to post more.

Option 4: Vote to close threads as duplicate of What is the root cause of the class imbalance problem?.

*Not just new threads, but also the other threads in the question.

$\endgroup$
-2
$\begingroup$

In the interest of taking a decision, I will post a few answers for you to vote on. Please feel free to post more.

Option 6: Close new questions as duplicates of all 4 threads in the question

Note that this seems to be a lot more work for the mods; I think it is best avoided for that reason.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .