-1
$\begingroup$

The following question was recently asked on the main site, and quickly showed up on Hot Network Questions and gained many views.

What's a real-world example of "overfitting"?

Let's work together to improve the question to make it shine, and make it as great an example of this site as we can. This looks like something that could be a great question -- one that could serve as a wonderful resource, and might be of interest to many folks learning the subject, or teaching it.

What can we do to improve the question and the answers? Please post suggestions.

My suggestions:

  1. Let's curate the answers. Look through the answers, and try to pick out the best ones. Vote up the good ones. For the bad ones, vote them down, and where appropriate, flag them for deletion.

    As it currently stands, only a small fraction of the answers actually try to answer the question that was asked (which asks for a real-world example of over-fitting). Many of the answers give artificial examples or try to give an intuitive explanation for what over-fitting is, which is nice but does not answer the question. I suggest that we flag those answers for deletion, either by flagging them as "Not an answer" or flagging them for moderator attention with a custom comment. Also, we can downvote those answers and vote to delete them once their total vote-count goes negative.

  2. Vote up the good answers, and edit them to improve them. A few of the answers try to give real-world answers. Let's vote them up, and then edit them to elaborate further so that the answer serves as a great example.

  3. I've edited the question to try to make it a bit more specific. If you can see ways to improve the question further, I encourage you to edit it to refine it further.

  4. What else? Other suggestions?

| |
$\endgroup$
  • $\begingroup$ Good idea! As for me: I am open to edits and suggestions about my answer since I have doubts if it is not overtly long and so if it does not miss the point I wanted to show. $\endgroup$ – Tim♦ Dec 12 '14 at 7:40
  • 4
    $\begingroup$ It is not completely clear to me that the OP meant by "real-life" exclusively real-life situations that actually did happen in the past; "real-life" could refer to and in particular could in this particular instance have been supposed to refer to an imaginary or hypothetical situation that directly relates to real life. I am against downvoting and strongly against deleting answers that interpreted the question like that. I am not even sure that I am happy with your edit of the question, but this is a smaller issue. $\endgroup$ – amoeba Dec 12 '14 at 9:46
  • 3
    $\begingroup$ I am also not sure about editing the question itself. Real-world is also "having to do with actual experience or practice, rather than being theoretical, idealistic, or impractical" or "having a practicable, verifiable quality" (cf. here) so it is not only "that happen in real life". I would say that statistics do not happen in real life ;) $\endgroup$ – Tim♦ Dec 12 '14 at 10:04
  • $\begingroup$ Why was this downvoted? $\endgroup$ – shadowtalker Dec 14 '14 at 15:08
  • 2
    $\begingroup$ @ssdecontrol: Downvoting questions on meta-CV means that people do not agree with whatever proposal OP was trying to make. Voting works a bit differently on meta-CV than on CV itself and has an actual "voting" flavour to it. This particular question has been upvoted +4 times and downvoted -5 times until now, indicating lack of clear consensus (but it's hard to say because some people might have upvoted as a way of appreciation that this topic was brought up or OP's intentions, while disagreeing with the specific suggestions...) $\endgroup$ – amoeba Dec 15 '14 at 14:35
  • $\begingroup$ I very much appreciate the raising of this issue (and the issue in D.W.'s other recent post), and while I agree with several points or suggestions in both instances, have difficulty upvoting either question because I also disagree with other parts. That is, I want to signal these are good questions (and I would happily upvote either on that basis), but I disagree with at least some part of the suggestions they contain. In the end, I'm upvoting this one, but not the other. On the other hand, and by way of clarification where I disagree with the actions, I fully support Nick Cox's answer. $\endgroup$ – Glen_b Dec 20 '14 at 22:51
12
$\begingroup$

At one level you urge just what we should be doing generally with every question, seeking to improve further what is good and working to correct and even delete what is wrong.

Moreover, there is no problem, in my view, with your drawing attention on Meta to a particularly interesting question on the main site as worthy of attention and further effort.

In this case, however, I suggest that your rewriting of the question goes much further than is appropriate. Edits should correct or improve presentation (grammar, spelling, punctuation, poor style, code and mathematical formatting, etc., etc.) and at most correct small and entirely uncontroversial errors. Even where a poster makes an obvious slip (e.g. confusing dependent and independent variable) it is best to flag it explicitly, as the poster may indeed be confused, on something basic as well as on something more advanced.

But edits should not usually be a rewriting of someone else's question to what the editor prefers. In this case you added a long interpretation of what the question means but kept use of the first person in writing. Whether we agree, individually or collectively, that it's now a better question is not to me the main point. Editing is not rewriting.

Conversely, it could be argued that the several answers that do not focus on real-world examples were underlining more general interest in over-fitting. If it is in order for you to steer the question in the direction you prefer, would it not be in order for someone else to edit the question to make it a wider-ranging question on over-fitting, and make more of the answers and comments fit better retrospectively? This is the dangerous territory you've entered. Once good taste or personal judgement is the arbiter, your good taste or judgement may not match anyone else's.

Further, in editing my own answer (which is, in principle, entirely in order) you introduced some arguably irrelevant material, which I've reversed.

On down-voting answers: I think this is best reserved for answers that are wildly wrong, wildly irrelevant or obnoxious (spam, offensive, etc.). I hope you are not suggesting otherwise. You seem to be urging a much more aggressive approach to material you disagree with than fits well on this forum. Down-voting has its place, deletion is sometimes called for, but by far the best tactics are to show reasoned disagreements (e.g. in comments), to up-vote the better material, and most of all to provide better answers. Yet further, whenever a question is controversial, a variety of views and styles of answer in reply can be highly valuable. Most of all, you should never urge others to down-vote or even up-vote on particular answers; that's the politics of the gang or the crowd. People should always vote on their individual judgement.

I think your motives are excellent, but there is an excess of zeal here.

| |
$\endgroup$
  • $\begingroup$ +1. I would roll back the edit made by the OP, but I guess by now it is better to wait a bit and see if there is a clear majority supporting this... If anybody is AGAINST rolling back, please say so explicitly. $\endgroup$ – amoeba Dec 12 '14 at 14:13
  • $\begingroup$ @amoeba, I have rolled the edit back. $\endgroup$ – gung - Reinstate Monica♦ Dec 12 '14 at 15:05
  • $\begingroup$ "Edits should correct or improve presentation (grammar, spelling, punctuation, poor style, code and mathematical formatting, etc., etc.) and at most correct small and entirely uncontroversial errors." - Well... I'm not sure that is StackExchange policy. StackExchange encourages editing questions to improve them. See stats.stackexchange.com/help/editing. That specifically lists other categories of encouraged edits, including "clarify the meaning of the post (without changing that meaning)". If I may speak in my defense, that's exactly what I tried to do. $\endgroup$ – D.W. Dec 12 '14 at 18:03
  • 1
    $\begingroup$ ... and if my edit didn't match what the OP had in mind, he/she was welcome to roll it back and edit the question accordingly. However keep in mind that here we have an absentee OP -- the original user provided a short question without many details, then disappeared and did not respond to requests for clarification. In my defense, my edit was a good-faith to improve the question to reflect what the OP seemed to be asking, not an attempt to change the intent of the OP. $\endgroup$ – D.W. Dec 12 '14 at 18:05
  • $\begingroup$ However, I hear you. If this kind of attempt to improve questions and answers does not fit the sort of thing the community here would appreciate, I will take that to heart and act accordingly in the future. $\endgroup$ – D.W. Dec 12 '14 at 18:08
  • 1
    $\begingroup$ The nub and the rub here is "without changing that meaning" on which people can disagree in good faith. It's clearly common here on CV, and perhaps even true of the majority of questions, that they show some misunderstanding of statistics. If we corrected the questions too, the answers would often be very difficult to follow. $\endgroup$ – Nick Cox Dec 12 '14 at 18:14
  • $\begingroup$ As someone very active on CV, and active on only one other SE site, I find myself a little cool on SE policy generally. I don't want to subvert or contradict it. But it seems to allow, for example, a higher use of vulgar epithets and pointed exchanges on Stack Overflow than would be welcome here. (N.B. that is quite tangential to this thread.) $\endgroup$ – Nick Cox Dec 12 '14 at 18:16
  • 7
    $\begingroup$ @D.W., although I have disagreed w/ your suggestions in these 2 posts, I want to say that I think you have done a good job in bringing these issues up for discussion. The questions are clear, well-researched & -articulated. On the main site, they would get upvotes. As you may know, the voting scheme is different on meta, w/ downvotes indicating disagreement w/ the suggestion, not low quality. In addition (whether I agreed w/ your actions or not), I do not think you have acted in bad faith at any point. & I think you have been very mature about this in the face of disagreement from others. $\endgroup$ – gung - Reinstate Monica♦ Dec 12 '14 at 18:17
  • $\begingroup$ @gung, thanks for the kind words. I definitely appreciate them. I understand that meta works differently and the downvotes don't bother me, but thanks for mentioning it, as otherwise I imagine it could be disheartening for someone who wasn't familiar with that aspect of SE. $\endgroup$ – D.W. Dec 12 '14 at 18:31
  • 2
    $\begingroup$ I strongly disagree about down votes. Most of us have little to gain or lose by 1 reputation except our pride, yet because vote counts are low those single votes can make a big impact on a question's visibility and momentum. Meanwhile, the basic downvote criteria include answers that don't answer the question. Therefore I think it would be better if we all became much more liberal with our downvotes. They can always be reversed $\endgroup$ – shadowtalker Dec 14 '14 at 15:14
  • 3
    $\begingroup$ We can all make our own choices. A downvote can also be very discouraging to people trying to learn the game here. $\endgroup$ – Nick Cox Dec 14 '14 at 15:17
6
$\begingroup$

+1 to @NickCox. He has done an especially good job of talking about edits to questions, let me add that I think that goes double for answers. As it happens, this same issue occurred yesterday with a very well-meaning user making changes to an old answer to correct what he thought were simple, innocuous mistakes on the part of the answerer. In fact, he turned out to have been right, but I believe as a general policy it should be left to the answerer to make those changes at their discretion. Here are my key positions:

  • Do not edit someone else's answer to change what you disagree with. This is not the proper use of edits. Leave a comment and/or downvote. The answerer can respond, and make changes or not as they choose, but either way future readers will be made aware of the issue.
  • Even if it is factually untrue, do not edit to change errors. Leave comments to ask the answerer for clarification / to point out the error. If you feel strongly enough, you can downvote; you don't have to downvote, I rarely do. It is just your strongest option.
  • Personally, I would probably not edit to make such changes even if nothing happened for a long period of time.
  • Although we may edit to correct, e.g., typos, typos are small things like spelling errors, not factual claims. "Build" instead of built is a typo. The distinction (probably) has no epistemic content in the context of a given answer. Claiming a statistical model has a property is a factual claim. We should let the factual claims of the answer remain as the author intended them. I think it is for the answerer to clarify them. My position is that we should not edit an answer to change a substantive claim.
  • I acknowledge that I have something of a double standard on editing: I feel fairly free to edit questions, but am quite reluctant to edit answers. I only make superficial edits to answers, but often make substantive edits to questions when they are statistically confused and/or not clear English. (However, I do not make changes outside of what I think the OP is trying to ask, and I typically leave a comment to the OP to make sure it still says what they want it to say.)
  • Adding your own answer is another good option.

Another suggestion in this thread is to delete answers that do not fit. I disagree with this also. I recently addressed this issue on meta.CV here: When should answers be self-deleted? (I should also note that asking moderators to delete answers that do not fit can also be dangerous, see: Why was an answer considered off topic and when was it deleted?) Although I do not have a problem with deleting non-answers (new questions, comments, spam, etc.), I think something that was intended as an answer (even if under a mistaken reading of the question) should be allowed to stay. In short, answers should not be deleted for being incorrect (or in this case not fitting well). As above, I think the proper remedies are to comment, downvote, or add another answer of your own.

| |
$\endgroup$
  • $\begingroup$ Although this may look like a small mutual admiration society, I am very happy to endorse this post. In this thread, the larger point was the OP's editing the question on the main site, but editing answers is hardly different in principle. $\endgroup$ – Nick Cox Dec 12 '14 at 17:55
  • $\begingroup$ @NickCox, personally I admire mutual admiration societies ;-), however, I am responding to concrete suggestions in the question. I just think your points extend more broadly than you happened to frame them, & coincidentally enough, an analogous situation occurred to me yesterday that I thought was worth relating. $\endgroup$ – gung - Reinstate Monica♦ Dec 12 '14 at 18:06
  • $\begingroup$ Don't worry: I would disagree with you as politely and as logically as possible if I disagreed with you. $\endgroup$ – Nick Cox Dec 12 '14 at 18:11
2
$\begingroup$

By the way, we could make wikis few questions that appear here again and again and are so marked as duplicates. There are at least a few examples of questions that appear and it would be good if there were one comprehensive answer, or set of answers, that could be referenced while marking duplicates.

The type of question that comes to my mind is, e.g., "What is and how to interpret a p-value?". There are multiple threads for this so it is even hard to mark duplicates and reference the already answered questions since you have to review many of those by yourself to find the best. I imagine also that lack of a single comprehensive answer makes some people ask "new" questions that are in fact duplicates of others lost somewhere between hundreds of similar ones.

| |
$\endgroup$
  • 1
    $\begingroup$ Frank Harrell had a similar suggestion about creating a FAQ tag. $\endgroup$ – Andy W Dec 12 '14 at 14:13
  • $\begingroup$ Thanks for pointing this one! I agree with his idea and it seems the same proposal reappears again and again. $\endgroup$ – Tim♦ Dec 12 '14 at 14:15

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .