10
$\begingroup$

We have a wealth of questions about cross-validation, such as

Which are, if not identical to eachother, do overlap by about 80%. Recently, someone asked the question

Which would seem to overlap with each of those previous threads substantially. The author expressed doubt about many of the threads that I linked to, but I'm not sure that the author's doubt is a great reason to open a new thread which covers highly similar, if not identical, ground as previous threads. This is the same reason we have one canonical thread on the coupon-collector problem, not a separate one for each type of collectible item in existence.

Or, we might look at this from the opposite perspective and say that we have a scattered collection of threads about how cross-validation is used and the problems it does (and doesn't) solve -- but no single, coherent, authoritative thread about cross-validation. I can see two ways to make CV better and more accessible in this regard:

  1. With malice aforethought, compose a question which fills in the interstitial gaps in our existing questions about CV, and then answer that question deliberately and authoritatively.

  2. Consolidate existing threads to form a one-stop (or perhaps two-stop) shop of answers about cross-validation.

I personally favor option 2, since lots of the work has already been done to lay the groundwork of what CV is. Option 1 will create yet another thread about cross-validation and will be harder to find unless it's highly up-voted immediately or you're the Rain Man of the search feature.

$\endgroup$
10
  • 6
    $\begingroup$ How many questions do we need on any topic? I'm not sure I see anything special here. Certainly when you have a duplicate, we can close it as such. Writing a canonical Q on a given topic does seem like a good idea. Regarding the new Q that overlaps the previous, the OP needs to clarify what they learned from those Qs & what is still ambiguous. I think you're right that at present, that Q is not going to lead to greater clarity. $\endgroup$
    – gung - Reinstate Monica
    Mar 31, 2016 at 14:55
  • $\begingroup$ To ansewr your question, if we understand a "topic" to be narrowly construed, I would say that we only need on thread per topic. What I was trying to emphasize is that with regards to basic concepts around cross-validation, there seems to be a large overlap among those threads, whereas with other "basic concepts" questions, we have done a better job of reducing duplication. $\endgroup$
    – Sycorax Mod
    Mar 31, 2016 at 15:03
  • 14
    $\begingroup$ To put it obliquely, we are not Wikipedia and that's the problem here. The mechanisms to catch duplicates do exist but they require omniscience and eternal vigilance to work well. We're always being frustrated by members who do little or no research, even within the forum, and just post any question that occurs to them. In principle, a thread consolidating several would be an excellent idea, but almost certainly far more work than you should want to do. It could be best to write a paper, debug a program, work on your thesis, mow your lawn instead. $\endgroup$
    – Nick Cox
    Mar 31, 2016 at 17:01
  • 9
    $\begingroup$ @NickCox Every time I think I'm the grumpiest CV user, Nick Cox reminds me that the British are uniformly better at it. Cheers! $\endgroup$
    – Sycorax Mod
    Mar 31, 2016 at 17:06
  • 7
    $\begingroup$ I guess that's a compliment on balance. I did say excellent idea, so if you have the energy and enthusiasm to do it go ahead with my blessing and good wishes. Someone close to me suggested that variations in statistical terminology should just be sorted out by reading all the books, tabulating what people use and then suggesting the right terms that everyone should adopt henceforth. My mind boggled at the amount of work entailed and I was reduced to speechlessness. Consolidation is not easy, in short. $\endgroup$
    – Nick Cox
    Mar 31, 2016 at 17:15
  • 1
    $\begingroup$ I think you should do a cross-validation study to determine the optimal number of questions about cross-validation. If you don't know how to set up such a study, perhaps you're having trouble determining the optimality criterion, data sources, or whatever, then you can post a question asking how. $\endgroup$
    – Mark L. Stone
    Apr 2, 2016 at 14:22
  • 1
    $\begingroup$ @MarkL.Stone While I'm at it, I'll perform a regression analysis using gradient descent. $\endgroup$
    – Sycorax Mod
    Apr 2, 2016 at 14:23
  • 1
    $\begingroup$ No problem, as long as you cross-validate. $\endgroup$
    – Mark L. Stone
    Apr 2, 2016 at 14:34
  • 1
    $\begingroup$ Actually,do you remember that this website's name is "Cross Validated"? That in itself might tend to attract questions pertaining to cross validation. Potentially, it could have been called "Probability and Statistics", which is what seems to be its scope. $\endgroup$
    – Mark L. Stone
    Apr 2, 2016 at 19:47
  • 1
    $\begingroup$ I see what you mean. Likewise, Twitter is a terrible ornithological guide. $\endgroup$
    – Sycorax Mod
    Apr 2, 2016 at 20:34

0

Reset to default

You must log in to answer this question.

Browse other questions tagged .