# Is there a generic question to which we can redirect questions of the type “why does my neural network not generalize well?”

Thanks to Sycorax, we have an excellent question which can be used to close as duplicate (most of) those questions about neural networks which can't overfit the training set:

What should I do when my neural network doesn't learn?

This helped to close a lot of "my network doesn't work, please fix" questions:

What's the best way to answer "my neural network doesn't work, please fix" questions?

I think we would also need a similar question for cases when the validation loss doesn't decrease (or does so only slightly). I tried to look for one, but I couldn't find it. Do we have one?

If the answer is "No", I'll write one with the same aim as Sycorax's. Now, to abuse the usual paraphrase from Tolstoy, "training issues are all alike; every generalization issue is particular in its own way", so it might be harder to give good answers to such a question, but I think that, if framed the right way, it could work. For example:

"My neural network doesn't reach the generalization error it should be able to reach, based on reputable sources or experiments with very similar NNs: which things should I check"?

For example, the first thing you need to check in this case is if you're overfitting the training set, and when (at which epoch) the training and validation losses start to diverge. There are a few questions where the OP only reports the accuracy on the validation set at a fixed number of epochs, which is pretty useless.

Update: this question has been created: What should I do when my neural network doesn't generalize well?

• Looking forward to your Q&A. – amoeba Sep 7 '18 at 7:21
• @amoeba I take it this means there's no such question, currently. Ok! Then I'll ask one later today. Hopefully it doesn't get closed as a dupe after 5 minutes ;-) – DeltaIV Sep 7 '18 at 8:13
• Having your question closed as a duplicate can be frustrating, but at least you would have found a way to ask your question that presumably nobody thought of before... when you have written your Q&A, can you please link there in an answer here? – Stephan Kolassa Sep 7 '18 at 12:52
• @StephanKolassa I've written the Q. I plan to write the A by the end of the day (and I will share the link here) but if anyone wants to answer it before, be my guest. – DeltaIV Sep 7 '18 at 13:14
• @DeltaIV "by the end of the day" :-/ – amoeba Sep 10 '18 at 8:21
• @amoeba ah, but then Sycorax wrote the currently accepted answer to this question, where he correctly notes that several hours of careful crafting should go in the answer....also, you know: weekend ;-) – DeltaIV Sep 10 '18 at 10:25
• Sure I understand. But if possible, don't wait too long: you are rapidly losing chances to get on the Hot Network list which would increase the visibility of the thread (and the number of upvotes) by an order of magnitude. @Sycorax'es thread got there right after posting (I think he posted Q and A together or maybe in quick succession) and that brought 100s of upvotes. – amoeba Sep 10 '18 at 21:43
• :-($\phantom{}\phantom{}$ – amoeba Sep 25 '18 at 20:23
• @amoeba sorry, you're right! The answer ended up becoming so huge that I could never find the time complete it...I'll try to do better in the next days. Concerning the Hot Network list, I don't think the question ever made it to that list. – DeltaIV Sep 25 '18 at 22:25
• @amoeba at long last, I posted an answer! Looking forward to constructive criticism. – DeltaIV Sep 30 '18 at 19:04