Are questions like How to resolve Error: cannot allocate vector of size 1.1 GB for glmnet in R on topic? On one hand, it asks about software, on another, this is about solving a practical data science problem and has general, software-agnostic, solutions (algorithms working in online mode etc.). Or maybe we should migrate it to datascience.stackexchange.com ?
I think that question, as currently framed, should be considered off topic.
The way I think about this is really pretty simple: I ask myself what the OP needs explained. If the answer is something about code / software, then it's off topic; if it's something about statistics / machine learning, then it's on topic, even if the question is asked in terms of some particular software (and may not even be askable without reference to software). It is true that the latter condition can mean that a given question cannot be answered by people who don't use that software, and may not even be understandable by them, but because the answer is an explanation of a statistical concept, it is ultimately a statistical question, and thus on-topic.
It's true that the specific thread at issue includes the line, "Or if you can recommend another way to proceed with the variable selection that would also work for me?", which may have been intended in a software-neutral way, but this seems like an afterthought. If the question were rephrased to ask about general computational strategies for large datasets, or about variable selection, then it would become on-topic, in my opinion. It would be fine if the question still makes reference to software as part of providing informative context for their situation. But the OP needs to show that they are clear that they are asking a statistical / software-neutral question, and that they will be happy with a sufficiently clear statistical answer, even if it were illustrated with, say, minitab. If the OP would not be OK with that, then they are not asking a question that is appropriate for this site.
With regard to migrating the thread to datascience.SE, I suppose that would be possible, but it probably isn't something that I would do. It is true that DS handles questions concerning topics related to the domains of CV and SO, but which would be off-topic on those sites for various reasons. In that light, the thread may be OK there, so I'm not necessarily opposed. It just isn't something that would even occur to me, to be honest.
I have initially thought that I agree with @gung (and even upvoted his answer) but the discussion in the comments showed that I actually disagree.
I think this question (after the clarifying edits) is on-topic.
If the question were rephrased to ask about general computational strategies for large datasets, or about variable selection, then it would become on-topic, in my opinion. It would be fine if the question still makes reference to software as part of providing informative context for their situation.
I think this is what already happened. The OP wants to run logistic regression with elastic net penalty -- a very clear & meaningful statistical setup. The dataset is too large and the standard tool (
glmnet) fails to work -- a very clear obstacle. This person comes to our forum and essentially asks: "What can I do?". This must be on-topic.
@Gung replies that
In this case, 'try package y in software z' is the kind of answer the OP seems to want
and I agree but I do not think this makes it off-topic. Imagine there is a more efficient algorithm of fitting this model that does not require so much memory. An answer describing this algorithm without mentioning the software package is pretty much useless because nobody wants to re-implement stuff like that from scratch. Everybody would want to have a working implementation.
Of course an answer "try package y in software z" and nothing else would be a bad answer. But a possibility of bad answer does not make the question off-topic. A description of the methodology plus a package recommendation would be a fine answer.