6
$\begingroup$

The question Suggestions on using model in production 1 test at a time was closed as off-topic.

To my mind, the question content is squarely about the generalizability of a specific kind of feature (categorical encodings of particular strings). This seems on-topic! The subject matter seems incidental because OP's question is about a more general kind of dummy variable trap, in the same way that you wouldn't use a categorical encoding of SSN or phone number in a regression model because it uniquely identifies each respondent.

I think that my answer directly addresses the statistical component in its first section. The second section of my answer is probably not on-topic, but since I happen to know about these models, I felt that I could offer some constructive advice. The third section of my answer is about how to construct generalizable features or else use feature-free architectures which support generalizable models.

Would anyone involved in the closure of this question be willing to elaborate on their decision to close?

$\endgroup$
  • 3
    $\begingroup$ I have reopened the question. I agree w/ others (eg, @mkt below) that the question does not appear obviously on-topic when read by someone who doesn't work on that specific issue. However, the answer makes it clear that the real issues are statistical, even if that's not obvious to most. (Nb, my test of on topic has always been: what does the OP need explained, if a statistical concept, it's on topic.) $\endgroup$ – gung - Reinstate Monica Jul 1 '19 at 17:50
  • 3
    $\begingroup$ That said, it is really difficult to see the statistical question until the answer has been read. Could you edit the Q to make the statistical issue more salient? The OP seems to agree w/ your interpretation of the Q, so I think adding a couple sentences at the end should be acceptable. Further, you could comment to the OP that if they disagreed with your edits, they could roll them back. $\endgroup$ – gung - Reinstate Monica Jul 1 '19 at 17:51
  • $\begingroup$ @gung Thanks for opening the Q. It's good to hear that you and I are both using the same test to evaluate the topicality of questions! I'll suggest these edits to the question. $\endgroup$ – Sycorax says Reinstate Monica Jul 1 '19 at 18:04
5
$\begingroup$

I voted to close as "Unclear what you're asking" because I genuinely could not follow the question. The main statement of the problem is:

My big question is how do I handle one test feature where part of it has not been trained?

...which I didn't really understand till I read your answer just now. And an apparent secondary problem was:

The issue that I am running into is the training set has seen the ChildPath "C:\Windows\System" and the ChildExe "cmd.exe" which are normal, but the training set has not seen the ParentPath "C:\Windows\Malicous" or ParentExe "badscipt.exe" so these have not been label or one hot encoded.

...which seems like more of a programming issue.

I notice that your answer also begins:

The challenge that you've discovered is a natural consequence of how you've organized your research project: your model has no generalizable information about new file paths or new names of .exe files.

This also seems to suggest a programming/organisational problem rather than a statistical/machine learning one.

That said, I don't have much expertise in this. So my confidence in my judgement is not very high and I'm willing to reverse it. I do think the question is poorly written, though, and would benefit from some editing.

$\endgroup$
  • 3
    $\begingroup$ Don't think the 2nd is a programming issue: 'C:\Windows\Malicous' is a level of the categorical predictor 'ParentPath' that is missing from the training data. I'd say the q's on-topic, but agree it could do with some clarification. $\endgroup$ – Scortchi - Reinstate Monica Jul 1 '19 at 8:34
  • $\begingroup$ @Scortchi I guess I misunderstood the question even after re-reading. $\endgroup$ – mkt - Reinstate Monica Jul 1 '19 at 8:52
  • 2
    $\begingroup$ Thanks for the elaboration! I can see where you’re coming from! $\endgroup$ – Sycorax says Reinstate Monica Jul 1 '19 at 12:42
  • $\begingroup$ @Sycorax No problem! $\endgroup$ – mkt - Reinstate Monica Jul 1 '19 at 14:21

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .