Is it appropriate to ask whether an idea you have is good and original?

Question

I have come into the stats/machine learning field due to a sideways shift from computational physics. I'm not in an academic environment at the moment but feel as though I achieve good practical results even if they way I work is a bit of a hack; I need results rather than theoretical rigour (not that I'm suggesting theoretical rigour is a bad thing). My knowledge of the literature in the field is not comprehensive, so I often feel I'm reinventing a wheel that must have been published somewhere.

Anyway, that long winded intro is just to give this question some context. I've developed/am developing a technique (or a series of techniques) for optimising hyper-parameters for machine learning algorithms. It seems to produce better results and run more efficiently than what I understand are standard approaches, based on some literature I have read published in the last year or two. I would like to know if a similar approach has in fact been published, since there may be some improvements I've yet to work out myself. I'd also like to know if there is some obvious flaw that I'm not seeing in the approach.

So, I could ask a question along these lines in the main site, but I'm not sure how appropriate it would be for the site. I am concerned that in order to give sufficient details, the question would be very long, essentially a mini academic paper and therefore so specific that is doesn't contribute to a good database of Q&As, even if I receive good answers that help me. It may require a significant investment of time for people to read and comprehend for content that could turn out to be flawed or simply banal. I'm also very slightly concerned that if this is in fact a useful idea then the question may prejudice any later attempts to publish. I don't think that's too likely though, it's not exactly an earth shattering result.

Generally speaking, are questions along the lines of "Here is a new idea I have, is it any good?" useful and appropriate questions here?

Answer 1

As always, it depends on what do you expect -- duplicates of older works and "obvious problems" (i.e. those you instantly see after sitting merely 10 years in a topic) are likely to be spotted fast and accurately, but a 3-page proof will end in oblivion. Whether it would get an attention also depends on how innovative the stuff is -- scientists are by-design being amused by dealing with something intriguing.

Thus I think it is a good idea provided that you will not have too high expectations.

About the length -- you can just actually write a mini paper, publish on arXiv or viXra and simply put a link here. Moreover, this way makes it easier to split your doubts in a separate questions (for a better clarity) without sacrificing context. And you can withdraw the mini paper from both (from viXra more completely).

Answer 2

If you were in the competitive world of publish or perish, the very thought of doing this would be considered crazy because someone might steal your idea and publish ahead of you. Sometimes in the environment of academic pressure ethics goes out the window. I remember when I was a graduate student many people would be highly secretive about their research.

Even without rigor your work could be publishable as you may have examples that you can compare it with other techniques where your method proves to be better. Great ideas often have started that way. A recent good example is boosting. Often a supporting theory is developed later. The bootstrap is a really great example of that.

It is not difficult to research the outside literature with all the technical advantages the internet now provides. But asking on this site could give you a quick answer but may not necessarily. Your idea is not protected doing it this way.

Of course I get the impression that you just want to know if it is original or if anyone else has done something like it. In that case go ahead.

Answer 3

If the idea is highly specific to the field, so that there are ~20 people who would really understand it, then you should write a paper, put in into arXiv, and circulate the link to those 20 people. And start pushing it into journals and conferences if you are interested in publishing it, or make a patent -- that process I have no clue about though.