It doesn't happen often, but I've occasionally seen questions that raise ethical concerns (but are on topic). A couple examples come to mind: 1) (on stackoverflow) Someone asked about designing a system to defeat CAPTCHAs. This could be used to set up many fake accounts, send spam, and (as has recently come to light) surreptitiously manipulate public opinion. 2) (here on stats) Someone asked for help with a machine learning system for making decisions in what sounds like an industrial/lab environment, where the consequences of a bad decision include "things blowing up". This isn't a situation where someone should take (or need) random advice from the internet, and mistakes could change (or end) lives.
Clearly such cases have to be handled on an individual basis, and it's complicated for many reasons:
It's not always clear what the situation is. In example 1, the poster could have been a security researcher (but didn't mention this). In example 2, the poster could have been framing things in a hypothetical or exaggerated way. Meanwhile, asking for clarification doesn't always lead to answers, and others may post answers in the meantime.
We also want the site to lead to good and not harm. But we don't want to have (or be) moral police.
Public discourse has great benefits and censorship has great costs. In some cases, the lines can be grey. For an extreme example, see the recent controversy about publishing work on the avian flu virus, which was artificially mutated to increase transmissibility.
I think people will already make nefarious or dangerous choices on their own, but I don't want to help them down that path. I also want there to be freedom to discuss technical issues, even when serious consequences are at stake. If there's a concern, I think a good practice would be to ask for clarification, write comments to make clear what's at stake, and shut the question down if there's reasonable suspicion that specific harm may come to pass.
I'd like to hear others' thoughts on these issues. Has anyone seen similar examples? On a lighter note, is this an appropriate use of the 'flag for moderator attention' button? And, is there a way to put a question on hold until it can be determined what's going on?