4
$\begingroup$

On Kaggle, recently there is a person who's been answering questions with what is likely customized AI. It's not Chat GPT, but probably something more boutique / hand rolled, and possibly a result of a number of different contests which have been created to solve similar q/a problems on Kaggle. This is an extremely well studied / well understood problem on Kaggle.

It's probably not just all AI but it's also likely the user is supplementing the material as well, via either more detailed prompts or just directly editing the material. They are clearly an expert in this field.

What's sort of cool about what they're doing lately is that they're starting to answer questions deep in the weeds. For example, they'll find a notebook that's been barely looked at by anyone and give deep and voluminous (and accurate!) feedback.

It's a combination of Q/A and summary generation, but this is absolutely not upvote grubbing behavior. They are already a GM with a huge number of gold medals and these postings will only marginally benefit their standing. It's someone who's playtesting their AI and trying to help folks in the community at the same time.

Let me be clear - the answers are actually quite good and they've been getting a number of gold medals for their postings. I frequently upvote the best ones myself. However, I am somewhat concerned that they are not being entirely transparent about how the content is being generated.

I know there is a ban on posting ChatGPT answers. I personally think Chat GPT should be allowed - but only if it's carefully cited / credited. Other than that, I agree people shouldn't be allowed to plagiarise, which I think is a terribly immoral activity. This goes for Chat GPT or any source material.

But what about something that is the effort of effective Human-AI collaboration? An expert working intelligently with AI that has been trained carefully to answer questions in a particular domain?

I personally think if they cite that AI helped generate the answer it should be allowed. Because of the cite, the answers will get careful scrutiny. If the answers are awful, the user's credibility will suffer. If the answers are good, more questions will get answered and we'll have an interesting chance to see what Human/AI collaboration is capable of. AI developers will also have a chance to better train and test their toolsets.

Of course, you could try to enforce a policy of just banning the answers completely, but given what I'm seeing out there lately - good luck with that.

$\endgroup$
5
  • 2
    $\begingroup$ "[...] you could try to enforce a policy of just banning the answers completely, but given what I'm seeing out there lately - good luck with that" - what are you getting at there? $\endgroup$
    – Scortchi Mod
    Commented Jan 21, 2023 at 16:04
  • 10
    $\begingroup$ I don't care how someone arrives at an answer. The post should speak for itself. If, however, they use chatbot output (modified or not) without checking it and wind up posting unhelpful answers, wrong answers, or just plain BS on our site, I will be extremely negatively disposed towards them due to the extra moderation burden they are imposing and the threat they represent of potentially using the chatbot to overwhelm us with useless and misleading material. $\endgroup$
    – whuber Mod
    Commented Jan 21, 2023 at 16:23
  • 1
    $\begingroup$ @Scortchi If an expert leverages AI trained to generate accurate content for a particular domain and even massages it to smooth out any rough parts, there's really no reasonable way that I can see that you can prove or ban that. As whuber says, does it really matter at how they arrived at the answer as long as the quality is reasonable? $\endgroup$
    – Blaze
    Commented Jan 21, 2023 at 17:28
  • $\begingroup$ It's a bit like how fully self driving cars are allowed, as long as someone is behind the driving wheel ready to take over in case something goes wrong. $\endgroup$
    – Blaze
    Commented Jan 21, 2023 at 17:35
  • 4
    $\begingroup$ @Blaze: Thank you. It would be difficult to prove: when I'm almost sure from the style that an answer's written by ChatGPT, it's the (weird) errors of fact & logic that fully convince me. (All the same, I'd trust that most people would abide by site policies, even if they disagreed with them.) $\endgroup$
    – Scortchi Mod
    Commented Jan 22, 2023 at 21:11

3 Answers 3

Reset to default
7
$\begingroup$

I don't see a need, at the moment, for a specific anti-A.I. policy. Stack Overflow's ChatGPT ban was a consequence of their getting so many answers wholly written by ChatGPT that they could no longer afford the time to consider each on its merits; & that's not the situation we're in.

Nevertheless, we oughtn't to, & don't, deal with low-quality content posted deliberately or negligently in the same fashion as with that posted in good faith, as an honest expression of someone's opinion. "If the answers are awful, the user's credibility will suffer"—no, we need to act quickly & firmly to stop other users' time being wasted by computer-generated dross. And users of our site haven't volunteered to take part in an experiment—any desire on the part of developers to "better train & test their tool-sets" must be satisfied as a by-product of constructive participation.

$\endgroup$
6
$\begingroup$

The use of AI is already ubiquitous. Examples are spelling control or language improvement and programs that help searching for resources (e.g. Google search).

It seems to me that it is not relevant whether AI is used, and instead it is more about how it is used and what result comes out of it.

Example of a bad case: If you get questions/answers from a language model (which is currently far from being good at logic and able to answer substantively; it can create a related language for some given content, but it can not create the content itself, at least not when it is complex), and if they are posted unedited without care, then

  • what result: you might get an increase in nonsense responses*. Nonsense will be moderated in any case. Whether it is human created or AI.

  • how: you might get people answering questions without appropriately crediting resources. I am not sure how to credit an AI language model and whether this is possible at all. At least one might give information that the answer has been obtained with the help of some AI, (not if this is for correcting spelling, but yes if the AI created an important amount of the answer), but there are potentially original resources that are not credited. The novel language models that harvest all of the internet do not provide a good way to credit original resources.

    I guess that, untill AI is able to appropriately credit resources, it is better to restrict the use of AI to trivial tasks (like correcting language) and not use it for creating publications/articles/answers/questions that tap into the intellectual property without crediting it.

But just as well as the bad stuff, you might also get good uses. I have found that chatGPT is able to improve my French writing by a lot (and probably it would do well with my English as well) and it is also good at correcting the tone of my texts. For people that are not great writers, a language model can be a good thing to improve their writing. Would you publish an article without having others check your writing? If no, then why not also use AI to help improve writing?

On the website here we regularly see problems with text in the form of images, equations that are poorly written and not in latex, or English that is at a low level.

From the point of view of the quality of the website it would not be bad to get this improved with AI. (From other points of view one might critise humans in using too easily some tools without improving themselves, but that is another discussion)

$\endgroup$
10
  • 1
    $\begingroup$ (+1) But "[...] then you might get an increase in nonsense responses"? Not might, will: we've seen it. $\endgroup$
    – Scortchi Mod
    Commented Jan 22, 2023 at 9:38
  • 1
    $\begingroup$ @Scortchi-ReinstateMonica I was using the more neutral "then you might get an increase" instead of "then you will get an increase". Even when in practice we have seen an increase (although I myself are not aware of the data about it), then it might be better to speak about it in a hypothetical sense. It may not be a necessary result that the use of AI language models leads to an increase of nonsense. $\endgroup$
    – Sextus Empiricus
    Commented Jan 22, 2023 at 11:23
  • 1
    $\begingroup$ Yes, fair enough: I just really, really don't want any reader to get the idea that doing that might be helpful, or acceptable. $\endgroup$
    – Scortchi Mod
    Commented Jan 22, 2023 at 15:08
  • 1
    $\begingroup$ @Scortchi-ReinstateMonica I agree that not every use is acceptable, and I have added some details because of that. $\endgroup$
    – Sextus Empiricus
    Commented Jan 22, 2023 at 15:59
  • $\begingroup$ I think the example is an unfortunate straw man and not what this question was about. It was about Human/AI collaboration. Also, I think the question can sometimes be about the prompt itself. Done carefuly, frequently no editing is needed. $\endgroup$
    – Blaze
    Commented Jan 23, 2023 at 10:53
  • $\begingroup$ @Blaze I am not sure which example you mean because there were several (spelling control, Google search, language model). If there is a strawman then could you specify more clearly/specifically what case it is that you had in mind. Are my examples not good cases of human/AI collaboration? ... $\endgroup$
    – Sextus Empiricus
    Commented Jan 23, 2023 at 11:53
  • $\begingroup$ ... The point that I made is that it is not about the principle of the human/AI collaboration, but about the practical implications. In practice we see already many successful collaborations that we don't want to get rid of, but there might be uses that are detrimental (and those might need to be treated more specifically and not be related to AI/human interaction in general). $\endgroup$
    – Sextus Empiricus
    Commented Jan 23, 2023 at 12:01
  • $\begingroup$ The one after "Example:" .. I think we all agree naive copy/past of chat gpt is a waste of everyone's time $\endgroup$
    – Blaze
    Commented Jan 23, 2023 at 14:07
  • $\begingroup$ @Blaze naive copy pasting is also collaboration between humans and AI. In my post I tried to indicate that there is a wide spectrum and it is not in principle bad to use AI, instead it is about how it works out in practice. If you did not mean to speak about collaboration in general but about a more specific case, then indicate that better. Otherwise, I disagree that the example is a strawman argument because I did not meant to use the example to replace your proposition or as an example of the total of human/AI collaboration. $\endgroup$
    – Sextus Empiricus
    Commented Jan 23, 2023 at 15:00
  • $\begingroup$ "naive copy pasting is also collaboration between humans and AI.", we'll have to agree to disagree on that point. $\endgroup$
    – Blaze
    Commented Jan 23, 2023 at 16:04
-3
$\begingroup$

I suppose until AI is considered to have to equal or at least more rights, the policy of Nature is likely what will stand for most reasonable organizations and hopefully stackexchange as well.

https://www.nature.com/articles/d41586-023-00191-1

First, no LLM tool will be accepted as a credited author on a research paper. That is because any attribution of authorship carries with it accountability for the work, and AI tools cannot take such responsibility.

Second, researchers using LLM tools should document this use in the methods or acknowledgements sections. If a paper does not include these sections, the introduction or another appropriate section can be used to document the use of the LLM.

They also make a point elsewhere which I strongly agree with:

https://www.nature.com/articles/d41586-023-00188-w

Some publishers say that chatbot use should be documented in the methods or acknowledgements sections — and that not doing so could be considered plagiarism.

edit to add: It's unfortunate that I have to state the obvious, but this discussion should be about about LLMs and AI (large language models and artificial intelligence) and not shallow ML, like spell checkers and grammar checkers.

$\endgroup$
18
  • $\begingroup$ It is exaggerated to prohibit all forms of using LLM and require acknowledgements for the use of computational tools or writing aids (should we also acknowledge the computer software and hardware for doing our computations?). Control of spelling and other language mistakes, which can be done and is being done with LLMs, is not the same as creative contribution. Even this message, being typed on an android phone, is using a rudimentary form of a language model (the keyboard has word suggestions). LLMs in writing are like editors/reviewers that correct but don't create the thoughts. $\endgroup$
    – Sextus Empiricus
    Commented Jan 31, 2023 at 6:56
  • $\begingroup$ Obviously one should not use a LLM to create a fake research, like asking chatGPT the question "write me an interesting article for Science". But that's not fraud because of the use of chatGPT, it is fraud because of making up the content of a research article. $\endgroup$
    – Sextus Empiricus
    Commented Jan 31, 2023 at 7:01
  • 1
    $\begingroup$ I don''t see why the policies of any journal are to be taken as guiding. First off, any decent journal has peer review and so some (serious) level of scrutiny to weed out unacceptable content. Here on SE just anyone can post an answer, and it's then a community problem to watch for poor content. Second, authors of journal articles (should) have their credibility to worry about and anonymity is very rare. Here anyone can hide their identity if desired. That's fine for most purposes, but not this one. $\endgroup$
    – Nick Cox
    Commented Jan 31, 2023 at 9:54
  • $\begingroup$ In short, the situation of journals here bears some similarity with ours, but saying that their policies should be ours ignores vital differences. The motives of people using AI seem to vary from feeling mischievous through game playing to regarding it as a positive tool. People's motives are for them, but poor content is our problem. $\endgroup$
    – Nick Cox
    Commented Jan 31, 2023 at 9:54
  • $\begingroup$ It's terribly silly and unproductive straw man to compare 'writing aids' to chat gpt. These types of digressions really don't help the dialogue at all, imho. ChatGPT is something entirely different and any reading of the drama unfolding in various communities should be an obvious signal for that. $\endgroup$
    – Blaze
    Commented Jan 31, 2023 at 21:41
  • $\begingroup$ The policy should be, fundamentally, if you use ideas from chatgpt (or any similar competent AI) you should acknowledge it. On stack exchange, or anywhere. Anything else is the moral equivalent of plagiarizing a human being. I'm not saying AI deserves some kind of right to life or anything silly (yet), but it does deserve that much. $\endgroup$
    – Blaze
    Commented Jan 31, 2023 at 21:42
  • $\begingroup$ 'spell checkers and grammar checkers', that is also AI. So you have to be clear what you exactly mean. Especially since recently there has been a hype about a particular program using GPT-3 and is an excellent spelling and grammar checker, a translator and helps with improving the tone of messages as well. That creates the idea that you are talking about the programs that currently originate from GPT-3. $\endgroup$
    – Sextus Empiricus
    Commented Feb 1, 2023 at 10:05
  • $\begingroup$ I am not sure what ideas you attribute to chatGPT, but regarding the questions that occur here, chatGPT can't "put a dent in a stick of butter" (but you can ask chatGPT what that means). Example: ... $\endgroup$
    – Sextus Empiricus
    Commented Feb 1, 2023 at 10:15
  • $\begingroup$ ... Recently with this question Given that X and Y are normally distributed as N(0,3) and N(0,5) respectively, what is the expected value of (XY)^2? I was thinking of the idea that we should ban questions that can be answered by chatGPT (similar too stats.meta.stackexchange.com/questions/67) rather than banning the use of chatGPT. So I tried it out and asked chatGPT the question, but it couldn't properly answer it. $\endgroup$
    – Sextus Empiricus
    Commented Feb 1, 2023 at 10:16
  • $\begingroup$ It's a ludicrously sophomoric fallacy to point to errors in chatgpt as a reason to ignore its results. There's 100 of thousands of answers on stack exchange which are wrong, so should we ignore any answers by humans? From that, I could easily argue that some of the arguments here are trivially erroneous and therefore further results should be ignored. Of course, that would be also be a mistake. $\endgroup$
    – Blaze
    Commented Feb 2, 2023 at 0:51
  • $\begingroup$ The debate and discussion in various content and professional communities is a very credible signal that the information density on ChatGPT is sufficiently high to place it in a separate class of artificial intelligence. Anything else is just purposely putting blinders on. $\endgroup$
    – Blaze
    Commented Feb 2, 2023 at 0:56
  • 1
    $\begingroup$ @NickCox: meta.stackexchange.com/a/384681/225179. Much worse. $\endgroup$
    – Scortchi Mod
    Commented Feb 3, 2023 at 0:35
  • 1
    $\begingroup$ @Scortchi-ReinstateMonica It's always gratifying when statistical people themselves provide data. I still don't know how many people prefer talk of $y$ versus $x$ to talk of $x$ versus $y$: each faction seems confident that their own usage is standard. $\endgroup$
    – Nick Cox
    Commented Feb 3, 2023 at 0:44
  • 1
    $\begingroup$ @Blaze I imagine that Scortchi would agree that naturally a deeper analysis of a fuller dataset would be greatly preferable to a report on a small dataset (which can always be dismissed as anecdotal). The rules of debate here that you follow seem curiously lopsided. You have yet to address my very simple point that your assertion of "100 of thousands of answers on stack exchange which are wrong" is utterly unsubstantiated. I do appreciate that there are ways of getting order of magnitude estimates, and anything else would be hard to get, but you've yet to say what yours are. $\endgroup$
    – Nick Cox
    Commented Feb 3, 2023 at 9:52
  • 1
    $\begingroup$ @NickCox: Of course, & the sample was non-random as well as small - I was just sharing my observations. All the same, anecdotal evidence of 19 out 20 patients killed by a purported treatment would seem to militate for some modification of the protocol before kicking off large randomized controlled trials, & a fortiori before approving it for general use. $\endgroup$
    – Scortchi Mod
    Commented Feb 3, 2023 at 14:56

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .