Yes, a site-policy against ChatGPT is safe and useful
This is a repost of my argument from Worldbuilding Meta. If you read it there, this will be some deja-vu.
I argue that ChatGPT answers are both qualtiatively and quantitatively different than run-of-the-mill low effort answers from new users, and that a rule against them will make things better.
ChatGPT acts like a ban-evading hydra
If we ban someone for being disruptive, and they come back under a new account (in ways that we can prove is the same user), we don't treat the new account as a new person and start over, giving the "new" member of their community a chance - we ban the sockpuppet.
If we find the current instances of ChatGPT disruptive enough to ban, we should treat the others as sockpuppets of the same. It's an extraordinarily well-resourced hydra that gets humans all over the world to sockpuppet for it, but it's a hydra, nonetheless.
ChatGPT users have no possibility for reform other than to stop using ChatGPT
If a human user posts an answer that is under-researched (or answers relative to the wrong system), they will receive downvotes and comments explaining what's wrong. They might fix the answer, delete it, or just leave it downvoted, but whatever else they do, they will also incorporate that feedback into their internal model (or "Learn," as we humans call it). If their goal is to get upvotes, they will research their answers better. If their goal is to avoid criticism, they will leave the stack. Either of these outcomes is acceptable.
ChatGPT users are different. Any feedback we give to ChatGPT users gets blackholed. ChatGPT is not going to update its model in response to our feedback, in part because it never receives our feedback. The user might look at the feedback (and may even learn something from it), but since we're never seeing their writing anyway, the quality of their posts does not improve.
I think most ChatGPT answers are made in bad faith
I suspect most ChatGPT users know full well what they're doing. The technology has been discussed enough now that its users can be expected to know that ChatGPT doesn't know anything they don't. People who continue to post uncited ChatGPT answers are likely doing so knowing full well that they cannot defend the answer's quality.
I can only speculate about their motivations for doing so. Some might be earnestly trying to see how well ChatGPT does with creative and esoteric prompts and others might be trying to test our community to see how well we can distinguish robots from humans, and still others might be trying to have some "harmless" fun - but I suspect most are trying to farm accounts with history and rep (and the privileges that accompany it), to sell them to spammers.
Regardless of their motivations, all motivations I can imagine have two things in common:
- A ChatGPT poster is not making a good-faith effort to provide a high-quality answer to the question asked.
- A user whose first few posts are all unedited and unsourced ChatGPT responses is unlikely to ever contribute anything else to the site, and we should act accordingly.
So why a policy?
A policy against ChatGPT serves two purposes: it will reduce the incentive to post such answers, and it will make moderation actions against ChatGPT trolls more efficient and consistent.
Reducing incentive to post
People who post bad-faith ChatGPT answers are doing so with a purpose: they want an account with rep and history. If we make it clear, in policy and action, that repeatedly posting ChatGPT answers will not get them an account with rep and history, we take away that incentive. If we fail to do so, others will join them.
Making it easier to moderate
We do not, as far as I know, ban people for making low-quality answers in
good faith. I certainly don't think we should ban users just because their posts are low quality (as long as they're responsive to feedback and don't repeat mistakes excessively). The problem is that usage of ChatGPT implies that someone will not be responsive to feedback and that they will repeat mistakes excessively. A policy against it allows moderators to act on that evidence, taking the usage of ChatGPT as evidence of bad faith, which will allow quicker, more consistent, and more objective moderation.
What about false positives?
I don't think anyone's suggesting a zero-tolerance rule for anything that looks like ChatGPT (like some of what's happened on r/art). ChatGPT answers might be difficult to distinguish from human answers, but ChatGPT users are not hard to distinguish from human users.
AI can be very convincing. If OpenAI, Google, or Microsoft made it a project to create an AI that could participate on Stack Exchange in a human-like manner, they might succeed - however, most of what we're dealing with right now are script kiddies playing with a new toy, and we can look for human factors indicative of AI usage.
- ChatGPT users can post long answers very quickly. (A user who posts a lot of long answers in a short amount of time is likely either using AI or plagiarizing from somewhere else)
- ChatGPT users don't meaningfully engage in comment threads, even if they have the rep to comment. (A user who engages in comment threads is probably not using an AI)
- ChatGPT users don't fix their answers when problems are reported. They often can't because they don't understand the corrections. (A user who makes improvements to their answer or addresses problems that are reported is probably not using an AI)
- ChatGPT users never have unique insight. (No frame challenges, no considering problems outside the question's presented scope, etc. - a user who does these things is probably not using AI.)