Should this stack ban ChatGPT?

Question

Over at Meta Stack Exchange, the guidance to date is that each Stack Exchange network site needs to decide for itself whether to ban ChatGPT-generated posts. RPG SE Meta is the correct format for arriving at a consensus for that. Stack Overflow has banned it. RPGSE does not have a per site meta entry in this thread. If nothing else, this Meta will allow us to share our policy, or discussion, on this topic with the other sites on SE.

I recommend that we ban it.

We have sufficient challenge curating the answers and questions posed by real people on a topic that has some subjectivity to it: game playing, and specifically TTRPGs. Under the basic premise that the SE value proposition is favorable signal to noise ratio, ChatGPT only provides noise.

Is the RPG SE community in general agreement that we ban it, or, are there valid reasons not to enact a ban?

---

@ThomasMarkov kindly added this meta to the long list on Meta SE.

Answer 1

Ban Them For The Foreseeable Future

...If we can.

In addition to messing around with these as a hobby, I have some formal academic knowledge as well. I think they should be banned because I am not convinced of their reliability in the domain of this stack and because they do not in any normal human sense "understand" what they say.

To take a tangent, consider the text to image engine of your choice. They all of them (to my knowledge, as of today) tend to produce portraits with malformed hands. To a degree, this is a training problem. The images are generated from 2D training sets which makes it extremely hard-- almost impossible-- for the AI to "understand" that a hand with fingers and thumb isn't some arbitrary set of flesh tentacles, but instead a very complex but still constrained shape with about a dozen and a half degrees of freedom. That lack of "understanding" causes really creepy and off-putting images.

Also, they really struggle with relationships between figures. You can often get an image of one person dressed or even posed a certain way, but getting two persons engaged in a specific activity or in a specific relationship is still very hard. The "understanding" of those activities and relationships is lacking.

The ChatGPT and its relatives are similar. What they do is manipulate symbols according to vast set of consistency constraints which provides a very compelling illusion of "understanding." But there is no understanding there.

In our case especially, there is simply no good reason to trust ChatGPT for anything approaching a good-subjective question, for the simple reason that the AI hasn't actually run or played in a game, and has no experience of it. At some point that will change-- just for fun someone will sign a bot up for an e-mail text-only game-- but even then, the AI will have no interior or emotional experience of the game as we do.

Answer 2

YES

AI clearly doesn't understand RPGs, or the differences. All the apparent AI generated content of the last days was so low quality, that we had to throw it out.

The main reasoning I suggest this is not to prevent its usage but to have a formal rule we can point to and based on that handle an offender's suspension.

Examples of text that reads like bad AI generation: https://rpg.stackexchange.com/posts/204619/revisions and https://rpg.stackexchange.com/posts/204618/revisions and https://rpg.stackexchange.com/posts/204642/revisions

Example of a better AI-generated answer, which is factually completely wrong: https://rpg.stackexchange.com/posts/204344/revisions

Let's ban Abominable Intelligence and its products!

Answer 3

Yes, a site-policy against ChatGPT is safe and useful

This is a repost of my argument from Worldbuilding Meta. If you read it there, this will be some deja-vu.

I argue that ChatGPT answers are both qualtiatively and quantitatively different than run-of-the-mill low effort answers from new users, and that a rule against them will make things better.

ChatGPT acts like a ban-evading hydra

If we ban someone for being disruptive, and they come back under a new account (in ways that we can prove is the same user), we don't treat the new account as a new person and start over, giving the "new" member of their community a chance - we ban the sockpuppet.

If we find the current instances of ChatGPT disruptive enough to ban, we should treat the others as sockpuppets of the same. It's an extraordinarily well-resourced hydra that gets humans all over the world to sockpuppet for it, but it's a hydra, nonetheless.

ChatGPT users have no possibility for reform other than to stop using ChatGPT

If a human user posts an answer that is under-researched (or answers relative to the wrong system), they will receive downvotes and comments explaining what's wrong. They might fix the answer, delete it, or just leave it downvoted, but whatever else they do, they will also incorporate that feedback into their internal model (or "Learn," as we humans call it). If their goal is to get upvotes, they will research their answers better. If their goal is to avoid criticism, they will leave the stack. Either of these outcomes is acceptable.

ChatGPT users are different. Any feedback we give to ChatGPT users gets blackholed. ChatGPT is not going to update its model in response to our feedback, in part because it never receives our feedback. The user might look at the feedback (and may even learn something from it), but since we're never seeing their writing anyway, the quality of their posts does not improve.

I think most ChatGPT answers are made in bad faith

I suspect most ChatGPT users know full well what they're doing. The technology has been discussed enough now that its users can be expected to know that ChatGPT doesn't know anything they don't. People who continue to post uncited ChatGPT answers are likely doing so knowing full well that they cannot defend the answer's quality.

I can only speculate about their motivations for doing so. Some might be earnestly trying to see how well ChatGPT does with creative and esoteric prompts and others might be trying to test our community to see how well we can distinguish robots from humans, and still others might be trying to have some "harmless" fun - but I suspect most are trying to farm accounts with history and rep (and the privileges that accompany it), to sell them to spammers.

Regardless of their motivations, all motivations I can imagine have two things in common:

• A ChatGPT poster is not making a good-faith effort to provide a high-quality answer to the question asked.
• A user whose first few posts are all unedited and unsourced ChatGPT responses is unlikely to ever contribute anything else to the site, and we should act accordingly.

So why a policy?

A policy against ChatGPT serves two purposes: it will reduce the incentive to post such answers, and it will make moderation actions against ChatGPT trolls more efficient and consistent.

Reducing incentive to post

People who post bad-faith ChatGPT answers are doing so with a purpose: they want an account with rep and history. If we make it clear, in policy and action, that repeatedly posting ChatGPT answers will not get them an account with rep and history, we take away that incentive. If we fail to do so, others will join them.

Making it easier to moderate

We do not, as far as I know, ban people for making low-quality answers in good faith. I certainly don't think we should ban users just because their posts are low quality (as long as they're responsive to feedback and don't repeat mistakes excessively). The problem is that usage of ChatGPT implies that someone will not be responsive to feedback and that they will repeat mistakes excessively. A policy against it allows moderators to act on that evidence, taking the usage of ChatGPT as evidence of bad faith, which will allow quicker, more consistent, and more objective moderation.

What about false positives?

I don't think anyone's suggesting a zero-tolerance rule for anything that looks like ChatGPT (like some of what's happened on r/art). ChatGPT answers might be difficult to distinguish from human answers, but ChatGPT users are not hard to distinguish from human users.

AI can be very convincing. If OpenAI, Google, or Microsoft made it a project to create an AI that could participate on Stack Exchange in a human-like manner, they might succeed - however, most of what we're dealing with right now are script kiddies playing with a new toy, and we can look for human factors indicative of AI usage.

• ChatGPT users can post long answers very quickly. (A user who posts a lot of long answers in a short amount of time is likely either using AI or plagiarizing from somewhere else)
• ChatGPT users don't meaningfully engage in comment threads, even if they have the rep to comment. (A user who engages in comment threads is probably not using an AI)
• ChatGPT users don't fix their answers when problems are reported. They often can't because they don't understand the corrections. (A user who makes improvements to their answer or addresses problems that are reported is probably not using an AI)
• ChatGPT users never have unique insight. (No frame challenges, no considering problems outside the question's presented scope, etc. - a user who does these things is probably not using AI.)

Answer 4

I'm going to reiterate the points I made when this question was proposed at the network-wide level in a more succinct form; you can read my answer in the link if you want the long form.

Yes, ChatGPT (and other AI-generative tools) need to be banned on RPG.SE

And it's for the simple reason that these tools, at least as they exist today, are incapable of producing output that does not commit plagiarism.

There's a few criteria I would line out that I think are necessary prerequisites before they should be approved for use on this site (or any site in the Stack Exchange network), and they are

• That the tool clearly cites every single work that was used as part of its training and ingested into its dataset
• That every single work that was used provided explicit, specific, opt-in permission by its original author to be used as part of that specific tool and its training process
• That the evidence of permission for each work that was used is easily accessible
• That the tool is able to cite, in its resulting output, the works sourced that were used to generate its output.

To my knowledge, none of ChatGPT nor any other publicly available AI-generation tool is capable today of satisfying all these criterion (and it's my understanding that most-if-not-all of these tools are technologically incapable of doing so), and they should not be permitted until/unless they are developed to be able to satisfy these criterion.

Answer 5

As a 20+ year AI developer, I say, Yes, we should ban it.

I have never touted my background on any answer on this stack before, but I want to say, I have done not just AI generally but natural language processing in particular, for a lot of companies, and I have over 20 issued (not just pending) US patents in this area -- and I say we need to ban it, because of how it works.

ChatGPT uses RLHI (Reinforcement Learning with Human Inputs). A number of systems like this are in the works from several companies. Each one is a statistical "big data" approach, through which it harvests snippets (phrases) from sentences all over the web (and/or from directly collected human responses), then parses them (first syntactically, then semantically), and finally, reassembles them to make an answer -- then waits to see if the user who asked it the question gives good or bad feedback.

The positive impact of that feedback is slow and gradual, at best, and depends on the quality and reliability of the feedback. The system tends to take no feedback as a soft confirmation that it is not wrong.

Until and unless a very strong domain-specific model is built within ChatGPT that keys directly into RPG systems (it would literally have to train, for example, on this stack) -- it will not be reliable. And if it did train on this stack, we would notice it - we'd see phrases from past answers getting parroted in bits and pieces in the new answers it creates.

And BTW, similar systems are coming from Microsoft and Google, and possibly later from Facebook and Apple, so, when we write the policy, it shouldn't say "ChatGPT", it should say something like "machine learning-based chat bots including but not limited to ChatGPT".

Answer 6

Yes, we should ban ChatGPT.

I think there are a lot of good reasons to ban ChatGPT, but the one I want to focus on is about site health. If we allow ChatGPT, and if it gets to the point where generative text AI can give good answers to questions here, then the dominant point-gain strategy of this site stops being "have expert knowledge, be an active participant" and becomes "be the first person to post whatever ChatGPT says".

To be clear, I don't see this as a short-term problem. I don't think ChatGPT as it stands today poses any risk of usurping us as RPG experts. But that can't be guaranteed forever. This assumes that all of the other reasons to ban ChatGPT are solved: that it doesn't plagiarize work, that it gives useful answers, that it has some sense of correct information and not just model fit, etc.

The point structure of Stack Overflow/Stack Exchange as a whole is built to encourage good answers from experts. I like helping people, but also I like seeing the score go up on a post I've made. Posting answers that get up votes makes me feel good, and makes me more likely to keep coming back. If my expert knowledge is being out-scored by an AI, why would I bother answering?

ChatGPT has its own value proposition. If people want to get AI-generated advice on TTRPGs, they can ask an AI. I think there's value on this site that gets substantially diluted if we allow people to repost answers taken from generative AI rather than relying on actual TTRPG knowledge taken from experience.

Answer 7

Does it even matter?

I don't exactly disagree with the other answers, but I think there's an important thing to be pointed out here: The main reason to ban ChatGPT is because it generates terrible, incoherent posts. Those posts are going to get downvoted and/or deleted because they're terrible regardless of what we decide here, whether or not they're recognized as AI-generated.

Future, better AIs might be able to make posts that are indistinguishable from human posts. At that point, such a ban will become unenforceable. But for now, I think it mostly doesn't matter whether we ban ChatGPT per se, because its content is de facto already banned for being garbage.

(Note that this doesn't apply to Xirema's answer, which is a separate reason to ban AI-generated content aside from its quality.)

Answer 8

Ban users that blindly regurgitate ChatGPT answers (or questions)

The problem we're facing is not that we have answers generated by ChatGPT. It's that we have an overflow of bad, low effort answers. To draw an analogy: suppose that in the early days of google, we had an influx of answers that simply google'd the question and pasted the top result. Would we ban using google? I'd hope that we'd instead ban users that exhibited this problematic behaviour instead.

So yes, we should have a category separate from LQ with a "fast track" to ban. Eg "keeps posting ChatGPT generated nonsense" where the fact that's generated exacerbates the problem and requires faster and more severe measures, especially if the entire answer is generated. But that doesn't require banning the tool itself.

How this answer is different from the others:

1. It focuses on user behaviour, not an arbitrary restriction on tools
2. It (hopefully) discourages a witch hunt: "oh, your third sentence really sounds like a ChatGPT one" and focuses on the merits of the post
3. It allows carefully curated usage, similar to using google to search
4. It avoids philosophical debates of whether an AI really understands RPGs
5. It keeps us impartial. I'll admit that when I first saw some generated text I felt a pang of resentment - wow, what's the point of reading and learning? This is obviously not the case; but I'd hate to have a policy that someone will look at and say "ha, you're just insecure".
6. It keeps the rule enforceable and doesn't entice people to view it as a challenge.

A note on ethical concerns: While it's true that there are some very valid points, I think it's the poster's responsibility. If their work was transformative enough and they ensured there's no plagiarism, I don't see why they should be treated differently. Training sets are an interesting conundrum; yet again, do we ask how the user's spell checking or grammar checking was trained? Do we even check if they purchased the DMG? Obviously this is something to revising once, as a society, we respond to this change.

As a disclaimer, I have not used ChatGPT to any of my answers, nor do I want to; but I strongly feel that our rules should be based on objective criteria and avoid sensationalist language and knee-jerk reactions. If ChatGPT can pass the RPG stack's Turing test? So be it; in the meantime, warn people against it and ban problematic behaviours.