A proposal for moving forward with Rules as Written

Question

With a lack of clear data on intended use being a major sticking point in the recent discussion on the rules-as-written tag, we’d like to make a proposal that should gather useful data on how often the tag is used for its intended purpose.

We’d like to propose that we do the following:

• Change the rules-as-written tag to rules-as-written-only to clarify its intended use to new users just looking for a rules tag, and updating its tag wiki to clarify its use per the meta question What, exactly, is the RAW tag for?
• On any question where it’s not explicit in the body that the intent of the OP (and doubly so for new users) is for RAW we ask if they’re sure RAW is what they want with a canned comment that makes it clear the tag’s intended purpose (rules only, no common sense interpretation/house rules).
• Remove the tag when an OP indicates that strict RAW is not their intent
• Aggressively police answers on RAW posts to remove answers that rely primarily on house rules or common sense interpretations.
• Each question that has had the tag removed will be counted as an incident of misusing the tag. Cases of the OP not responding but then accepting non-RAW answers or objecting that the community removed non-RAW answers that were useful will be counted as mistagging as well, and the tag removed. Questions that keep the tag during the collection period will count as correct taggings.

The current suspected problem with rules-as-written is largely tied up in the fact that it A. gets applied to questions where the OP is simply looking for how to interpret the rules and grabbed the only tag that really starts with “rules”, and B. that it’s often answered with “this is what the rules say, and here’s interpretation/house rules that fill in the gaps.” These two suspected problems make the tag relatively meaningless in those contexts. The process presented here seeks to collect data on how often the tag is misused, if at all, allowing the community to proceed in a data-driven way toward resolving the perception of issues with the tag.

We’d like to start this about a week from today, and run this data collection for about a month. In a month we can evaluate and summarize the data, with the intention of eliminating the need for any stakeholders to guess about whether and how much mistagging is a real problem.

The community can help this process by aggressively flagging answers that are non-RAW on questions that have had the rules-as-written-only tag confirmed as understood and intended by the OP.

While it would generate better data, we don’t think that question closures pending OP confirmations of intent is necessary.

Answer 1

I have a modification to your proposal to make it compatible with how RAW works when it's functioning just fine.

This answer isn't an endorsement of the plan. Treat it as an "if this goes ahead, do it like this."

Be sensitive to the RAW tag's community during this

Be sensitive to the requests of the people active in RAW here, like HeyICanChan and KRyan, when you're moderating this tag.

Frankly, this is just so that you don't risk looking like you're mucking things up — it would suck to carry out this plan, keep a good score, and to reach the end of it with the community saying, "yeah, but we only got this tally because the mods mucked things up." Put the ball in the court of the RAW tag's regulars by listening to them, that way if the experiment does indicate problems, the data's going to be fairly reliable (or at least, if there are problems, it won't be all your fault).

Mxyzplk and SevenSidedDie have both expressed they don't like RAW play. To their credit, SevenSidedDie's expressed he'll fight to support it here anyway. You need to be careful you don't look like you're accidentally breaking things through ignorance or deliberate harm.

Police these normally, and by community consent.

A couple of parts of this proposal are concerning in that they may hurt RAW questions & answers more than help them, but I think it's pretty easy to make workable.

Aggressively police answers on RAW posts to remove answers that rely primarily on house rules or common sense interpretations.

As described in my answer on what the RAW tag is for: "[Homebrew is] useful if the rules are terrible, such as in einn and tveir." Good RAW answers should absolutely definitely tackle the RAW.

However, the examples I provide do that and then also go on to offer alternatives from homebrew and common sense and whatnot. That comprises a good chunk of their usefulness. So really, this should be: "Police answers on RAW questions that don't tackle the RAW situation at all."

I suggest you also keep the policing gentle. If nobody else has done so, leave a comment requesting improvement: "You don't seem to have described the RAW at all, so this isn't answering the question. Could you do that? Also, have you tried these house rules you're suggesting, how did they work out?" Do delete answers where the author refuses to do that (or never gets around to it), but that's just business as usual, I think. Implementing Game Rec levels of draconian deletion is unnecessary for data collection.

Cases of the OP not responding but then accepting non-RAW answers or objecting that the community removed non-RAW answers that were useful will be counted as mistagging as well, and the tag removed.

Non-RAW here should be answers that never adequately tackle the RAW. As in, crap ones as I described above. If it handles the RAW but then also provides common sense and house rules, don't count it as a "non-RAW" answer.

Keep a public register of RAW tag misuse

If I understand correctly:

• You're going to be counting misuses and correct uses of the tag.
• We're going to be keeping score of how many of these incidents occur, in order to get some data on what's going on.

So I suggest: Start a meta Q&A recording these incidents publicly. When you encounter a question or answer indicating a misuse, record it there. Link to it, quote comments if it helps. This is so we can get eyes on what's happening as it happens, and make sure the data's good. (This is part of the whole "don't risk looking like you're mucking things up" thing I mentioned above, too.) I presume there'll only be like, a couple of dozen RAW questions max during the month this runs, so it should be manageable.

Allow for people to point out false matches if they can show why. Let's take that "einn" example from above: it's a good RAW answer to a good RAW question, and also contains some homebrew. Let's assume a mod takes it as indicating a misuse: we're human, they're human, everyone makes mistakes. They put it on the list, it puts the score up 1, we get to point out "uh, that's actually a perfectly good RAW answer", it goes off the list and the score goes down 1.

Discussion over whether something's a false positive should probably be confined to the questions themselves, not the meta Q&A — comments on the meta Q&A should draw attention to a question at best, the meat of the discussion shouldn't happen in there.

Answer 2

OK, this is just patently absurd, and I cannot believe that this is what we have come to. I am just going to address the elephant in the room, because I believe that too much of this discussion has suffered for trying not to do so.

The moderation team is not competent to perform the policing that they propose

Two-thirds of the moderation team have an avowed and public “hatred” for rules-as-written as a playstyle. The list of questions that mxyzplk chose to bring up this discussion, which he labeled as not being about the rules as written, was, in my opinion, poorly chosen, and further indicate that he is not in a position to make these judgments.

Moreover, quite frankly, the moderation team cannot be trusted to perform this policing. Again, two-thirds of the moderation team have publicly stated a hatred for this playstyle. One of them has done an admirable job of mostly keeping his opinions out of his actions on this site, but the other has a long history of vitriol, personal attacks, inflammatory statements, and outright persecuting the rules-as-written community.

On his personal blog, mxyzplk writes:

You know what? It’s time to bring back some of the pejorative terms of gaming 20 years ago.

Rules lawyers. Munchkins. Power gamers. Monty Haulers. You’re on notice. Somehow your filthy habits have become mainstreamed, over Gary Gygax’s dead body apparently. But you’re not welcome, around here at least.

That’s from a personal blog, but that same attitude has influenced and informed his interactions with the rules-as-written community here, as well. This is difficult to demonstrate because of comments being deleted, so I can only assure you that it is the case—but for myself, I have literally never had a conversation with him in which he did not insult me and everyone I play with. That unwelcome is something I have experienced constantly on this site, almost entirely due to his presence—and while I have fought tooth and nail to carve out a place for myself and my preferences, he has driven off numerous others, and still more have avoided joining this website because of him.

mxyzplk not only has avoided suspension despite engaging repeatedly, flagrantly, and over a long period of time numerous activities defined as being cause for suspension, he has furthermore retained his moderator position, which is a strong sign to the rules-as-written community that they are treated here as second-class citizens. Leaving an avowed bigot in charge is not appropriate, but that is what has happened. So far as I can tell, the only result of his behavior has been that he has been told to tone it down and keep his opinions to himself.

And now we are being asked to trust that self same opinion in the judgment and policing of a topic that he actively despises.

No. Under no circumstances will I accept this proposal from this moderation team. I am actually flabbergasted that they had the gall to think they were in a position to suggest it.

If the moderation team wants to try something like this out, they need to show that they are actually accepting of rules-as-written enthusiasts as a part of this community. They cannot do that while mxyzplk is still a moderator. He has to go, or this is just untenable.

Answer 3

Don't rename the tag during the study

You say:

With a lack of clear data on intended use being a major sticking point in the recent discussion on the rules-as-written tag, we’d like to make a proposal that should gather useful data on how often the tag is used for its intended purpose.

So you want to find out how people respond to and use the RAW tag, right? That's a great idea, and we need that data sorely so we can see what problems exist, and agree on where to go with it from here.

But then you say:

• Change the rules-as-written tag to rules-as-written-only to clarify its intended use to new users just looking for a rules tag, and updating its tag wiki to clarify its use per the meta question What, exactly, is the RAW tag for?

This is an admirable thing to do, but it compromises your earlier stated mission. We need data on how people are using a tag called [rules-as-written], right? Well, study that tag then, not a new one you're experimenting with.

Renaming the tag here is a bit like saying "we want to see how apes respond to bananas. But, also, first we're going to replace all the bananas we have with apples. Then we'll measure their response to apples. That will give us data on how they respond to bananas." Which is, y'know, a little nuts.

I get what you want to do is clarify things and that's a good idea, but it's going to equal tainting the data right now. Here's the process you should follow, which I think you're jumping ahead on:

1. Study the behaviour in the rules-as-written tag as-is for a while.
2. Review data, see if we should take any actions.
3. Assuming we decide it is a good idea: rename the tag to rules-as-written-only as a trial. Study that tag for a while.
4. Review data, see if it improved things, and if there's more action needed.

Your study's goal is to help us reach step 2, but it really gets us to step 4 but without even any data from step 2 to compare against.

You want to reduce misunderstandings, but how often that happens in the first place is actually valuable data we need for step 2. We also won't understand if a rename to [rules-as-written-only] has any effect without that data.

Be like David Attenborough. If you want to observe the thing in its natural habitat with its normal behaviours and document it, leave it in peace as much as you can.

Answer 4

Extend the timeframe beyond 1 month

I counted up the questions per month in the rules-as-written tag since it can be counted fairly quickly. (This doesn't cover tag removals and deletions but since I have no info on that, I'm ignoring them.) There were:

• 30 questions for January 2016
• 21 in December 2015
• 25 in November
• 31 in October
• 46 in September
• 39 in August
• 34 in July
• 33 in June
• 33 in May
• 31 in April
• 25 in March

History repeats itself and that looks like we usually get 30-ish RAW questions a month.

30-ish is a poor sample size to operate on. Uncommon events can be misrepresented by not showing up at all or being over-represented at that scale.

I suggest we gather data for at least two months. At the end of two months, we consider if we should run it for a third. As an absolute limit, once we reach 100 questions, the month we're on becomes the last — we'll keep gathering questions for that month, but we don't renew for another. That'll probably mean 3-4 months tops, and a quarter or a third of a year is a pretty decent sample. That's a while but good data collection doesn't come cheap and easy and if we're going to do it at all we might as well do a good job of it.

Answer 5

With respect to the fact there's some ongoing matters to discuss connected to this (Let's do that thing where we use our words), I'd like to suggest an alternate proposal for moving forward that takes stock of the comments and answers thus far (and the hefty amount of upvotes distributed).

This is intended to replace the original five "We’d like to propose that we do the following" bullet points. Most of these changes are on the basis that if we want to gather data on business as usual, we should not be altering business as usual at the same time. The rest are just expanding on how data gathering will happen.

I propose we do the following:

Moderation (by the community & diamond mods)

• On any question where it’s not explicit in the body that the intent of the OP is for RAW, we ask if they’re sure RAW is what they want with a canned comment that makes it clear the tag’s intended purpose (primarily, delve into RAW even when it gets complicated, propose house rules and common sense only when that's been done to satisfaction).
• The canned comment is discussed by the community first because we all know this is very likely to start fights if that doesn't happen first.
• Diamond moderators moderate as they usually would.

The RAW tag gets removed in the following cases, because they indisputably show misuse of the tag:

• The OP indicates (in response to that canned comment) that strict RAW is not their intent.
• The OP uses the [rules-as-written] tag, then accepts an answer that does not cover RAW. (Or does so utterly and indisputably inadequately.)

Dispute over these actions gets discussed in the post's comments as normal.

Data gathering

(This mostly was just compressed into the last bullet point of the original proposal. Public logging is the plan, which is great, and it should get more detail in the proposal itself.)

Data will be publicly logged in a public meta Q&A by all participating users including diamond moderators, with links to the questions or answers involved. The following categories of events in particular will be highlighted because they're of particular interest and relevance to the debate:

• A RAW question gets asked at all. Some of these will have the tag removed, so we'll log them here anyway for later reference. (Because people would no doubt wonder how many we've covered so far, I suggest we keep count by putting them in a numbered list, and number all items with "1." in markdown - which is just a neat way to auto-number them.)
• The RAW tag gets removed for the above reasons.
• The accepted answer covers the RAW in nebulous detail.
• The accepted answer covers the RAW, as well as house rules or homebrew.

(Naturally, posts can belong to more than one of these groups.)

Other categories can come up, preferably as/before we begin in order to ease maintenance. Each category should be handled in a separate answer to minimise edit collisions (tightly related categories can stay in one answer, use your judgement). New categories can be brought up in new answers by whatever groups or individuals care to curate it, so that whatever details are of interest to various parties for debate & discussion purposes can be covered. (Some important categories may only become evident once we've already begun.) Duplicate categories should be avoided. Useless categories should earn downvotes.

Records that go obsolete (the accepted answer gets changed, for instance) should be kept on the list as a thing that did happen and marked obsolete. (If in later doubt and we have such records, we might need diamond mods to tell us whether it's true.)

Data will be considered indicative, not comprehensive. This is a sample, not a census, and has all the strengths and weaknesses of a sample. Statistical anomalies will creep in, but the data will at least give us a general idea of what occurs or can occur in the tag.

Gone completely:

• Renaming the tag. Don't change anything about it. (Except maybe updating the tag wiki. Since tag wikis are notorious for rarely if ever getting read, and the existing one is already accurate, if a little lacking in straightforwardness, I expect it won't change behavior much.)
• Any notion of "aggressive" moderation. It's completely unnecessary. Moderate per usual.
• The notion that "RAW" means "no answers should ever mention house rules or common sense interpretations".

Answer 6

March 2015 RAW question Analysis

I have examined all RAW tagged questions for March 2015. My detailed notes are in a text file that can be found at this link. The games covered were mostly D&D/Pathfinder sorts of various editions, one from Spycraft, and an OSRIC which is AD&Dish.

Caveat: as I have no idea how many comments were deleted, I have no sense of how much back and forth happened on all questions. Quite a few still have significant comments remaining.

I did not do a detailed analysis of edit history on each question, but I am not sure I need to.

1. Summary. 25 total questions (two "duplicates" so 23 total questions assessed).

2. Results: (A second review was done to ensure that text supporting need for the tag was found in the question to support valid use of the tag.)

   Good use of RAW tag in the question: 15
   Poor question and / or use of RAW tag not needed in the question: 8
   Good RAW answer: 26
   Poor or non-RAW answer: 9
   Deleted answer: 3

3. A few other notes:

   Four questions did what this one did and explicitly stated in text a very hard RAW requirement.

   Some answers, this is an example, had text refer RAW tag and made a solid RAW type answer.

   In a five of the answers, the responses agreed that the rules are muddy at best -- at this point the better RAW style answers expanded upon that and the less good answers didn't. In one muddy rules case, I'll pull a comment from this question as an example of what was referred to in that other meta as where a RAW question may needs a frame challenge to get a decent answer. Where the are hard to parse, the rules being messy should not negatively influence use of the tag. (See also if there are so many books one knows not of a rule ...) The other answer got more votes, though the gap in the rules didn't help.

   +1: While not exactly fulfilling the RAW tag, this is still a great answer as it provides a practical way of playing this scenario. – > GreySage Jul 16 '15 at 16:21

   This one could have done without the tag. Neither the question nor the answers were of high standard.

   On this one, the usual trouble folks have with how alignment is handled in 5e, as opposed to previously. I think that is a root cause for this poor question -- RAW or not.

   This one was a bit of an anomaly. The accepted answer had one vote. The comment from the querent was that the answer was accepted as the best at being RAW fosuced -- all of the answers were decent, one had 11 up votes ... three other answers total available with more votes than accepted answer.

Answer 7

Yes

We should do this thing. All three diamond mods support it. It seems like a concrete step to gathering more data to make a decision in a way that is minimally disruptive to the site.