7
\$\begingroup\$

With a lack of clear data on intended use being a major sticking point in the recent discussion on the tag, we’d like to make a proposal that should gather useful data on how often the tag is used for its intended purpose.

We’d like to propose that we do the following:

  • Change the tag to to clarify its intended use to new users just looking for a rules tag, and updating its tag wiki to clarify its use per the meta question What, exactly, is the RAW tag for?
  • On any question where it’s not explicit in the body that the intent of the OP (and doubly so for new users) is for RAW we ask if they’re sure RAW is what they want with a canned comment that makes it clear the tag’s intended purpose (rules only, no common sense interpretation/house rules).
  • Remove the tag when an OP indicates that strict RAW is not their intent
  • Aggressively police answers on RAW posts to remove answers that rely primarily on house rules or common sense interpretations.
  • Each question that has had the tag removed will be counted as an incident of misusing the tag. Cases of the OP not responding but then accepting non-RAW answers or objecting that the community removed non-RAW answers that were useful will be counted as mistagging as well, and the tag removed. Questions that keep the tag during the collection period will count as correct taggings.

The current suspected problem with is largely tied up in the fact that it A. gets applied to questions where the OP is simply looking for how to interpret the rules and grabbed the only tag that really starts with “rules”, and B. that it’s often answered with “this is what the rules say, and here’s interpretation/house rules that fill in the gaps.” These two suspected problems make the tag relatively meaningless in those contexts. The process presented here seeks to collect data on how often the tag is misused, if at all, allowing the community to proceed in a data-driven way toward resolving the perception of issues with the tag.

We’d like to start this about a week from today, and run this data collection for about a month. In a month we can evaluate and summarize the data, with the intention of eliminating the need for any stakeholders to guess about whether and how much mistagging is a real problem.

The community can help this process by aggressively flagging answers that are non-RAW on questions that have had the tag confirmed as understood and intended by the OP.

While it would generate better data, we don’t think that question closures pending OP confirmations of intent is necessary.

\$\endgroup\$
  • 5
    \$\begingroup\$ "Aggressively police answers on RAW posts to remove answers that rely primarily on house rules or common sense interpretations." - is this any change from the norm, or just what you already do? I'm not sure I'm ready to ask for more aggressive policework than what you already do, and I don't want it to become the next Game Rec that's only accepted because of draconic enforcement. \$\endgroup\$ – doppelgreener Feb 3 '16 at 3:57
  • 2
    \$\begingroup\$ @doppelgreener that's something we want to avoid long term as well. Aggressive policing will be largely a part of this fact finding mission and if too much is required to make this work, the tag probably isn't viable (or an alternate solution will need to be pursued)...Ideally we will find that that kind of effort isn't required. \$\endgroup\$ – wax eagle Feb 3 '16 at 3:59
  • 15
    \$\begingroup\$ I'm generally in favour, but I'm concerned that genuine frame-challenging will be censored when it really shouldn't be. Tagging RAW shouldn't obviate good frame challenges, and accepting a good frame challenge as an answer shouldn't mean that the RAW tag was in error. How can we keep this good practice from becoming a casualty of the experiment? \$\endgroup\$ – BESW Feb 3 '16 at 4:31
  • \$\begingroup\$ @BESW That's a very fair concern. I think that they will have to pass the same bar that the questions themselves have to: show clear understanding that this is a RAW question and clearly present a frame challenge (thought not necessary to do so by name). Answers that might be frame challenges when squinted at, or might be just failing to read the question, would be deleted. I haven't deeply pondered this issue though, so that's a first stab at what seems both reasonable and useful during this collection period. Does it sound reasonable and useful? \$\endgroup\$ – SevenSidedDie Feb 3 '16 at 5:03
  • 4
    \$\begingroup\$ Toward the end of the month we might want to check if we have enough data, and consider running it another month for more. \$\endgroup\$ – doppelgreener Feb 3 '16 at 6:51
  • \$\begingroup\$ Is the tag wiki updated already, or is that a work in progress? \$\endgroup\$ – KorvinStarmast Feb 3 '16 at 14:02
  • \$\begingroup\$ @KorvinStarmast It isn't updated and hasn't featured as a relevant issue in the discussion yet. We should discuss that in the question about the tag wiki, not here. I plan to do it this weekend unless I have time earlier, but I probably won't. If someone else beats me to it, all the better. \$\endgroup\$ – doppelgreener Feb 3 '16 at 14:45
  • \$\begingroup\$ @doppelgreener Understood, will review as time permits. \$\endgroup\$ – KorvinStarmast Feb 3 '16 at 14:46
  • 2
    \$\begingroup\$ I wish to echo @doppelgreener's point from a comment lower down that "aggressively flagging" is not necessary as it may create an unintentional bias, and certainly will create the perception of bias in at least part of our community. Insofar as analysis is concerned, we may need to break up the task into large chunks on the historical basis, per a exchange between me and SSD on how big a job it is to gather comprehensive data. \$\endgroup\$ – KorvinStarmast Feb 5 '16 at 15:46
  • \$\begingroup\$ "On any question where it’s not explicit [that they want RAW] we ask if they’re sure RAW is what they want with a canned comment that makes it clear the tag’s intended purpose (rules only, no common sense interpretation/house rules)." Please vet this canned comment with us, or open the floor to discuss what it should be, before things go into action. That comment has a lot of potential to influence the outcome here toward certain biases. For example, the bit in brackets stated as such is inaccurate for what's expected in the RAW tag (see comments on my answer). \$\endgroup\$ – doppelgreener Feb 7 '16 at 15:21
  • 2
    \$\begingroup\$ My problem is this part: "B. that it’s often answered with “this is what the rules say, and here’s interpretation/house rules that fill in the gaps.”" That makes no sense. RAW questions must accept that sometimes the RAW does NOT answer the question. I would assume it is analogous to the "don't do that" answers on other stacks. \$\endgroup\$ – user2102 Feb 22 '16 at 18:32
  • 2
    \$\begingroup\$ -1 for "Aggressively police answers on RAW posts" that is what up/down votes are for, but given the other issues with this tag, I certainly won't be answering (m)any questions sporting it. \$\endgroup\$ – user2102 Feb 22 '16 at 20:03
  • \$\begingroup\$ FWIW, I fell into the trap of this tag today, in particular because the tag wiki is less than clear. "Published" how? "Written" where and by whom? We have house-rules, so maybe official-rules could be a clearer counterpoint. \$\endgroup\$ – Raphael Sep 3 '16 at 20:34
31
\$\begingroup\$

I have a modification to your proposal to make it compatible with how RAW works when it's functioning just fine.

This answer isn't an endorsement of the plan. Treat it as an "if this goes ahead, do it like this."

Be sensitive to the RAW tag's community during this

Be sensitive to the requests of the people active in RAW here, like HeyICanChan and KRyan, when you're moderating this tag.

Frankly, this is just so that you don't risk looking like you're mucking things up — it would suck to carry out this plan, keep a good score, and to reach the end of it with the community saying, "yeah, but we only got this tally because the mods mucked things up." Put the ball in the court of the RAW tag's regulars by listening to them, that way if the experiment does indicate problems, the data's going to be fairly reliable (or at least, if there are problems, it won't be all your fault).

Mxyzplk and SevenSidedDie have both expressed they don't like RAW play. To their credit, SevenSidedDie's expressed he'll fight to support it here anyway. You need to be careful you don't look like you're accidentally breaking things through ignorance or deliberate harm.

Police these normally, and by community consent.

A couple of parts of this proposal are concerning in that they may hurt RAW questions & answers more than help them, but I think it's pretty easy to make workable.

Aggressively police answers on RAW posts to remove answers that rely primarily on house rules or common sense interpretations.

As described in my answer on what the RAW tag is for: "[Homebrew is] useful if the rules are terrible, such as in einn and tveir." Good RAW answers should absolutely definitely tackle the RAW.

However, the examples I provide do that and then also go on to offer alternatives from homebrew and common sense and whatnot. That comprises a good chunk of their usefulness. So really, this should be: "Police answers on RAW questions that don't tackle the RAW situation at all."

I suggest you also keep the policing gentle. If nobody else has done so, leave a comment requesting improvement: "You don't seem to have described the RAW at all, so this isn't answering the question. Could you do that? Also, have you tried these house rules you're suggesting, how did they work out?" Do delete answers where the author refuses to do that (or never gets around to it), but that's just business as usual, I think. Implementing Game Rec levels of draconian deletion is unnecessary for data collection.

Cases of the OP not responding but then accepting non-RAW answers or objecting that the community removed non-RAW answers that were useful will be counted as mistagging as well, and the tag removed.

Non-RAW here should be answers that never adequately tackle the RAW. As in, crap ones as I described above. If it handles the RAW but then also provides common sense and house rules, don't count it as a "non-RAW" answer.

Keep a public register of RAW tag misuse

If I understand correctly:

  • You're going to be counting misuses and correct uses of the tag.
  • We're going to be keeping score of how many of these incidents occur, in order to get some data on what's going on.

So I suggest: Start a meta Q&A recording these incidents publicly. When you encounter a question or answer indicating a misuse, record it there. Link to it, quote comments if it helps. This is so we can get eyes on what's happening as it happens, and make sure the data's good. (This is part of the whole "don't risk looking like you're mucking things up" thing I mentioned above, too.) I presume there'll only be like, a couple of dozen RAW questions max during the month this runs, so it should be manageable.

Allow for people to point out false matches if they can show why. Let's take that "einn" example from above: it's a good RAW answer to a good RAW question, and also contains some homebrew. Let's assume a mod takes it as indicating a misuse: we're human, they're human, everyone makes mistakes. They put it on the list, it puts the score up 1, we get to point out "uh, that's actually a perfectly good RAW answer", it goes off the list and the score goes down 1.

Discussion over whether something's a false positive should probably be confined to the questions themselves, not the meta Q&A — comments on the meta Q&A should draw attention to a question at best, the meat of the discussion shouldn't happen in there.

\$\endgroup\$
  • 8
    \$\begingroup\$ Public logging is the plan, yes; and answers that amount to “the raw works like so but is bad/dumb/broken, so instead [Good Subjective alternative]” are definitely good answers to RAW questions. (Questions looking for RAW are sometimes going to be about places where the RAW has holes, so pointing out that RAW doesn't exist or is broken is definitely engaging with the question well.) The other notes on handling, tone, and logging details are good. \$\endgroup\$ – SevenSidedDie Feb 3 '16 at 6:57
  • \$\begingroup\$ I don't know that I agree on the homebrew point. If someone answers a rules question with rules where they're clear, common sense rulings where there's gaps or conflicts, or homebrew where there's a gap - that's "how any question would be answered," not RAW. There seems to be some assumption that loads of homebrew is the obvious set of answers for a rules question not tagged [RAW] and that's not the case on the site. The only place for a RAW tag is RAW only, if it's "RAW, or, you know, something else" then that's proof the tag is not needed. \$\endgroup\$ – mxyzplk Feb 3 '16 at 12:37
  • 7
    \$\begingroup\$ @mxyzplk Take a count of those as well if you're interested to see how many turn out that way. Don't count them as "non-RAW" and take action, though, since it would be unwelcome. Useful compromises get accepted when the RAW sucks, but it doesn't make it stop being a question that was about and hoping for RAW just because it didn't get the kind of answer it was hoping for. If you want to change the behaviour and norms around this, I suggest you leave that for after the study, not execute that change during. \$\endgroup\$ – doppelgreener Feb 3 '16 at 13:06
  • \$\begingroup\$ How is that different from any rules question then? IMO if someone asks for RAW but is willing to accept homebrew, they don't understand the term and it's a reason we should change it. Let's say we still had the plain [rules] tag - how would you expect its behavior to be different? If RAW isn't pure RAW then it's just [rules], which we got rid of because it's the default way people act. If we "just count" them they're getting counted in the "death of the tag" column. Policing them will make the difference clearer to people so behavior will align with the tag, making the data better. \$\endgroup\$ – mxyzplk Feb 3 '16 at 13:37
  • 11
    \$\begingroup\$ @mxyzplk Ok, look, this is the thing that other question's about. I'm under the impression this is a data gathering plan, isn't it? Not a plan for the mods to enforce the changes they want on the tag, which the community hasn't agreed with, because we don't agree on what problems actually exist in the tag, because the data we have about it isn't very good, which is the whole reason we have this plan now: so we can make agreements. So this is not the time to force changes on the tag. Run with the accepted idea that RAW answers, that have tackled RAW, can suggest homebrew. Gather data. \$\endgroup\$ – doppelgreener Feb 3 '16 at 14:07
  • 12
    \$\begingroup\$ @mxyzplk The difference is between these: a) OP asks for RAW, gets a RAW answer and a rulings-ful answer, and prefers the rulings over the perfectly-fine actual RAW rules; b) OP asks for RAW, but it doesn't exist and accepts that they either get nothing, or they get a non-RAW substitute. (a) is tag misuse but (b) is showing full understanding and intent to use the tag to mean RAW. In other words, an OP doesn't stop meaning to have asked for RAW just because it turns out it doesn't exist or Good Subjective RAWer experience says respecting it is a bad idea. (a) won't happen super often though. \$\endgroup\$ – SevenSidedDie Feb 3 '16 at 16:20
  • 1
    \$\begingroup\$ @SevenSidedDie That is an accurate summary. Thanks for that. \$\endgroup\$ – doppelgreener Feb 3 '16 at 22:47
  • 1
    \$\begingroup\$ @mxyzplk am I incorrectly reading an exacerbated/defensive tone to your comments? \$\endgroup\$ – user2102 Feb 22 '16 at 18:38
15
\$\begingroup\$

OK, this is just patently absurd, and I cannot believe that this is what we have come to. I am just going to address the elephant in the room, because I believe that too much of this discussion has suffered for trying not to do so.

The moderation team is not competent to perform the policing that they propose

Two-thirds of the moderation team have an avowed and public “hatred” for rules-as-written as a playstyle. The list of questions that mxyzplk chose to bring up this discussion, which he labeled as not being about the rules as written, was, in my opinion, poorly chosen, and further indicate that he is not in a position to make these judgments.

Moreover, quite frankly, the moderation team cannot be trusted to perform this policing. Again, two-thirds of the moderation team have publicly stated a hatred for this playstyle. One of them has done an admirable job of mostly keeping his opinions out of his actions on this site, but the other has a long history of vitriol, personal attacks, inflammatory statements, and outright persecuting the rules-as-written community.

On his personal blog, mxyzplk writes:

You know what? It’s time to bring back some of the pejorative terms of gaming 20 years ago.

Rules lawyers. Munchkins. Power gamers. Monty Haulers. You’re on notice. Somehow your filthy habits have become mainstreamed, over Gary Gygax’s dead body apparently. But you’re not welcome, around here at least.

That’s from a personal blog, but that same attitude has influenced and informed his interactions with the rules-as-written community here, as well. This is difficult to demonstrate because of comments being deleted, so I can only assure you that it is the case—but for myself, I have literally never had a conversation with him in which he did not insult me and everyone I play with. That unwelcome is something I have experienced constantly on this site, almost entirely due to his presence—and while I have fought tooth and nail to carve out a place for myself and my preferences, he has driven off numerous others, and still more have avoided joining this website because of him.

mxyzplk not only has avoided suspension despite engaging repeatedly, flagrantly, and over a long period of time numerous activities defined as being cause for suspension, he has furthermore retained his moderator position, which is a strong sign to the rules-as-written community that they are treated here as second-class citizens. Leaving an avowed bigot in charge is not appropriate, but that is what has happened. So far as I can tell, the only result of his behavior has been that he has been told to tone it down and keep his opinions to himself.

And now we are being asked to trust that self same opinion in the judgment and policing of a topic that he actively despises.

No. Under no circumstances will I accept this proposal from this moderation team. I am actually flabbergasted that they had the gall to think they were in a position to suggest it.

If the moderation team wants to try something like this out, they need to show that they are actually accepting of rules-as-written enthusiasts as a part of this community. They cannot do that while mxyzplk is still a moderator. He has to go, or this is just untenable.

\$\endgroup\$
  • 4
    \$\begingroup\$ @SevenSidedDie Fair enough, updated. \$\endgroup\$ – KRyan Feb 3 '16 at 20:01
  • 8
    \$\begingroup\$ @SevenSidedDie And have moderators, who have access to the flag logs, ever contacted them about this subject? Because I know I have certainly called on them to do so numerous times. I do not have access to all of the information I would need to make a compelling case. It’s the only reason I have tiptoed around this subject for literally years. \$\endgroup\$ – KRyan Feb 3 '16 at 20:04
  • 2
    \$\begingroup\$ Community Managers do have access. If you've investigated the option seriously, you already know that it is a virtual face-to-face process where the CM hears the issue and conducts their own investigation with more tools than even mods have access to. The moderators also cannot meaningfully contact the CMs on a user's behalf. \$\endgroup\$ – SevenSidedDie Feb 3 '16 at 20:06
  • 3
    \$\begingroup\$ @SevenSidedDie I think clearly it’s the former. \$\endgroup\$ – KRyan Feb 3 '16 at 20:09
  • 9
    \$\begingroup\$ @SevenSidedDie A moderation team that hasn't ousted an outright bigot who has persecuted a section of the community for years is a moderation team that I cannot trust, least of all with the subject of that bigotry and persecution. \$\endgroup\$ – KRyan Feb 3 '16 at 20:20
  • 4
    \$\begingroup\$ @SevenSidedDie Also, you are incorrect about the process. “The process will be initiated by a formal request from one or more moderators on a site, sent privately via an email to community@stackexchange.com.” Handling Calls to Remove a Moderator \$\endgroup\$ – KRyan Feb 3 '16 at 20:36
  • 2
    \$\begingroup\$ It is accurate for the goalposts you set up before: the equivalent of suspension. Removal of a moderator is not like a temporary user suspension. To have a moderator reprimanded in the same way is ≠ to having a moderator removed. For this new goalpost: yes, there has been a distinct lack of wax and I initiating a coup to oust mxyzplk, just as there has been a distinct lack of you following the paragraph under the one you quoted. (I remain perplexed by this.) \$\endgroup\$ – SevenSidedDie Feb 3 '16 at 21:39
  • 3
    \$\begingroup\$ On the topic at hand, in the other question everyone wanted more data. Refusal of any data gathering is understandable from those who want the status quo, but it's not going to fly - we need to get real information one way or the other, and that's the clear takeaway from the previous Q. So suggest some other way to do so. \$\endgroup\$ – mxyzplk Feb 4 '16 at 0:11
  • 3
    \$\begingroup\$ For the record, the RAW playstyle is welcome here, per our "All playstyles are welcome" clause, even if I don't play that way. I support all playstyles here on RPG.SE. As long as they conduct themselves civilly, which is I think where you and I have problems (cf. this very post) and you generalize those problems to "he must be against my playstyle and trying to kill it." That's not the case. \$\endgroup\$ – mxyzplk Feb 4 '16 at 0:13
  • 7
    \$\begingroup\$ I would suggest that maybe y'all should just police as usual, and leave out any plans to "aggressively" police things. The normal level of mod policing should be fine, since it's presumably worked OK so far. If we're looking to figure out what's going on, that doesn't actually require aggressive policework at the same time. \$\endgroup\$ – doppelgreener Feb 4 '16 at 0:52
  • 7
    \$\begingroup\$ Honestly... I think this should be a separate meta question. You're calling out at least one of the mods as being biased to the point where they're not fulfilling their role. If there's any truth to that claim, it affects the whole of RPG.SE, not just this one issue. I guess we could suspend the investigation of the rules-as-written tag while we go through whatever the process is for investigating alleged moderator bias, but you could make a similar case for suspending all meta.SE policymaking. Hmm... \$\endgroup\$ – GMJoe Feb 4 '16 at 6:22
  • 9
    \$\begingroup\$ I up voted this comment, and for emphasis wish to repeat this part of it, as stated by @doppelgreener I would suggest that maybe y'all should just police as usual, and leave out any plans to "aggressively" police things. It seems to me that any added energy is better spent in a data collection mode. \$\endgroup\$ – KorvinStarmast Feb 5 '16 at 16:02
  • 5
    \$\begingroup\$ @mxyzplk As the other silver rules-as-written badge holder (I know! Who knew?) who's had no similar problems (that I'm aware of, anyway), I'd like to know if the stance taken in the blog post remains accurate. After all, the blog post is, like, over 5 years old. \$\endgroup\$ – Hey I Can Chan Feb 8 '16 at 18:02
  • 2
    \$\begingroup\$ I could, but I don't intend to justify my off-site writings here, I think that sets a bad precedent. I can only imagine what KRyan says on other sites. As far as my mod voice goes, all game styles are welcome here and I never have or will discriminate against them. All site users are treated equally regardless of game style, insofar as they follow the rules. I have my own thoughts and philosophies on gaming, which find expression in many Q&As here - mods do not need to step back from participating in the site as usual because they're mods (in fact, they bear a responsibility to be active). \$\endgroup\$ – mxyzplk Feb 9 '16 at 1:15
  • 4
    \$\begingroup\$ @mxyzplk That's cool. While I think there's a difference between a gaming site moderator, for example, hating a playstyle on his blog and hating The Princess Bride on IMDB (which, to be clear, you didn't because then you'd be a monster!), I respect wanting to keep the worlds apart. However, were I in a similar position, I know that I wouldn't want to be taken to task for a position I took 5 years ago without being given the opportunity to update that position, so I figured I'd extend you the same courtesy. \$\endgroup\$ – Hey I Can Chan Feb 9 '16 at 17:09
14
\$\begingroup\$

Don't rename the tag during the study

You say:

With a lack of clear data on intended use being a major sticking point in the recent discussion on the rules-as-written tag, we’d like to make a proposal that should gather useful data on how often the tag is used for its intended purpose.

So you want to find out how people respond to and use the RAW tag, right? That's a great idea, and we need that data sorely so we can see what problems exist, and agree on where to go with it from here.

But then you say:

  • Change the rules-as-written tag to rules-as-written-only to clarify its intended use to new users just looking for a rules tag, and updating its tag wiki to clarify its use per the meta question What, exactly, is the RAW tag for?

This is an admirable thing to do, but it compromises your earlier stated mission. We need data on how people are using a tag called [rules-as-written], right? Well, study that tag then, not a new one you're experimenting with.

Renaming the tag here is a bit like saying "we want to see how apes respond to bananas. But, also, first we're going to replace all the bananas we have with apples. Then we'll measure their response to apples. That will give us data on how they respond to bananas." Which is, y'know, a little nuts.

I get what you want to do is clarify things and that's a good idea, but it's going to equal tainting the data right now. Here's the process you should follow, which I think you're jumping ahead on:

  1. Study the behaviour in the rules-as-written tag as-is for a while.
  2. Review data, see if we should take any actions.
  3. Assuming we decide it is a good idea: rename the tag to rules-as-written-only as a trial. Study that tag for a while.
  4. Review data, see if it improved things, and if there's more action needed.

Your study's goal is to help us reach step 2, but it really gets us to step 4 but without even any data from step 2 to compare against.

You want to reduce misunderstandings, but how often that happens in the first place is actually valuable data we need for step 2. We also won't understand if a rename to [rules-as-written-only] has any effect without that data.

Be like David Attenborough. If you want to observe the thing in its natural habitat with its normal behaviours and document it, leave it in peace as much as you can.

\$\endgroup\$
14
\$\begingroup\$

Extend the timeframe beyond 1 month

I counted up the questions per month in the tag since it can be counted fairly quickly. (This doesn't cover tag removals and deletions but since I have no info on that, I'm ignoring them.) There were:

  • 30 questions for January 2016
  • 21 in December 2015
  • 25 in November
  • 31 in October
  • 46 in September
  • 39 in August
  • 34 in July
  • 33 in June
  • 33 in May
  • 31 in April
  • 25 in March

History repeats itself and that looks like we usually get 30-ish RAW questions a month.

30-ish is a poor sample size to operate on. Uncommon events can be misrepresented by not showing up at all or being over-represented at that scale.

I suggest we gather data for at least two months. At the end of two months, we consider if we should run it for a third. As an absolute limit, once we reach 100 questions, the month we're on becomes the last — we'll keep gathering questions for that month, but we don't renew for another. That'll probably mean 3-4 months tops, and a quarter or a third of a year is a pretty decent sample. That's a while but good data collection doesn't come cheap and easy and if we're going to do it at all we might as well do a good job of it.

\$\endgroup\$
  • \$\begingroup\$ having gone all the way back to before the "rules" tag was put away, I see this time frame as perhaps being a better fit for a decent look. I offer to begin with March Last year and go post by post to gather some data on the last year's use of the tag in detail. That gives us a sample size in the range of 300ish. \$\endgroup\$ – KorvinStarmast Feb 8 '16 at 17:13
  • 1
    \$\begingroup\$ @Korvin The historical records are of limited use. The study relies on interacting with new questions as they arrive. If we were to start in the beginning of April, we'll begin studying each RAW question that comes to the site until the end of May at least, under this suggestion. (Or just to the end of April in the original one month span.) \$\endgroup\$ – doppelgreener Feb 8 '16 at 23:51
  • 1
    \$\begingroup\$ Also I don't think any analysis of past questions would be accepted by the more passionate principals here. \$\endgroup\$ – mxyzplk Feb 9 '16 at 1:06
  • \$\begingroup\$ An analysis of past questions would be personally interesting (and I've begun doing one), but yes, only one circumstance has been accepted by objectors to be meaningful (an asker switching from the [rules-as-written] tag to the [house-rules] tag after comment discussion), and unfortunately that circumstances has also been pre-emptively rejected as acceptable for determining a trend that would justify action in their eyes. Given that pre-emptive rejection, we've abandoned the idea of gathering data on past questions as a practical way to engage with objectors. \$\endgroup\$ – SevenSidedDie Feb 9 '16 at 23:29
  • \$\begingroup\$ @mxyzplk The past at this point (based on doppelgreener's data plan) is limited to a year ago and more recent, well after "rules" tag went away. I am not sure the passionate folks will object. I'll go ahead and dig into it and I guess we'll find out. \$\endgroup\$ – KorvinStarmast Feb 22 '16 at 22:06
  • \$\begingroup\$ @Korvin There's earlier questions than these. I just recorded back a full year because it makes for a more solid projection, and because irregularities can be prone to repeat themselves at the same time of year if the time of year is the cause - was worth examining if there were any. \$\endgroup\$ – doppelgreener Feb 22 '16 at 22:38
  • \$\begingroup\$ @doppelgreener I don't think we need to go further back than a year, since the last round on this topic was around Dec 2014. I think your package of about 300 questions in that one year time frame is a good enough number, and won't overlap a previous kerfluffle so we get a decent "how have we done since last time?" look. \$\endgroup\$ – KorvinStarmast Feb 22 '16 at 22:56
10
\$\begingroup\$

With respect to the fact there's some ongoing matters to discuss connected to this (Let's do that thing where we use our words), I'd like to suggest an alternate proposal for moving forward that takes stock of the comments and answers thus far (and the hefty amount of upvotes distributed).

This is intended to replace the original five "We’d like to propose that we do the following" bullet points. Most of these changes are on the basis that if we want to gather data on business as usual, we should not be altering business as usual at the same time. The rest are just expanding on how data gathering will happen.

I propose we do the following:

Moderation (by the community & diamond mods)

  • On any question where it’s not explicit in the body that the intent of the OP is for RAW, we ask if they’re sure RAW is what they want with a canned comment that makes it clear the tag’s intended purpose (primarily, delve into RAW even when it gets complicated, propose house rules and common sense only when that's been done to satisfaction).
  • The canned comment is discussed by the community first because we all know this is very likely to start fights if that doesn't happen first.
  • Diamond moderators moderate as they usually would.

The RAW tag gets removed in the following cases, because they indisputably show misuse of the tag:

  • The OP indicates (in response to that canned comment) that strict RAW is not their intent.
  • The OP uses the [rules-as-written] tag, then accepts an answer that does not cover RAW. (Or does so utterly and indisputably inadequately.)

Dispute over these actions gets discussed in the post's comments as normal.

Data gathering

(This mostly was just compressed into the last bullet point of the original proposal. Public logging is the plan, which is great, and it should get more detail in the proposal itself.)

Data will be publicly logged in a public meta Q&A by all participating users including diamond moderators, with links to the questions or answers involved. The following categories of events in particular will be highlighted because they're of particular interest and relevance to the debate:

  • A RAW question gets asked at all. Some of these will have the tag removed, so we'll log them here anyway for later reference. (Because people would no doubt wonder how many we've covered so far, I suggest we keep count by putting them in a numbered list, and number all items with "1." in markdown - which is just a neat way to auto-number them.)
  • The RAW tag gets removed for the above reasons.
  • The accepted answer covers the RAW in nebulous detail.
  • The accepted answer covers the RAW, as well as house rules or homebrew.

(Naturally, posts can belong to more than one of these groups.)

Other categories can come up, preferably as/before we begin in order to ease maintenance. Each category should be handled in a separate answer to minimise edit collisions (tightly related categories can stay in one answer, use your judgement). New categories can be brought up in new answers by whatever groups or individuals care to curate it, so that whatever details are of interest to various parties for debate & discussion purposes can be covered. (Some important categories may only become evident once we've already begun.) Duplicate categories should be avoided. Useless categories should earn downvotes.

Records that go obsolete (the accepted answer gets changed, for instance) should be kept on the list as a thing that did happen and marked obsolete. (If in later doubt and we have such records, we might need diamond mods to tell us whether it's true.)

Data will be considered indicative, not comprehensive. This is a sample, not a census, and has all the strengths and weaknesses of a sample. Statistical anomalies will creep in, but the data will at least give us a general idea of what occurs or can occur in the tag.

Gone completely:

  • Renaming the tag. Don't change anything about it. (Except maybe updating the tag wiki. Since tag wikis are notorious for rarely if ever getting read, and the existing one is already accurate, if a little lacking in straightforwardness, I expect it won't change behavior much.)
  • Any notion of "aggressive" moderation. It's completely unnecessary. Moderate per usual.
  • The notion that "RAW" means "no answers should ever mention house rules or common sense interpretations".
\$\endgroup\$
  • 1
    \$\begingroup\$ Diamond mods, is this paragraph from data gathering acceptable and feasible-sounding to you? "Records that go obsolete (the accepted answer gets changed, for instance) should be kept on the list as a thing that did happen and marked obsolete. (If in later doubt and we have such records, we might need diamond mods to tell us whether it's true.)" \$\endgroup\$ – doppelgreener Feb 9 '16 at 0:34
  • \$\begingroup\$ Yeah, if we were going to move forward with data collection, this is ideally how I'd like to see it done. \$\endgroup\$ – SevenSidedDie Feb 25 '16 at 6:06
5
\$\begingroup\$

March 2015 RAW question Analysis

I have examined all RAW tagged questions for March 2015. My detailed notes are in a text file that can be found at this link. The games covered were mostly D&D/Pathfinder sorts of various editions, one from Spycraft, and an OSRIC which is AD&Dish.

Caveat: as I have no idea how many comments were deleted, I have no sense of how much back and forth happened on all questions. Quite a few still have significant comments remaining.

I did not do a detailed analysis of edit history on each question, but I am not sure I need to.

  1. Summary. 25 total questions (two "duplicates" so 23 total questions assessed).

  2. Results: (A second review was done to ensure that text supporting need for the tag was found in the question to support valid use of the tag.)

    Good use of RAW tag in the question: 15
    Poor question and / or use of RAW tag not needed in the question: 8
    Good RAW answer: 26
    Poor or non-RAW answer: 9
    Deleted answer: 3

  3. A few other notes:

    Four questions did what this one did and explicitly stated in text a very hard RAW requirement.

    Some answers, this is an example, had text refer RAW tag and made a solid RAW type answer.

    In a five of the answers, the responses agreed that the rules are muddy at best -- at this point the better RAW style answers expanded upon that and the less good answers didn't. In one muddy rules case, I'll pull a comment from this question as an example of what was referred to in that other meta as where a RAW question may needs a frame challenge to get a decent answer. Where the are hard to parse, the rules being messy should not negatively influence use of the tag. (See also if there are so many books one knows not of a rule ...) The other answer got more votes, though the gap in the rules didn't help.

    +1: While not exactly fulfilling the RAW tag, this is still a great answer as it provides a practical way of playing this scenario. – > GreySage Jul 16 '15 at 16:21

    This one could have done without the tag. Neither the question nor the answers were of high standard.

    On this one, the usual trouble folks have with how alignment is handled in 5e, as opposed to previously. I think that is a root cause for this poor question -- RAW or not.

    This one was a bit of an anomaly. The accepted answer had one vote. The comment from the querent was that the answer was accepted as the best at being RAW fosuced -- all of the answers were decent, one had 11 up votes ... three other answers total available with more votes than accepted answer.

\$\endgroup\$
  • \$\begingroup\$ For the text file, you might consider Github Gist or a pastebin. \$\endgroup\$ – SevenSidedDie Feb 22 '16 at 21:48
  • \$\begingroup\$ @KorvinStarmast You can post to PasteBin without an account. You just can't edit it later. \$\endgroup\$ – AceCalhoon Feb 23 '16 at 2:39
  • \$\begingroup\$ @KorvinStarmast Yeah, paste the text. \$\endgroup\$ – AceCalhoon Feb 23 '16 at 3:07
  • \$\begingroup\$ @Korvin On what, where? You might want to link to it. That gives me 0 results in either pastebin or gist. \$\endgroup\$ – doppelgreener Feb 23 '16 at 3:55
  • \$\begingroup\$ @KorvinStarmast I've added that link to the question. \$\endgroup\$ – wax eagle Feb 23 '16 at 21:35
  • \$\begingroup\$ Anyone mind if these comments are cleaned up? There're not the usual discussion-type comments that meta gets that we like to preserve, so I think they're properly obsolete, but leaving them is OK too. Meta comment threads get hairy enough that I just had the thought that it would be tidier without them, if they're really no longer needed, is all. \$\endgroup\$ – SevenSidedDie Feb 23 '16 at 21:47
  • 1
    \$\begingroup\$ Thank you for doing this analysis! It gibes with my own informal observations that there's a solid core of real RAW questions with a fuzzy halo of ambiguous or badly-tagged non-RAW questions around the edges. \$\endgroup\$ – SevenSidedDie Feb 23 '16 at 21:49
  • \$\begingroup\$ @SevenSidedDie No skin off of my nose, the help with how to get the info posted easily was appreciated. \$\endgroup\$ – KorvinStarmast Feb 23 '16 at 21:51
  • 1
    \$\begingroup\$ Agreed, thanks. I agree that there's a good core of RAW questions but the tag getting misapplied 1/3 of the time also tells me there's a somewhat compelling need for clarification thereof. \$\endgroup\$ – mxyzplk Feb 24 '16 at 2:32
  • 1
    \$\begingroup\$ How on earth do you determine the 8 questions that misuse the tag have misused the tag? No, I don’t trust your speculation that the author did not want the question answered by RAW, I want to see evidence that suggests this. That means author involvement: statements that contradict the use of the tag, or at least an accepted answer that is not RAW. Merely “it turned out that RAW or not wasn’t terribly relevant to the answer since it was straightforward” is not a misuse of the RAW tag. \$\endgroup\$ – KRyan Feb 24 '16 at 5:23
  • \$\begingroup\$ @KRyan I read the text. Based on that month, it is being correctly used more often than not. More to follow. I'll be posting my results. If you'd like to take a different approach to the analysis, please analyse the posts and share it with us. Also, some of the questions were bot bad questions AND the tag didn't apply. I cited one of them in my notes. Those might be consigned to a different category in the final analysis. Maybe they should be. Your feed back is very much appreciated, the tone of your first two sentences is not. I'm trying to help here. \$\endgroup\$ – KorvinStarmast Feb 25 '16 at 18:46
  • \$\begingroup\$ @mxyzplk With 2/3 being used fine, and 1/3 not, I don't see that we need to do anything now as that is the first of 12. My instinct is that "so far so good" and it is not being badly used, unlike what you found with the game rec problem. (IIRC, only about 1/4 of those were going as planned. Nothing compelling at present, and the analysis is incomplete. \$\endgroup\$ – KorvinStarmast Feb 25 '16 at 18:48
  • 2
    \$\begingroup\$ @KRyan I am really, really not interested in a resumption of some of the ill will that has recently come across in the comment strings in the recent questions that are linked to this tag. I do not, as of this time, have any data to support a firm conclusion that the tag is out of whack. With all of the churn that has occurred, I have chosen to try and get some information and data that will give us all better information to work with. If you disagree with the methodology, fine, but please, let's keep this neutral and assume good faith. I find the tag useful, personally. \$\endgroup\$ – KorvinStarmast Feb 25 '16 at 18:54
  • \$\begingroup\$ @KRyan For more on where I am coming from, please see this answer. \$\endgroup\$ – KorvinStarmast Feb 25 '16 at 19:59
  • \$\begingroup\$ @KorvinStarmast I did not assume bad faith, but I did suspect poor methodology, which was compounded by a lack of description of that methodology in the answer. Most significantly, I think it is a very rare question where you can look at it, without interacting with the author, and say “rules-as-written is misused here.” Did those questions explicitly request houserules or homebrew? That might be an indication. But I don’t believe that was the case. “You could ask this question without the rules-as-written tag” is not sufficient to make the claim that you are with that label. \$\endgroup\$ – KRyan Feb 25 '16 at 21:06
-6
\$\begingroup\$

Yes

We should do this thing. All three diamond mods support it. It seems like a concrete step to gathering more data to make a decision in a way that is minimally disruptive to the site.

\$\endgroup\$
  • 3
    \$\begingroup\$ The down votes confuse me, since gathering data would be of benefit in determining whether any change/decision even needs to be made. scratches head \$\endgroup\$ – KorvinStarmast Feb 5 '16 at 16:00
  • 3
    \$\begingroup\$ The down votes are strictly because he has a diamond on his name. \$\endgroup\$ – Tritium21 Feb 5 '16 at 22:34
  • 8
    \$\begingroup\$ I don't think a whole nine of us (that's how many are downvoting) are that petty we'd downvote this just because a diamond mod posted it, @Tritium21. Pretty sure it's signalling "don't do this thing." The proposal in its current shape has some problems with it, so a "don't do that" signal is unsurprising. If it changes and there aren't any more issues, a reversal to an "ok, do that" signal would also not be surprising. \$\endgroup\$ – doppelgreener Feb 6 '16 at 0:42
  • \$\begingroup\$ @doppelgreener I am pretty sure you are wrong, and it is just that the mods are untrustworthy or evil or something. Because no problems can stay in one discussion, and people have to go run off to meta.se. \$\endgroup\$ – Tritium21 Feb 6 '16 at 3:51
  • 5
    \$\begingroup\$ @Tritium21 Uh huh. So, meta.se is the place for discussions concerning the site, so I'm not sure how that's a problem. I suggest you don't continue down this line of oblique condescension, mostly, unless you're trying to start a fight with someone. \$\endgroup\$ – doppelgreener Feb 6 '16 at 4:12
  • 1
    \$\begingroup\$ @Tritium21 Given that the diamond mods are, AFAIK, unpaid volunteers, I understand your position (and your now deleted answer elsewhere is taken by me as a reaction to the issue being personalized with no value to the site). I suspect that some votes on the main site and here are based on "who" writes and answer, but that most (the vast majority) are not. I share your general position that the mods are doing what the mods ought to be doing, but would like to see if the data supports the suggestions on how to, or if, tag modification would be best addressed. \$\endgroup\$ – KorvinStarmast Feb 7 '16 at 13:56
  • 7
    \$\begingroup\$ While it's true when I notice mxyzplk's or SSD's name on a post I expect to downvote, because we used to bump heads rather often, I certainly don't automatically downvote. In this Q alone, I find I've upvoted SSD's a few times. If logical improvements to the plan hadn't been made that I like better, I would be upvoting here despite mxyzplk and I having philosophical differences. (As a note, I find myself agreeing more with SSD now that he has the diamond; he's more neutral and less aggressive than I remember him being, so I don't blame the diamond.) \$\endgroup\$ – gatherer818 Feb 9 '16 at 8:03

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .