To what extent should Back It Up! be enforced?

Question

Currently, the Back It Up! is a bit vague to me. The policy is:

Your answers (to subjective questions) must be based on either:

• Something that happened to you personally
• Something you can back up with a reference

It isn't clear to me to what extent something has to have happened to you personally, nor what kind of reference needs to be used. Sometimes it seems to me this policy is weaponized, used to disparage answers, or used as a form of gatekeeping.

Here is an example: What should I do about a religious player who refuses to accept the existence of multiple gods in D&D?

This question is highly subjective, but many people have had some kind of experience that may be relevant. This experience ranges from "perfectly relevant - I am the OP", "highly relevant - I have been in this exact situation", "relevant - the situation and context were different, but I have some insight".

The top answer says "I haven't encountered this explicit situation where a religious player is unwilling to adapt, but I've experienced plenty of other scenarios where people were resistant to the unknown, the unfamiliar, or something they believed was "wrong" for various reasons." and that's good enough for me. But what would the minimum look like? Having interacted with another human? Talking to someone about their dislikes?

I find the vagueness makes the policy easy to weaponize, especially against newer users who don't know better. Here is a comment on a now-deleted (for unrelated and extremely valid reasons, so I won't link it), please note that the original answer does not mention a balking religious player:

What was the outcome when you did this with your balking religious player? Did they adapt, or did they walk? Was it a smooth transition or was there some struggle? How did(n’t) it impact the group’s enjoyment of the first few sessions?

Clearly this is a far higher bar than we realistically need. The comment implies that this is needed for an answer to be valid. Ultimately the answer was deleted for unrelated reasons.

Here is a much better answer. They go over some things that most humans have experience with: having an honest talk and compromise. However, they are told "You've got some good ideas here, but they really do need to be backed up per our good subjective policy. While your ideas are good, we should not be upvoting answers that aren't backed up unless you can support them." To me this answer has the same level of experience as the top answer.

On the flip side, especially with 5e questions, we have a problem of over-quoting. Since everything can easily be referenced, often simple questions are answered with a half dozen rules quotes. I don't think it's as harmful as this problem, so that's a discussion for another time.

I am finding it difficult to understand where the line lies with regard to what experience is considered relevant. How can the vagueness of the policy be addressed in a way that does not risk problems with new users?

Answer 1

I Don't Think You'll Get Crystal Clear Arbitration Rules

We are, after all, talking about tabletop RPGs, where questions span the gamut from, "What happens by the rules in this situation?" to, "What's the best mechanical way to achieve X?" to, "I have this nasty conflict with my GM/a player, so why am I right and they're wrong?"

Some of these are amenable to easily interpreted answer-- rules are cited, answers are compared, and voting happens. Others aren't because they are "Good Subjective" questions-- questions that are in some obvious way subjective, but which nevertheless other players and GMs may have some experience with.

But note, when trying to sift out Good Subjective answers to Good Subjective Questions, the guidelines are going to be... subjective, based in community practice, guided by higher rep users and occasionally brought to a head in Meta. But the Meta does not-- can not-- remove the subjectivity entirely, it usually just codifies it and gets people on the same page.

What's The Point?

...Of the Subjective Guidelines, I mean. I didn't fully come around to the GS/BS mindset myself until I came to a conclusion (perhaps personal, perhaps more widely shared) about why the guidelines exist.

To quote and then expand on an answer I particularly liked in a similar discussion:

The purpose of GS/BS is to enable meaningful voting behaviors. That’s it: to make sure that the Stack Exchange system actually works, actually filters out noise and highlights strong signals.

On the question end, this filters out questions so subjective that there is no meaningful criteria for casting votes on the answers. On the answer end, especially for the querant who may not have expertise with the precise subject matter, the backing of experience will help them in the selection of their best answer. Collectively, they prevent the stack sites from morphing into free-form discussion and brainstorming forums.

How Much Experience Is Enough?

It is, as I alluded to before, subjective. As far as I know, the most recent guidance on Subjective Answers is here, with the relevant passage being:

This means when you provide a subjective solution you believe will resolve the situation, we expect that you cite analogous experience of how it has worked out in actual practice — your own experience or someone else's. Describing how it worked out is more preferable to describing how well it worked out: “these things happened, the players felt this way about it” is preferable to “it worked well”, but either is preferable to no citation of it actually being tried at all.

The experience does not have to be of the exact same situation, although that would be ideal. Experience of similar or analogous situations is also relevant and helpful, at least to the extent they are similar/analogous enough to be relevant.

(Emphasis mine.)

Not much more detail than that is given, so it is up to the community to determine that in practice: Answerers decide when writing, querants decide when selecting an answer, and readers decide by casting votes (explained or not.)

In more extreme cases (ideally but not always a note on a question that seems a little close to the edge and may start generating brainstorming) someone might leave a comment. The usual endpoint of this process is that the question generating too many Bad Subjective answers gets closed either by vote or by mod.

For more egregious cases, I suppose there are various options to delete an answer, but those are not the norm, and not really what we're talking about here, I hope.

Does It Work?

I think so. This is anecdotal at best, of course, but once I understood the point of the guidelines, I found myself more than once (and still do!) thinking that I have an answer that I think is really good and want to share... but in the process of writing it up, I realize that the experience I have probably isn't close enough to warrant the advice I'm giving.

And so I close the answer and think about it overnight, and either reconstruct a better version, or stay silent.

A Footnote

As I mentioned earlier, these issues do occasionally come to a head here in Meta; this is one such example. But it's worth pointing out that several months after the Question and Answers you're looking at, a diamond mod noticed rising friction on this topic in the community and took up the task of soliciting feedback on the issue.

Since then, I think the stack has reached about the right balance. There is much less friction on this issue that I can see.

This was, in fact, the motivation for my setting an arbitrary limit of three years in my comment-- sadly, I thought that realignment was farther into the past than it actually was.

It was also the basis for my observation that the guidelines had been used over-zealously in the past. But I don't think I ever saw a case of turning them into a weapon against a disliked user, or as an intentional gateway mechanism to keep any person or people out.

Answer 2

The Code of Conduct is first and foremost.

That should go without saying; but sometimes users don't want to bother or put the time and effort into understanding the whole picture. As you mentioned, this is likely the primary reason why the answer you're referring to was deleted. Basically, their approach was "Their values don't matter", taking the "my way or the highway" solution, which just is not acceptable here.

"Back it up!" is dependent on the situation.

As it may have been made clear through context; "personal experience" or a link depends on the type of question that's being asked. In this situation, "personal experience" is the more appropriate, though obviously if it did not happen to you, a blog or podcast, etc might have the evidence to support your question; so a link would work well. Generally in rule clarifications etc, a link is the more appropriate.

That said, "Backing it Up!" is what makes the difference between a good answer, and a better answer.

Again, you've mentioned that some people do give good advice, but since they're just "ideas", they haven't been tried and tested. That doesn't mean that they're bad answers though - it just means that while these ideas are good, the answer that has evidence of its efficacy is better.

To quote Barbossa, the "Back it Up!" policy is more of a guideline, than an actual rule

Much like many other "rules" we have here. We aim for a 1:1 Question to answer ratio (purely to reduce noise, duplication, etc). That doesn't necessarily mean that more than one answer on a question is unacceptable. Votes don't determine the wright or wrong answer, or necessarily a good or bad question. A lot of the time these "rules", like the questions they apply to, are subjective.

The main thing you want to follow is the Code of Conduct - Be Nice