How to revisit citation expectations for subjective questions

Question

Based on questions like Medix2's, Medix2's other question, and my earlier one, there is a big disconnect between the historical guidance for supporting subjective answers and what the community likes/expects/wants.

My original question about how to back up was specifically for addressing this. I felt that there was a clear expectation on how to back up subjective answers based on earlier questions but noticed community pushback on it. When Mxy answered with a largely reproduced answer (but also supplemented with additional very helpful guidance) and then it was closed as a duplicate of the earlier metas on how to back up subjective, it seemed clear that community did agree this had been asked and answered previously.

But with the recent slew of questions and responses about backing up subjective, it seems like it is not clear at all.

I'm not sure which questions need to be reopened (if any), or really how to approach dealing with it. But it does seems like there is vocal percentage of the community who believes that the guidance to date is incomplete with regard to assessing subjective support.

It also seems like there hasn't been a clear direction given in how to address assessing support in that case. I don't think "it seems like they know what they're talking about" is sufficient. But I do want to understand other ways of assessing if they are available.

How do we clarify this? How should we be approaching this? What questions should be new vs needing new answers to old questions?

Answer 1

Can we stop already?

We literally do not need to discuss this. We do not need examples, we certainly don’t need an exhaustive list, we don’t even need rules. The GS/BS guidelines are exactly that, guidelines, and they work well. We don’t have any large problem with answers that are insufficiently backed up. If we have any problem, it’s with people beating the citation question to death.

So there’s nothing to discuss: things are fine. If you personally sincerely feel that an answer might be doubted and does not give readers enough evidence by which to judge their veracity, go ahead and say so. We’d all appreciate it if you didn’t get nit-picky about it, but as the GS/BS guidelines say, if someone says an answer could use more backing up, it probably could. If it’s really just going overboard, the comment can be flagged as no longer necessary—an irritation, but a minor one. Dragging things over to meta, making a big discussion about it, fretting over whether or not something technically counts as evidence, that’s a more serious irritation, but also easily ignored (cf. the fact that I have avoided participating in these discussions as much as I possibly could).

So seriously, please, can this be the end of it? If your question is “should we have policy XYZ with respect to backing things up, and write it up all pretty and add it to the FAQ and incessantly point users to it?” the answer is no, the answer should always be no, and we should be able to just skip the discussion altogether because we should all agree on that.

Answer 2

Back to Stack Basics

1. Guidelines have already been provided. We have a thorough explanation of those already, but the original SO/SE guidelines have been good enough since this site was initiated.

2. Encourage authors to "back it up" in a comment where there is a lack of support; include links to these two guidelines that already covered the "how to" for this SE.

   This is never a bad idea. Some authors will upgrade their answers, some will not.

3. Let the votes work. If the author won't back it up, don't pester the author, vote as you deem appropriate. Better answers will tend to get more votes.

4. Let the votes work. On a Stack Exchange, the votes are supposed to stack answers from best to least-best by number of votes. This latest crusade versus GS/BS looks to me like an attempt to indirectly micromanage votes before the fact. The voters who are moved to vote will vote, which is the stack working as intended.

5. If now and again a not great answer moves up the stack, (1) don't worry about it (2) flag it. Other stacks run into this now and again. (There was a famous example on SO that I read a few years back where a wrong answer had something like 200 votes. It happens, but it's rare). Let the votes do the stacking.

6. Use flags. They are a tool. For an answer that is just plain bad or utterly unsupported, flag it for mod attention so that it gets into the review queue as "very low quality." We have plenty of folks who work the review queues; our community mod efforts are generally effective. As an added bonus, flags get moderator eyes on potentially problematic posts. For those rare exceptions, there are our four mods who are supposed to be exception handlers.

7. There isn't an easy button, but there is a suite of tools. Let's use all of the tools. We don't need additional policy, we already have guidelines. (SO guidelines summarized at the end)

Don't Chase Exceptions

Chasing exceptions is what Deming called "tampering" and is a great way to make bad policy and bad "corrections" to a given process. I learned this the hard way in a number of real life organizational change efforts. I have been on the receiving end of this, and on the policy writing end of this. It stinks no matter what end of the stick you are holding.

Guidelines > policy in this case

This was added thanks to a brief discussion with Rubiksmoose in chat.

I have found that with "policy" people now and again use it as a bludgeon on other people (for one reason or another). This is across the board in groups large and small. Organizational Behavior 101. I made comments on the effects of bad policy here. (Humorously, we eventually figured out that it wasn't even policy but that some users perceived that it was). Guidelines are not hard and fast rules. In a community (SO) built from an industry that is rule bound - computer code is a set of commands that can only have one meaning - the default idea that "we need another rule" to fix that is a reflex that often gets in the way of productive discourse, may create a new problem, and may not even solve a problem.

This is a people site, not a code site. There's been a move over the past year and a half among the SE/SO Powers That Be to be more attuned to people. (Yeah, it's been a ride down the Colorado River rapids, not a serene Danube cruise).

Thanks for the dissertation, Korvin: what can I do?

Encourage fellow users to write good answers, and vote (or don't vote) based on the quality of the answer. When I first arrived here, encouraging posters like NeilSlater and HeyICanChan made me feel welcome, as did SevenSidedDie. Others did not. The difference? An encouraging tone.

---

Guidelines for Great Subjective Questions

1. Great subjective questions inspire answers that explain “why” and “how”.

2. Great subjective questions tend to have long, not short, answers.

3. Great subjective questions have a constructive, fair, and impartial tone.

4. Great subjective questions invite sharing experiences over opinions.

5. Great subjective questions insist that opinion be backed up with facts and references...And good subjective questions make this clear from the outset: back it up!

6. Great subjective questions are more than just mindless social fun.

Answer 3

Proposed changes to good subjective guidelines:

Explicitly state that what qualifies as "backed up" is subjective (up to human judgement)

I do believe that explicitly stating that "evidence" and what counts as "backed up" is subjective would help. This removes power from the sentence "You need to back up your answer!" because whether or not that is true would explicitly rely on human judgement.

Provide a list of examples of how one might support their answer

Regardless of how the expectations of a Good Subjective answer might be being implemented/enforced right now, I do believe examples of "possible ways one might back up an answer" would be helpful. It provides a more tangible list and a way to better see if an answer meets the guidelines.

I am an individual who has great difficulty in understanding how others might interpret my writing, and my worries about whether my subjective answers are "backed up" (whatever that may mean) have lead me to primarily avoid answering subjectively. I do not do especially well with the vague notion of "needing to be backed up".

A list of examples or possible ways to provide evidence/support/experience, would provide go-tos for showing "Good Subjective" so long as the examples aren't set apart as the only ways to have a good subjective answer.

The meta question on backing up subjective answers states that they should be backed up, but it's unclear to me how one can do/show this and something akin to a list of examples would help me immensely.

How would this impact already existing answers?

It absolutely shouldn't; I believe any new guidelines for what counts as "good subjective answers" shouldn't make previously supported answers no longer supported (this would be a lot of hassle/work and also seem rather unfair). Guidelines would show methods of providing support, but any answer that was previously supported should still be considered supported.

---

Small sidenote: My feelings on change in general

When concerns arise regarding current policy/rules/guidelines, I feel it would better to open a new question marking the old one as a duplicate (perhaps copying over answers as well). This allows the collection of votes on the change to be from active members, those who will (presumably) exist under/after the change. It also avoids oddities where one answer starts with a disproportionately large number of votes, especially from no longer active members, simply because the answer has existed longer.

Thus I believe that the already existing question on citation expectations should be closed as a duplicate of some new question and a new answer be provided there.

Note that this last part is more my own take on things and I was asked to include it here, my main point is the prior sections, and if the current guidelines were simply edited to include examples I would be perfectly alright with that as well.