On meta, a new information-gathering project about weighted voting for trusted users started that I think might affect us considerably later down the line: Should trusted close/reopen voters and reviewers have their votes weighted more than others'?
For your convenience, I quote the entire discussion prompt and invite you to discuss it here in particular regarding rpg.se.
[the following was posted by Catija, a Community Manager employed by Stack Exchange]
In Q3 2021 I finalized my research around three vote close/reopen and I was pretty happy with the outcomes - as were the sites that got to test it out. While the benefit to each site was different, the most important thing was that there were no negative impacts of that test and all of the sites wanted to keep the reduced numbers of votes to close. We will be continuing to make this change on other sites that have requested it in the near future, so if you're waiting for a 3-vote close/reopen test, it'll be coming soon.
One of the things that came up during that process was a question about expanding the gold badge close/reopen privileges to other users as an addition or alternative to three vote close/reopen. We've discussed it internally and want to do some research before deciding whether to ask the Public Platform team to build a new feature to allow this - unlike 3-vote close, there's nothing already built that will allow a change of this sort.
This question is the second part of that research - I asked a version of this question to the Moderators in November 2021 and am following up here to see what others think, too. Based on their initial feedback, they were concerned about a few things and it didn't seem there was a huge amount of interest in this path forward, at least as proposed below. I've adjusted the description below to answer questions the moderators had and concerns they expressed.
What is the goal?
First things first - this project is purely information gathering as mentioned in the Q4 roadmap. We're trying to see whether it's something that would be worth building at some point in the future and we have no specific timeline related to actually building it. We'll be gathering data and community sentiment before deciding whether to move forward. Additionally, this wouldn't be shipped network-wide if we did build it. Like 3-vote close/reopen, it'd be up to the site to opt-in. That out of the way, let's look at the goal of the feature...
As my research showed, there's a need across the network to make curation tasks easier to complete and require fewer people to participate. Three vote close/reopen is a way to do that and it's also a very even-handed one - essentially, everyone with the privilege to vote to close/reopen gets the same decision weight when it comes to participating in reviewing and voting to close and reopen.
In general, this is fine but it means that we're not necessarily recognizing the specialized knowledge and experience of people who are very involved in the curation process, and I would say that recognition is deserved!
So, in the end, the goal is: Make it easier for engaged curators to close and reopen questions more quickly, for any reason. Additionally, we hope that people who are working to curate the site will feel like their efforts are more valued and that there are fewer close/reopen review tasks needing their attention.
While we can't really hope to make reviewing fun, we can hope to make it feel less futile.
Why are we investigating weighting close votes as a solution?
Up to this point, weighting votes has been limited to the gold badge "duplicate hammer" - which, as its name implies, only has any power over duplicate closures.
There are other things about the gold badge hammer that may not be a good fit for this usage:
- It's only particularly useful on sites that are huge (like SO) or have some very deep interest and expertise in specific tags (like SFF) since getting a gold tag badge is a very high bar to set.
- Having a gold tag badge doesn't necessarily mean you know when to close/reopen posts - that said, having this only relate to duplicates minimizes that risk.
- It only relates to duplicates.
- It's unilateral. This is a pro and a con - for duplicates it's likely good but for other close reasons, it may not be as sensible.
That said, when we initially rolled out this feature, half of the answer announcing it says that we'd consider it for other uses in the future - and that was in 2014! Seems like it's about time.
Beyond this, we think this is a good potential solution for the goal we're trying to achieve. While it's not something we can start implementing tomorrow, since we'd have to get it built into the site, we do have some data and personal experience that we can look at from duplicate closures.
What are the initial feature ideas for this?
I've got some ideas floating around in my head of what I think might make a good feature set to ask for here and that's based on my personal experience and my thoughts about the duplicate hammer. I'm open to being convinced that I'm wrong about any of these points! I'll put them here in a numbered list with my thoughts so that y'all can call them out by number in your answers:
- Increase the close/reopen vote weight to 2 for users who meet certain criteria - I think that making these unilateral has the risk of making it a bit too easy to close or reopen questions without a lot of oversight.
- I could also go the route of making it a weight of 2 on sites with 3-vote close/reopen and 3 on sites with 5-vote close/reopen.
- Unlocked by voting to close/reopen (including reviews) n times + - Since reopening is less common, this would get imbalanced if we relied on each action individually. We don't necessarily want to rely on reviewer badges only as they're not very common. Review stats:
- Steward badges (1k reviews) - only 88 sites have at least one person with the badge and only 42 have ≥10 †.
- Reviewer badges (250 reviews) - 132 sites with at least 1 and 81 with ≥10 †.
- Requires maintaining n% of close/reopen votes & reviews deemed "valid" - Whether someone's reviews were actually "good" and whether someone should be able to robo-review and still get this privilege is a concern to me. I don't think they should.
- "Good" defined by being validated by other community members (e.g. was the post closed/reopened?) We'd put some work into finding a reasonable way to define "good".
- Alternative option: Only weight votes cast in review - We could make it so the privilege is only awarded for reviewing and would only weight votes cast in close/reopen reviews.
- Applies to all close reasons - There's really no reason to restrict this to any specific close reason over the other. Except for duplicates, we don't necessarily want to incentivize people opting for one reason over the others.
- This would not replace the gold badge hammer - so, no loss of privilege for those users when it comes to duplicate closures.
Those are the broad-stroke points. There are clearly minutiae, such as determining which close reason "wins" to poke at but those things seem like secondary bits that we can work out once we answer the big question - "Is this idea worth pursuing?"
† - These badge counts are queue agnostic, so it's likely an overcount if someone has a Steward or Reviewer badge in LQP instead of Close or Reopen.
What do I need from y'all?
Poke at my ideas, question them. Point out things that I've missed or where I'm looking at things wrong.
- Is this concept (even ignoring the specific rules I've outlined above) good/bad/silly?
- Would this be something you think the sites you use would gain value from?
- Is there something different that you think will address the goals we've mentioned?
Don't feel like you have to answer any or all of these questions, they're here as ideas for what sort of feedback I'm looking for.