Should we reprotect the 700 or so questions that were unprotected in the last few weeks?

Question

Someone recently and quietly unprotected about 700 mainsite questions. (Whether or not one should do that is under discussion here.)

Should we re-protect those questions in advance of a broader meta discussion around what should be protected, what guidelines might be put forth for what makes sense to unprotect, &c. Effectively, reset the clock?

(I'm aware that "we", actually, can't really do this--there's no way to find them and it'd be an onerous task. I'm pretty sure mods would have to ping staff to get them to run some queries and other computer-stuff, so I'm putting this out on meta to gather votes as evidence that staff should do so.)

Answer 1

Thought I'd add my PoV here - the mods kindly reached out to me about this when it first happened and I'd been talking with them about it. The ask on our end isn't as simple as Dopplegreener's answer makes it out to be. We'd have to build a dev route that doesn't currently exist to re-protect them, rather than undoing the protection. It's not a five-minute fix. It's doable but not a 5-minute change.

On Protection

When I was first a mod on IPS, there actually was a user who was just protecting things that had lots of answers - not spam, just... lots of answers, some of them low-effort or low quality. So that frames a lot of my thinking about protection. I, personally, find it to be overused and I find it being permanent is more likely detrimental to a community than beneficial. We, as users with reputation can make it off as an easy task to get past protection -

Just find a different question you can answer and get one upvote - that's it!

But this ignores that many people find a post from searching their favorite search engine. Maybe they found the question and have a better solution but they can't answer because it's protected and they don't know the site yet to know how to find a new question to answer. Heck, they may assume that all questions are protected. They just want to answer that one question, and they can't. So, maybe it's idealistic but I want to remove barriers to participation as much as possible because it leaves more opportunity for new people to join a community. Yes, even if only a tiny fraction of questions are protected.

This is exacerbated when sites show up in the HNQ often and the only reason the protection is in place was to reduce activity from a short-term event. Once the HNQ spotlight has passed on, the protection becomes unneeded. Since we didn't track which posts were HNQ until January 2019, we can't compare whether that was a likely cause of protection here and the ability to kick questions off the HNQ list along with the duration on the list since that time likely means it's a reduced issue.

On mass un-protection

Y'all had an event. And... well... it didn't go the way it should have. Big things like major tag reworks or reviewing protected questions is something that should be a community effort with buy-in from everyone who's interested. You should have a decision about how to act and then act on it. That's the way a community works together.

That didn't happen in this case and that's unfortunate - an opportunity for community communication and collaboration didn't happen, instead, the community was put on the defensive due to a single user taking action. I hope the person who unprotected all of these questions learns from this experience and chooses to involve the community before making big changes.

Who uses protection?

The Community user - which acts when a post receives a lot of low quality answers or many answers from low-rep users - has only protected 136 questions (SEDE Query). Now, that doesn't mean that only those questions would have been protected, eventually. Some of them may have been manually protected by a user before they got to where Community would have acted. But the bulk of the protections, close to half of them, were protected by a single user. It'd be interesting to know what they use to decide when a question should be protected.

There's a great analysis of the data in another question already, so go read that for more details. In short, the Community user accounts for a sizeable but not the largest number and many questions that receive spam answers aren't protected. Some of the questions that were protected are likely from garnering many answers while a question is in the HNQ list

What's next?

My recommendation is to have your discussion about what makes a question qualify for protection and then use the queries you already have or ones you create to determine which of these questions should be re-protected, if any. Also figure out if you want to regularly review protected questions to see if they should be unprotected - as a community.

I understand that this may be frustrating to y'all. Protection is an important tool and it is up to the community to decide when it should be used. So, have those discussions. If more than half of the questions that were unprotected don't need to be protected any more... then me getting a dev to protect them all again is actually more work unprotecting for y'all.

And, remember, coming to a community consensus on when a question should be protected or unprotected doesn't mean that the community needs to decide on each individual protection - you have the guides in place to inform people so that they can be trusted to make those decisions on their own.

1. Pick 15-25 or so of the unprotected questions as examples, have a discussion based on those to decide the signals to look for. Some examples you might consider:
   • deleted answers (how many?)
   • answers from low-rep users (how many?)
   • HNQ status
   • spam answers (how many?)
2. Define when it shouldn't be used, too (important but often overlooked!).
3. Let your privileged curators review the remaining questions.

By telling people what to look for, they have clear things to signal risk of future spam or low-quality answers rather than relying on subjective things like "this looks like it's going to get a lot of junk answers, so I'm going to protect it". If there's edge cases people come across, discuss them and adjust your guidelines. Be open to review from time to time... maybe even start a list of questions that should be protected but just temporarily, as a reminder to unprotect later.

Yes, this is work but it's good community work and y'all will come out of it with a better understanding of how protecting works and when it should and shouldn't be used. Having a dev re-protect the questions may actually make more work for you if your rules end up meaning only a small number of the 700 should actually be protected. If you do an analysis and think that most of the questions should be protected... then let me know and I can see what we can do. I recognize that only 15 k users can protect a question but it doesn't take a ton of added effort for someone who's reviewing a question to either protect it or drop a link into a chat room that y'all are using for this project.

Answer 2

Yes, we should absolutely reprotect those questions. We didn't protect them for kicks, and having all of them become unprotected is actively harmful for us. Doing so takes no effort on our part; Rubiksmoose already stated that “[Stack Exchange staff] have the ability to identify and reverse all the questions this user unprotected” so they're just waiting on our mods to say the word.

As BESW summed up in chat, questions tend to get protected for one of three reasons: (1) a question hits HNQ and attracts nonsense, (2) a question is controversial and needs a bit of brakes applied, and (3) a spam algorithm has identified that question as a target via keywords. And of those three, the last one is far and away the most frequent primary cause. We already have users who periodically unprotect questions that were protected that no longer needed it, so the protection list we had was after frequent case-by-case culling.

Later edit: Rubiksmoose has crunched the numbers, and apparently spam-based protections don't establish the majority of our protection list just prior to the change (it's about a third?). There probably are many questions that can yet be unprotected, but I believe our prior protected list should be reinstated, rather than let a sweeping removal that never should have happened stay in place, then we can assess what kinds of questions we want to clear out of it if any.

---

I see folks saying here that only a handful of questions will become a problem, and that's just factually plain wrong. For those unaware: we get spammed quite a bit. It's consistently about 3–4 times a week or a dozen times a month. Most of it comes from one source (spellcaster spammers) and our spam sources are very well understood: they search keywords, every now and then one of our questions (new or old) shows up matching those keywords, and they spam there if they can. They spam the same question repeatedly if able, coming back to it every few weeks for as long as it remains unprotected.

This means nearly every single question we have that has ever received spam (which is hundreds of questions) is guaranteed to get spam again, as long as there's a text box the spammers can post to.

Over the years, by protecting every question they post spam to as they come up, we've worked to minimise our surface area to spam. It's thanks to this defensive capital that our community's built up over the years that our site gets as little spam as it does. If you think spam isn't a big problem here, that's because of this work we've done.

This mass unprotection basically wipes that slate clean and resets the defensive capital we've built up to nil. It's like waking up one day and finding all the walls to the fortress have been disassembled overnight. Of course we should be putting them back up as soon as we can.

We're discussing this because it got opened up for debate, but would we blink if the moderators said “Yeah, wiping out that much stuff without checking with anyone isn't appropriate, so we reverted it”? I venture we probably would not. Would we have a debate if it was instead all the downvotes got wiped indiscriminately, or every single closed question got reopened regardless of why it was closed? Probably not, because we understand palpably the harm that represents. This represents harm too.

---

Background: I've spent multiple years actively involved in Charcoal, the organisation that is SE's last line of defense against spam. I was active in events like that spam wave that occurred in 2018, and was busy hammering down nearly six hundred spam accounts over the course of that wave. I've seen spam like you wouldn't believe, dear readers.

Answer 3

I recommend a complete reversal.

First of all, a disclaimer: I’m actually of the opinion that we had too many protected posts, and that we--as a site--should be more proactive about unprotecting things after a time.

That said, for me this issue is less about the end-state than it is about the process, and what we can learn from it. I believe we should be on the lookout for opportunities to build up the community: its coherence around principles, its sense of agency, its understanding of the state of the site. Blanket-unprotecting 700 questions and then query-driven reprotecting a hundred of them short-circuits all of that: no single site user develops any more understanding of what should and shouldn’t be protected; we feed back no wisdom to SE staff on whether autoprotection is currently working well or poorly; we don’t hash out any of the boundary areas and, in doing so, reveal guiding principles we can collectively embrace.

And, hoo-boy, do we need to. Addressing the elephant in the room: our theory of protection, generally, is vastly under-developed on meta. We currently have 4 meta posts devoted to question protection. We have more posts about site-promotion (psst: we don’t do any of that) and feeds to chat than we do about a feature that unlocks at fairly high rep and that locks the majority of users out of the site’s core activity: answering.

We can leave this batch largely unprotected, use a query to re-protect those that were protected by the community bot, and in doing so throw away literally every human’s judgment that went into that list. (Well, except the one human who decided to unprotect it all.) And we’d be right back at square 1: some posts will invariably get protected, few will ever go back to look at whether those posts should be unprotected, any who come to meta for guidance will find none but that they shouldn’t unprotect everything at once. We will have gotten a nice-and-small protected-list without learning from the curating of that list.

Or we can revert fully the recent unprotections. Ask the relevant meta questions: how do I know when to unprotect a post? These ten posts have no downvoted answers and I see no discussion of why they should be protected, can I unprotect them? The last downvoted/deleted answer on this handful of posts came in 4 years ago, shouldn’t they be unprotected? In some of those cases it’s going to clearly come down: “leave protected.” Others: “unprotect, and do it yesterday!” And some we’ll have to furrow our brows for and post competing answers and we’ll learn something about those posts and about our preferences.

We can do the actual work, as a community, that will guide future users. There’s no rush: we can take a year or so to give ourselves an education in what gets protected, whether we like how much or little that tool gets used, and how we can know whether a post should be unprotected. We don’t need to discuss every question. But in discussing some questions we’ll grow the site’s understanding of this tool.

Answer 4

Leave unprotected and gather data for 60 days, then decide.

This is not an endorsement of the actions taken by this user, they should have talked to the community before unprotecting virtually every question on this site. However, this is about doing the best thing for this community going forward now that we are here.

tl;dr We learn nothing by reprotecting all of the posts right now and we lose nothing by waiting to see if any of the posts have issues. Lets gather the data and be smart about it.

From everything I and other experienced users can tell, the vast majority of protected posts no longer need protecting like they did when they were first protected.¹

Reprotecting all of them would require a decent amount of dev-time work and could leave as many as 600+ answers protected without any need (thus preventing new users from answering them).

Instead, I suggest we don't reprotect any of the posts for now and see if any problematic posts even come up during a 60 day test period.

And during the test period we can discuss what we want our guidelines for protection to be. And by the end, we'll have data to help back us up. And when the test period is over, we can decide which of all of the questions we want to be protected and submit that to the CMs if necessary.

This way, we can actually learn something from this incident. It is impossible for us to learn which posts protection is actually benefiting when they are protected, since the very thing you'd need to see to judge success would be answers that are prevented from being submitted. And that is obviously impossible to measure. Instead, let's let the problem posts show themselves to us over time and we can use that experience to help lead our discussion about how we want protection on this site to work going forward. (Keeping in mind that there is only confusing and vague advice given by SE about this — confused even more by the recent change to the wording of the protected feature.)²

During the test period we can track and collect data on each of the unprotected questions. And then we can report back here and revisit the issue. I suspect that <10 of the posts will even get a single post and fewer that are problematic. If we have no posts during the time that are deemed to be in need of protection or harmed by the lack thereof, I'd propose we just make it permanent. And if there are issues, we can cancel the experiment at any time and reprotect the whole bunch.³

In the end, we lose nothing by waiting for actual issues to surface and we have potentially a decent amount to gain.

---

^{1 - According to KorvinStarmast (admittedly only one user, but an experienced one):}

^{OK, after 120 out of 720, there were 3 I'd recommend protecting so far}

^{If that ratio holds true to the way the community also sees protection we'd end up with a total of 18 posts to protect out of 720.}

^{2 - We also don't need to reprotect the questions in order to learn from the previously protected questions since we can refer to this list and review them as a community at any time.}

^{3 - If any problems do arise during the experiment, those posts will be dealt with according to the best practices we have. No experiment should result in the degredation of how the site operates. So if a spammy post starts getting hammered, for example, we'll protect it like we would normally.}

Answer 5

No, we shouldn't re-protect them

While the mass unprotection was a rash move, the fact is that almost all protects are for time limited reasons - hot network questions, squabbles, and so on. If even 5 of these 700 questions pop a problem and need to be re-protected I’d be shocked. Just leave it alone and re-protect if and as needed.