Very soon we intend to start a discussion about how protection is used on this site going forward. To inform that discussion, I've tried to dig deep and find some data that could help guide this discussion. It got so lengthy that I decided that posting it as its own post was probably the best way to go to get the most eyes on it.
Note that I am not a SEDE wizard. I have tried to verify the results for all the SEDE queries as best as I can, but it is entirely possible that somethings have slipped through. I'd appreciate any extra eyes on them at all and any suggestions for corrections or improvements. If you think additional queries would be helpful, drop a comment and I'll do my best to add them.
With that out of the way, here's more than you ever wanted to know about spam and protection on RPG.se. I hope it will be helpful.
Feel free to use the answers to point out anything interesting or to dig in deeper.
What is a protected question?
Luckily there is a Meta.SE post that goes into great detail.
To quote the essential part from it though:
all users must have earned 10 or more reputation on that specific site to answer a protected question.
The other important thing to note is when questions are automatically protected by the community user:
- The number of deleted answers from users with <10 rep, plus the number of answers with helpful spam flags, is at least 3. (Note that spam answers from new users are counted twice.)
- Generally, this means that three deleted answers from new users will cause auto-protection, but if at least one of those answers is spam, only two answers will trigger it.
- Five answers from users with <10 rep were posted in the last 24 hours. [...]
The system will never unprotect a question automatically, even if the deleted answers are later undeleted or the spam flags are cleared.
Over the history of the site (~10 years) we have had 495 questions with at least one answer with a spam flag on it and 637 questions that have been marked as spam. That averages out to 113 posts that attract spam a year and 9 posts a month.1
We don't really have any tools to prevent question spam, so if we just look at the answer spam (which we at least have some tools to prevent) the average is about 4 maybe-preventable spam posts a month.
Looking at 2019, we had 127 questions with spam answers and 43 spam questions for an average of 14 spam posts per month (11 spam answers per month). (2020 is so far tracking a bit lower with about 11 total spam posts/month)
Out of all these questions that have received spam2 only 212 have ever been protected (so 58% of posts which receive spam are never protected).
Speaking of protection, over the history of the site we have had 943 questions ever be protected (out of ~40,300 currently undeleted questions, a bit over 2%). Given the previous data, this would mean that only 212/943 = 22% of protection activity involves spam.
Of all the protected questions, 149 questions, (15%) ever got unprotected (not counting the recent mass-unprotection event which will likely be reversed).3
~50% of all non-automatic protections on this site have been carried out by 1 user and the top 3 users combined make up 71% (the third user hasn't even been active on the site for years).
Of all the 136 automatically protected questions, 79 (or 58%) had at least one spam answer on them.
Out of the 943 questions that have ever been protected, 834 have at least 1 deleted answer (88%). Out of all the questions that got unprotected, 130 have at least 1 deleted answer. 31 of the questions that got unprotected had at least one spam answer on them.
Since Feb. 28, 2019 (which is when the HNQ status started being tracked), 51 questions have hit HNQ and also been protected at some point. This represents ~2% of the 3052 HNQs we’ve had over that same period of time. 12 of these protections were done automatically by the community user.
1 - These numbers include 2018 where we went through an enormous spam wave, so this number is actually lower if we remove that outlier. If we exclude 2018 from our averaging we’d get 153 spam questions and 381 spam questions over 9 years, so 60 posts a year and 5 a month.
2 - I’m not counting spam questions here because spam questions get deleted entirely, and are never protected.
3 - Per Should we reprotect the 700 or so questions that were unprotected in the last few weeks?. For the full list of questions that were unprotected see this Google doc.