20
\$\begingroup\$

We, the elected moderators, wanted to take a minute to check in and see how the community feels like things are going on the site.

We did this in 2017 and 2018 and it was very helpful. So again we'd like to share our thoughts and also get yours on what is going well and what we could improve.

How this specific Q&A is run so that we get good value out of it

One clear premise per answer

  1. We'll post some of the things we've done/observations we have/things we think still need improvement. Please share your thoughts as well! Make sure and note whether you think the thing is an improvement, a problem, or some mix of the two (one person might see the same change as bad that you see as good, or vice versa)...

  2. Post one kind of thing per answer, so that when people upvote/downvote based on whether they agree or not it's more clearly actionable - if you write an essay about 4 different things, it's not going to be clear what part(s) people agree or disagree with and thus it becomes unactionable. You can of course contribute multiple answers.

  3. Upvote or downvote the answers based on your agreement with whether you see that thing happening and concur with the answer's premise that it's good or it's a problem. (In other words, if someone says "We get too many new users and I hate them," you would upvote if you agree, and downvote if either you don't think we get too many new users or if you don't hate them.)

No long comment threads

  1. If you disagree with an answer, post your own answer, don't argue in comments. If you post more than one comment on an answer, you should consider if that is useful at all. We're interested in overall community sentiment as shown by votes, not so much that one person is so irritated they post 10 comments.

  2. This isn't the place to workshop solutions - if a problem gets a lot of votes, we should open a new meta question to do justice to that issue. Solutions hidden in a comment thread on one of these questions are unable to be clearly vetted and voted on so they will tend to go undone.

As usual, Be Nice applies to meta as well as the main site.

You may strongly disagree with other users or with the mods or whoever, but we trust you can find ways to express what you like or don't like without being hostile or insulting to others. Focus on actions rather than characterizing people.

So here we go - our thoughts on things that are going well or need improvement. Feel free and add on!

\$\endgroup\$
33
\$\begingroup\$

Homebrew review questions still aren't great

It seems like there are more of them, and they are generally following the new guidance How can I ask a good homebrew review question?.

Unfortunately, IMO they are not a great part of the site. We get multiple revisions of the same thing clearly with no playtesting (in between or initially, by the poster or the answerers, see also When do homebrew suggestions need to be playtested by an answerer?). They're a tough fit for the site Q&A format in general.

We should probably have a new meta on how to improve these or what we want to do with them.

\$\endgroup\$
  • 7
    \$\begingroup\$ Wishes to add bounty to this answer :) | when I answer a question like this, with no other answers, and the person immediately revises with my suggestions (usually only around +3 votes) and posts a new question, it just feels... off... \$\endgroup\$ – David Coffron Apr 30 at 15:45
  • \$\begingroup\$ I am also a little disappointed in how the homebrew thing is working out so far - it does feel a bit clunky in terms of how we need to approach it. My druid homebrew question got good responses, and I have not yet posted my last version due to what you point to: no play test. What I thought was going to happen, didn't. ). \$\endgroup\$ – KorvinStarmast Apr 30 at 22:20
  • 11
    \$\begingroup\$ I think they are a valuable part of the site and absolutely on topic. But we should more strictly enforce the policy, particularly the waiting a few days before posting a revision. \$\endgroup\$ – linksassin May 1 at 2:06
  • 3
    \$\begingroup\$ @linksassin I'm curious, what makes you think of them as valuable? \$\endgroup\$ – Miniman May 1 at 2:30
  • 4
    \$\begingroup\$ @Miniman Happy to discuss but this isn't the place for it. See chat.stackexchange.com/transcript/message/50122282#50122282 \$\endgroup\$ – linksassin May 1 at 2:49
  • 1
    \$\begingroup\$ @DavidCoffron is there perhaps a way to have a popup on the edit page on questions tagged with homebrew that says to not edit suggestions into the original question? E.g., "Looks like you're editting a question tagged homebrew. Please do not incorporate suggestions from answers into this question." \$\endgroup\$ – Captain Man May 3 at 18:26
  • 1
    \$\begingroup\$ This answer should be discussed on its own meta post here: rpg.meta.stackexchange.com/q/9096/48759 \$\endgroup\$ – linksassin May 13 at 3:10
31
\$\begingroup\$

The community is continuing to perform most of the site moderation activity

In a trend continuing from last year, the community is performing the majority of close/edits, is flagging plenty of comments, and in general is making the mods do less; I know my site workload continues to slowly shift downward, which is great.

See also 2018: a year in moderation

\$\endgroup\$
  • 3
    \$\begingroup\$ If anybody is interested, I have crunched some unscientific numbers regarding moderation activity for all of 2018 for posts and comments that looked positive from a numerical standpoint as well. \$\endgroup\$ – Rubiksmoose Apr 30 at 15:32
24
\$\begingroup\$

Answers in comments may not be reduced, but are handled quickly by the community

I'm not sure if we are getting less comments in the answers1 (as I had hoped in my answer to last year's community check-in), but I think we are dealing with them much quicker and with much less squabbling then in the past.

In my experience, they are generally left by newer users and are generally swept up by community flags and/or mods before any issues could be caused.

We may even be over-flagging some types of comments as answers when they shouldn't be and this might have to be addressed at some point.

Overall though, I think we are much improved over last year.


1 - An informal survey of my personal flags might indicate a slight increase over last year at the same time, but that can be due to many factors (traffic, how actively I'm flagging, etc.). There's still no easy way to figure out a quantifiable way to measure the issue that I've heard or been able to come up with unfortunately.

\$\endgroup\$
  • 2
    \$\begingroup\$ As someone who occasionally finds this rule personally frustrating... can confirm that it is enforced aggressively. \$\endgroup\$ – Ben Barden May 1 at 17:09
  • \$\begingroup\$ @BenBarden out of curiosity, are you saying that you're frustrated by things you think aren't answer-comments getting deleted, or that you disagree with the policy and are frustrated to see answer-comments go? \$\endgroup\$ – nitsua60 May 1 at 20:28
  • 6
    \$\begingroup\$ @nitsua60 I am frustrated in that there are times I wish to offer useful feedback that is not a full answer, and the current policy prevents that. I understand why, and I'm not challenging the policy, but I do have the experience of being someone who might at times wish to offer answers in comments but is quite certain that any attempt to do so would be shut down rapidly and thoroughly. It seemed a useful viewpoint to offer on the topic. \$\endgroup\$ – Ben Barden May 1 at 20:45
  • \$\begingroup\$ @BenBarden gotcha--thank you for chiming in. \$\endgroup\$ – nitsua60 May 1 at 20:55
  • 5
    \$\begingroup\$ I barely see comment-answers in non-D&D questions, just my two cents. \$\endgroup\$ – ShadowKras May 2 at 12:21
  • 4
    \$\begingroup\$ This community is great about this. It has changed my opinion on things stuff like "not answering your question specifically but XXX". I wish more sites had this mentality. \$\endgroup\$ – Captain Man May 3 at 18:17
16
\$\begingroup\$

Meta is healthy and productive

There's a steady growing stream of meta questions and answers that are well participated in by the community and are not degenerating into fights. This shows me that the community has settled in and understands how to use meta to make decisions and explain site policy.

\$\endgroup\$
15
\$\begingroup\$

The Hot Network Question algorithm seems to be bad at picking out hot RPG questions

Currently it seems to me that way too many of our questions make HNQ status (even if only for a little bit). Questions that are clear from a human glance have no lasting "hotness" to them and are definitely not worth showing off to other stacks about. Of course they also do sweep up a real hot one every now and then. To be clear, I'm not arguing that HNQs as a concept are a poor fit for the site or even that the current implementation is causing harm, but it does seem like the algorithm is a bad at getting hotness right on RPG.se.

I've crunched some numbers which show an inordinate number of our questions making HNQ:

March 2019

enter image description here

enter image description here

It seems highly unlikely that 44-50% of all of our questions are truly "hot" and we often note in chat about individual cases where it made no sense to us that it was being labeled as such. It seems the algorithm is too sensitive here at least (I'm unsure if this is just something we experience here at RPG.se or if this is also seen on other stacks).

I've also seen this viewpoint echoed by several people on Chat (especially since the introduction of the HNQ bot in chat).

I realize that this is not something that we can change here, but seeing that I get the impression that SO is looking to maybe continue to make tweaks to the HNQ system, I thought it might be worth mentioning here.

\$\endgroup\$
  • 14
    \$\begingroup\$ We are obviously objectively hotter than all the other stacks. \$\endgroup\$ – mxyzplk May 6 at 20:38
  • 1
    \$\begingroup\$ I've opened a chat room for posting questions that ought to be on HNQ but aren't, for the purpose of feedback on training the HNQ algorithm: RPG.SE HNQ Worthy Questions \$\endgroup\$ – doppelgreener May 7 at 13:38
10
\$\begingroup\$

We dealt with a very RAW issue as a community

From my point of view, the conversation around the current resolution to the longstanding issue of the rules-as-written tag went down with significantly less drama than it had in the past as well as a broad show of support for the proposal to remove — the top two answers got +38/-9 and +27/-5 respectively which are impressive numbers for meta on any issue.

Regardless of how one feels about the solution we arrived at I think that the process itself showed a strong feeling of community maturity.

It wasn't the solution many of us wanted to resort to, but I think the community recognized a need and did something about it in a largely positive and constructive way. All the diamond mods did a good job in the presentation of the issue and in interfacing directly with people in a way that preempted and discouraged any nastiness.

Obviously it is early days in the post-RAW-tag era, but I think the process of actually handling the issue went much better than past attempts and left me at least with a positive view of the community and our ability to come together to tackle big issues maturely.

\$\endgroup\$
  • 4
    \$\begingroup\$ I was in favor of getting rid of the RAW tag, but I couldn't see how the vote counts indicated a clear consensus. There were plenty of net-negative answers in favor of getting rid of it, and plenty of net-positive answers in favor of keeping it or offering some middle ground. I was unconvinced there was a consensus. (This is my one comment on the matter, just expression my doubt/confusion about the consensus.) \$\endgroup\$ – Bloodcinder Apr 30 at 17:59
  • 1
    \$\begingroup\$ @Bloodcinder Well the best-made and most upvoted answers have very strong ratios, in my opinion (38/9 and 27/5 specifically). I don't consider the lower answer to be indicative of views on the whole matter though, after all some of them had huge flaws in their arguments. Regardless, I respect and appreciate your PoV on the matter! Mine certainly isn't the only one that matters (and voting certainly seems to indicate some disagreement with it). \$\endgroup\$ – Rubiksmoose Apr 30 at 18:04
  • 6
    \$\begingroup\$ Perhaps replace consensus with strong majority. 9 dissenting isn't "consensus" per se, but +29 is a super compelling vote total. We don't require consensus for actionability on meta. \$\endgroup\$ – mxyzplk Apr 30 at 18:45
  • 1
    \$\begingroup\$ @mxyzplk good thinking. I've tweaked the wording to be more accurate to what I was trying to convey \$\endgroup\$ – Rubiksmoose Apr 30 at 20:23
10
\$\begingroup\$

A brief look at the numbers

The elected mods have access to some "under the hood" analytics, and this meta had me thinking that we could be a little more proactive about sharing some top-line observations. So here are a few things that jump out at me, feel free to comment if there are other things that you think would be instructive to collate year after year.

  • Visitors and active users (post, edit, vote, flag, &c.) trend up a little bit. For exactness, I'm comparing the April '17-April '18 numbers vs. the April '18-April '19 numbers: users who visited >=1 dy/wk, users who visited >=5 dy/wk, users who acted >=1 dy/wk, users who acted >=5 dy/wk. They're all up somewhere from a few to ten percent, by eyeball.

  • Page views are up double year over year and rising quickly so far in 2019. During the last 16 months (when they started tracking this for us) we see pretty regular month-over-month growth, excepting December (which craters every year). But January/February/March this year saw a big surge in those numbers--like 50% higher than the couple of preceding months. The numbers are coming back down recently, so it's not easy to determine whether that's a "real" growth in visibility and we're in the middle of a little dip, or if it was just a bulge and we're reverting to a more-stable mean now.

\$\endgroup\$
  • \$\begingroup\$ 3-mo. update: this post was written at the peak of a apparent Spring Surge in page views. The numbers are back down to what a naked-eye extrapolation of April '18--January '19 would have predicted. \$\endgroup\$ – nitsua60 Aug 5 at 20:08
6
\$\begingroup\$

We lack a consensus regarding system-agnostic questions

I shall start by noting that this is by no means a major issue, since the volume of system-agnostic questions has been fairly low in the previous months, but let me dig up a bit of discussion from earlier this year anyway.

System-agnostic questions (and the related tag) have been subject to some controversy in chat. Basically, many prominent community members (myself included) tend to view purportedly system-agnostic questions with an inherent suspicion; it has a history of being misused in various ways (usually boiling down to the assumption that all RPGs are either DnD or "close enough").

However, during 2019 the topic has been discussed several times in chat and there are certainly dissenting opinions. I do feel that perhaps, over the years, we might have overcorrected on the acceptable types of system-agnostic questions resulting in a possibly too strict culture. Discussing for a new community consensus is appropriate --- the topic was last discussed in depth in this Meta from last February, but has had relatively little traffic and discussion, and the top answer's suggestions have not, in my experience, been comprehensively adopted by the community.

I think we need a new discussion with the intent of defining better guidelines on how to draw the boundary of answerability for system-agnostic questions, and what kinds of answers do we find acceptable (in particular, I would really want to avoid system-agnostic questions getting clearly DnD-minded answers by default). However, the main reason to have this discussion is simply to get back on the same page as a community, even if to reaffirm any existing practices if nothing should change.

\$\endgroup\$
  • 1
    \$\begingroup\$ I don’t think another discussion is needed, that meta is recent and has more discussion and voting than most meta Qs; if someone is ignoring our clear meta guidance, point it out when it’s happening. \$\endgroup\$ – mxyzplk May 24 at 12:05

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .