Are "is this balanced" questions on-topic?

Question

Before I start: I thought about the matter when reading this answer to the How is the community doing? [2018] post, which claims a larger percentage of negative reactions towards homebrew questions compared to other questions, and also urges more open-mindedness towards homebrew questions. The latter could potentially be relevant when answering this question, while the former will be of relevance down below.

---

To provide a short summary, "is this balanced" questions (RPG SE search query) usually present a homebrew "thing" and ask the community about their assessment of its balancing.

While I personally like these questions, both because it's refreshing to see other people's creative content (so-to-speak) and because answering them allows me to participate in this creative process, I don't feel like they fit in with the general objective and evidence-supported answer style that is commonly encouraged (or enforced) on the site. Sure, we have the knowledge and experience on the site to evaluate the balancing aspects of homebrew - but so do /r/dndnext or many other forums.

Now, usually, homebrew questions should be supported by experience instead of simply being reformulated brainstorming results. But how could you possibly provide an experience-based answer if the only person who has ever been in contact with the respective homebrew so far is its creator, who also asked the question? I highly doubt that anyone is ever going to take some potentially unbalanced homebrew (or even their own, rebalanced version) and actually playtest it, just to provide an answer to an RPG SE question. Yes, it might happen on occasion, but I don't believe that those very rare (if not legendary, to use 5e game terms) cases should be relevant for answers to this question.

Another argument against such questions is that they're not really going to help future readers - which doesn't automatically disqualify a question, but hugely limits is usefulness (after all, providing signposts is one of the main upsides of duplicates).
Asking homebrew questions such as Does not having flight nerf a warlock's familiar? could be useful for future readers, and not just for someone who wants to restrict familiars with the ability to flight, but also someone who's trying to figure out what to actually do with a familiar (RP aside).
Questions such as Is this homebrew elemental spear balanced?, however, are highly unlikely to help future readers, even if someone else needs to balance an almost identical homebrew item of their own (or would you do an extensive, unlikely-to-yield-useful-results google search to see if someone already made a similar item, before you simply asking right away if it's balanced?). The only thing those questions are good for - aside from solving OP's issues, of course - is giving someone reading the question ideas for their own homebrew items.

---

While researching existing questions before posting this, I also stumbled upon these questions: Are homebrew evaluation questions OK? and How to deal with feedback questions about house-rules?.

Both of the linked question's answers would classify most "is this balanced" questions as unfit for the site. Still, many of the questions that are listed by the search query mentioned above have neither been closed as "primarily opinion-based" (or for similar reasons), nor have they have received notable numbers of downvotes (questions that show a lack of research effort aside).

---

Therefore, I believe we should establish a policy of sorts on how to treat the described questions currently floating around somewhere between the balance- and homebrew-tag. In the case that they are considered on-topic, then either a custom tag or changes in the description of the balance tag would probably appropriate - I personally can't really tell what the 7-year-old phrasing

Questions about perceived inequities between characters derived from differences in capability.

actually tries to tell me - but it's not what I would expect from the "balance" tag.

Answer 1

I don't have a strong opinion on whether they're on-topic or not, but they often go like this:

Question: Is X balanced?

Answer: No, here's what would need to change for X to be balanced.

(Here's where things go horribly wrong...)

Question edited: Is X balanced now that I changed it?

That's not something that works in the SE format. It undercuts answers, undermines the voting system, and so on.

I'm posting this because it happens so often. Understandably, homebrew creators don't want to stop at "is version 1 balanced", they want to go through the process and have a balanced end product. And that highlights a problem with this type of question - it really works better as a back-and-forth dialogue, rather than a static question-answer. That makes them another thing forums do better than us, which doesn't necessarily mean that they're off-topic, but it is a potential reason why they maybe should be.

Answer 2

Yes, they're on topic and should remain so. We can answer them effectively and can get pretty satisfying answers to these questions that are good examples of our quality guidelines. We agree they're relevant to RPG expertise and they get answered based on a mix of subjective expertise and objective reference.

Declaring a category of RPG questions off topic usually only happens after there's clear evidence of ongoing systematic major breakdowns & malfunctions inherent to handling the topic on our site. Whilst these questions have some hitches (like sometimes OPs revise/update the homebrew in the question after feedback and expect answers to update accordingly, which isn't how we work here) that's nowhere near the kinds of breakdown levels that renders stuff off topic and can be handled well by normal community moderation.

Citation standards on homebrew balance questions

Now, usually, homebrew questions should be supported by experience instead of simply being reformulated brainstorming results. But how could you possibly provide an experience-based answer if the only person who has ever been in contact with the respective homebrew so far is its creator, who also asked the question?

You seem to be referencing our policy on Is homebrew an acceptable answer to a question?, but you're taking it too broadly: it's not meant to be applied to every answer that is in some way dealing with homebrew. The context of this meta policy is that we were running into situations like the following:

Q: How can I speed up combat in D&D 3.5e?

Bad homebrew answer: Here, use this custom super-fast combat system I made up just now in about ten minutes. I'm sure it will work great and speed up combat, even though I've never used it nor even seen anything like it and have no evidence it even works at all in practice.

Good homebrew answer: My group was also dissatisfied with combat speeds. We used these techniques/​mechanisms/​tools/​homebrew to speed it up. Here's how it worked out for us in practice, and here's the specific benefits and drawbacks we experienced. Here's how I recommend applying it to your situation.

That policy was written in the context of parsing apart those bad answers from the good answers. It's a narrower application of the Back It Up! principle from Good Subjective, Bad Subjective. Boiling it down, you should ground your answers in objective references and/or in comparable subjective experience by you or someone else whose experience you can cite. In that narrow application, it means we don't want random ideas but actual tried-and-tested solutions that we can see will work. In general it means we want to have some confidence you know what you're talking about and aren't basing your answer entirely off your personal opinion.

The homebrew balance questions I usually see reference back to both personal experience and objective fact. I'd check the homebrew+balance tag intersection as well as your own query: answers to these questions frequently use existing game features as a comparison baseline (e.g. comparing a homebrew cantrip to other spells) or point to the game's own balance guidelines (e.g. the loot guidelines, monster construction guidelines, or encounter difficulty guidelines). One of the questions you cite (Is this homebrew elemental spear balanced?) received relatively mediocre answers, but they still reference standard wealth-by-level guidelines and draw comparison to other similar items.

In your answer to Does not having flight nerf a warlock's familiar?, you're answering based on both objective comparison to other familiars and based on subjective experience about how familiars work out in gameplay. This is the kind of subjective expertise we like to see people draw upon when they back up their answers: you're demonstrating an understanding of how things play out at the table, and using that to inform the advice you're providing, so as to give us some confidence your advice can work out. Similar expertise gets drawn upon when examining a pokémon master class's balance: an experienced player can draw upon comparable personal experience to advise on the homebrew in question.

General helpfulness now & in the future

Another argument against such questions is that they're not really going to help future readers - which doesn't automatically disqualify a question, but hugely limits is usefulness (after all, providing signposts is one of the main upsides of duplicates).

Our primary mission is to answer the question someone has and solve their specific problem. Helpful guidance for future readers is our secondary goal that's produced as a side-effect that we also value and curate. Our homebrew balance questions meet both goals: (1) we give the person feedback on the relative balance implications of the homebrew they've brought up, and (2) we provide a source of homebrew balance advice that may apply to future readers in comparable situations. (They may be few and far between, but we don't aim solely for those questions that maximize impact and relevance.)

All seems OK here.

So, homebrew balance questions are relevant to our expertise and are answered drawing upon that expertise in line with our objective and subjective citation guidelines. There are some mediocre answers and some bumps we run into from time to time, but that's par for the course. There's no major breakdown or mismatch warranting off-topic-ness. We can assist people effectively, and over time build up a body of relevant advice about homebrew balance. Everything seems OK enough to leave them on topic.

Answer 3

I have seen a few questions and answers that provide both a structured, answerable question, in a "yes" or "no" format. As a very simple example:

Is this "Timberwolf" balanced?

The question provides the Homebrew (in this case, a creature), with all information required to give an answer. And the answer is "No, and here's why..."

What this does is provides, firstly, a source of information for other users - in this case, the CR calculation table that might often get overlooked (I personally didn't know of its existence until recently, and even reading through the PHB for the first time, I overlooked a lot of stuff, simply because I didn't know what to look for).

Over on Gaming.SE, there is a very similar problem with Minecraft crash questions. A lot of the time, people will simply say "Minecraft crashed. Halp." Obviously not very informative. Other times people might provide system information, situational information, and sometimes a crash dump, which on its own can be enough information to provide and answer.

What I'm getting at is that while Homebrew questions can be very open, non-specific, or highly situational, when formulated in the right way, you still can get information from them, and more importantly, inspiration as well.