# What is the issue with questions looking for averages or statistics?

Over my relatively short tenure here so far, it is my impression that there seem to be two schools of thought in the community of active contributors:

• one sees practical value in using averages based on generic assumptions or on collected statistics (this is my current perspective)
• the other sees such averages as meaningless or not of value, and recommends asking questions for either a concrete situation, or about immutable mathematical truths

The second perspective can also motivate people to vote to close questions as opinion-based, like this question about the average number of encounters per day in a D&D 5e game.

What I understood from discussion at the time is that this could vary widely from table to table, and, so the conclusion, therefore it would be of no value in providing an averaged view; which, I think, is a slightly different argument than it being opinion based, it is an argument that it is artificial and not useful – but there is no closure option for that.

What reminded me today of this subject was that Thomas Markov put a bounty on an answer that is the equivalent of "we cannot say" (in this case, for a question on what the average to-hit chance per level is). I am grateful that he is not going quite as far as voting to close the question as opinion based (the fate of a closely related one on if one can use an average AC per level chart in DPR calculations), but he is offering 500 of his hard-earned Internet-Points for making great answers to support that view.

If I understand it right, he is a professional statistician, or at least, works in a role that requires professional use of statistics, and this makes me wonder if there is something fundamental I am missing.

Staying with D&D 5e, as a thought experiment to support such a view, I could imagine a campaign world set after a magical cataclysm, where our heroes are among the only survivors, everything else they encounter is either a Zombie (AC 8), or a magically animated armor (AC 18). In this world the average encountered AC would be 13. Clearly a character would never get to fight an opponent with an actual AC 13, and an ability that allowed the character to always hit AC 8, 50 % hit AC 13, and never hit AC 18 would have a very different performance than what one would expect from that average in individual encounters with either monster type.

Such an extreme could of course exist, and here is exaggerated to make the point. In practice, I think extremes and outliers are rare, and because they are rare, there is still value in averaging, even if it will not work in a situation like this. But of course, to prove that with a low statistical chance of being wrong, one would need to sample a large number of modules or tables.

Most other questions that do not just ask for rulebook exegesis, for example how to handle problem players or how to deal with issues of player agency do not get closed as opinion-based, even though answers can often only be based on personal experience of one writer, and not on aggregated, objective knowledge across many tables, either.

But when the question is about game statistics encountered in play, it appears that personal experience from years of play is not enough any more. So what I'm wondering about are particularly about these related aspects:

1. Do questions about numbers have to be mathematically stringent, or are rough approximations that do not hold in some (maybe even many) cases acceptable, without counting as opinion-based?

2. Why would questions that are asking for statistics – either based on rulebooks, from surveys, or from collecting data – be opinion-based?

• There might be some irony here in that the issue is far more about the nuances of individual questions than about general perspectives on broad categories of questions. Oct 31, 2022 at 23:48
• The usefulness of averages is context dependent. There are circumstances the average is useful and meaningful, and circumstances when an average is not; they are not magical truth generators but instead circumstantial tools useful in specific situations for specific kinds of determinations that have to be applied thoughtfully when appropriate (there's multiple kinds of average for that reason). Your assessment that there's two school of thought—that averages are always useful or always not—is overlooking that these perspectives might be applied by any one person under different circumstances. Nov 1, 2022 at 1:19
• Related question, How do I ask statistics questions on this site?, and my answer there is very much the same flavor as my answer below, and really basically the same thing I said to you in this comment when that question was asked. Nov 1, 2022 at 20:23
• "everything else they encounter is either a Zombie (AC 8), or a magically animated armor (AC 18). In this world the average encountered AC would be 13": it depends. If the percentage of zombie is 90% and the percentage of animated armor is 10%, then the average armor is 9. Just this small example shows how one must clearly state the specifics of the scenario. Nov 2, 2022 at 15:24

### Many of these "what is the average of...?" question should be setting off your XY alarm.

You mentioned my occupation, so I'll give some insight into how it applies here. Part of my job involves collecting all sorts of data, finding meaningful patterns, and presenting those meaningful patterns to stakeholders so they can make good business decisions. Sometimes, those stakeholders have ideas about ways to look at the data, other descriptive statistics they want to see. Part of my job is to tell them if looking at the data that way can actually be helpful, or if it doesn't really apply to our process. As I have asked you many times, I have asked my vice president of operations: what problem are you trying to solve with this information? He wants to know if trying to use that information is going to be a waste of the company's time, and perhaps more importantly, if trying to use that information could lead to poor business decisions. So I ask why he wants to know, he tells me why he thinks that information could be useful, and I advise him on the limitations of the data, the limitations of the statistics he's looking for, and whether or not I think it can actually tell him what he thinks it's going to tell him. In this business, there is no such thing as information for information's sake. If it isn't going to make us more money, we aren't going to waste time trying to figure out.

Obviously, the stack is not a business, at least, not for us. We can handle questions that are looking for trivia. But, the guiding principles for asking good questions are very similar. Just as my boss wants to know if the answer to his questions are actually going to help him make sound business decisions, the best Q&As here are those that are going to lead to positive outcomes at the table of play. Of course, not every question has to even attempt to do that, What's the greatest number of hands I can have to annoy my mother-in-law with? comes to mind as one of our successful "just for fun" questions with zero consequences for the table of play (though I hope Bluemoon was able to patch things up with his mother in law).

But when an investigation does appear to have potential consequences or application at the table of play, it is the sort of question where knowing "what are you planning to do with this?" or "why do you want to know this?" provides answerers with the tools needed to write fantastic answers that improve your gaming experience. And this is where your XY detector should be going off. Consider your question What is the average number of encounters per day?. I explained in a chatroom on the question how it runs into the XY problem. You wrote in that question:

How much damage such a feature can be expected to contribute to combat depends heavily on how many encounters you'll have in a day, because once you run out of uses, it won't contribute any more damage. When planning your character and estimating average expected damage, you therefore often have to make an assumption about the number of encounters per day that will use up some of those limited resources.

I'll just quote my take from that chat:

From what I can tell, the problem you are trying to solve is "I want an idea of how many encounters per day I can expect when I am building a character, because different builds perform differently with different numbers of encounters per day".

You have asked a question about what you think a good solution Y is, "what's the average encounters per day?", because you think knowing that will equip you to better build your characters.

Classic XY problem.

Instead, an on topic question might be something like "Some character builds are really powerful with only a few encounters per day, but taper off quickly as their resources are depleted. I want to take this into account during character creation, so how can I know how many encounters to expect on average at a given table?"

This is the sort of question where experience based answers are what we are after. Because it is experience solving your actual problem, not experience with a mostly unhelpful solution you thought of.

Your question asked "what is a good assumption for the number of encounters per day?", and the answer is "there is no good assumption because it will be wrong more often than it will be right". Instead, the best solution to your problem there, and the answer to the question I suggested in chat, is to just discuss it with the dungeon master and other players at a session zero or before the game starts so you can inform you character creation decisions accordingly. The average number of encounters per day varies from table to table, so just figure out what it is likely going to be at your particular table, rather than bringing to bear some arbitrary expectation that is probably going to be wrong for most tables.

The thing I really want to know is whether or not the answer to a question actually leads to worse outcomes at the table. Using a statistically accurate estimation of encounter per day as a tool for informing your character creation decisions is likely to lead to worse outcomes for you at the table of play. You will disappointed when the expectation you had was not met. This is why it is important to tell us why you want to know something. And this is why I bountied Theik's answer: they observe that bringing the expectation of 65% chance to hit to the table of play just isn't a realistic expectation, even if it is statistically accurate.

This is also my concern with your more recent question, What is the average damage per attack for monsters by CR?. You state in the question:

The DMG on page 274 has guidelines about damage per round for monsters of each Challenge Rating. However, most monsters deal that damage as a combination of multiple attacks (and sometimes other abilities like breath weapons).

The reason this information is given is because it motivates the CR calculation used for monsters, and CR is the primary metric used in encounter design. In other words, damage per round is an important consideration for assessing the challenge a monster brings to a fight. This is the metric you should be concerned about when designing encounters. However, your question asks about damage per attack, but doesn't give any motivation for why. You already know the expected DPR for a given CR, which is the metric I'd expect you to be using in encounter design, so I have no idea why you want to know damage per attack independent of damage per round. My XY alarm is going off. Damage per attack is obviously going to be an inferior metric to damage per round for the purpose of encounter design. If you are trying to solve a problem with encounter design using damage per attack, I want to know what it is so that I can tell you if damage per attack actually helps you. And if you just want to know for the sake of knowing, just say that. It's okay. I withheld my vote to close that question because it is a question about the averages of readily accessible numbers. It isn't opinion based. But I've downvoted it because I don't think it is a good question, and I think the correct answer to the question could lead to bad outcomes at the table of play if it is used in the wrong way. Or as the downvote helper text says:

This question does not show any research effort; it is unclear or not useful

• Thank you for taking the time to write this in depth answer. With the damage per attack, I guess at the moment it is pure curiosity, as I do not have a concrete problem to solve with it right now. It was caused by me trying to estimate, how much damage an improvement in AC would save per typical attack, for a question that tries to use the Dodge action in a homebrew manner. But I was answering that without it, as I concluded the AC in the end may not be the deciding factor in the context of that question. Nov 1, 2022 at 16:38
• So, in a business context, you would probably not invest in solving it unless you have a concrete problem granted. I can think of such a problem like "How good would a feat be that grants you Uncanny Dodge -- would you need more to make it balanced against other feats", but it is not one I have right now. Nov 1, 2022 at 16:39
• For that question about encounters, I believe that the variance from day to day of adventuring, at least in my experience, is probably larger than that from table to table. We have days with only one large battle, others with up to 5 or 6 small encounters when exploring a dungeon. I think you may be right that if you use an average of 3, and get to the wrong table, then it may do more harm then good. And, even though it is not easy to separate myself from my preconception, the question here was an honest one to better understand your (and others) perspective. Nov 1, 2022 at 16:45
• This, like many alleged XY-problem concerns in this stack and others, is counter productive. I've read this answer three times, focusing especially the "average number of encounters" example, and to my eye it looks like you're getting so caught up in the form of the question that it is damaging both your ability to provide reasonable answers and the ability of querents to ask questions in natural language rather than the language of your expertise. Nov 7, 2022 at 17:36
• Groody asking "What is the average number..." does not compel you to answer in the form of a number. It does not require you to reject it until the phrasing explicitly permits you to give your preferred answer. You are an expert in statistics, yes. But your role as expert should not be to reject answers until they are as well formatted as you would expect in your professional capacity. Your role as expert in both stats and RPGSs should be to provide what you believe to be the correct answer (as a frame challenge if necessary) despite the phrasing, perhaps with a brief explanation of why. Nov 7, 2022 at 17:36
• If the answer to the question is, "There is no good answer, here is why..." than that is the answer. You can say that. You should say that. Much of the rest of this dance of voting to close until the question meets your satisfaction just adds friction and frustration to the process. I would urge you to reconsider your position on this; to find better ways to use your expertise without adding that necessary friction; and to not make this into our new obsession on how to reject questions. Nov 7, 2022 at 17:36
• @Novak I think you’re responding to a different position than the one I’m taking, but that likely has something to do with my presentation here. I don’t think being an XY problem is a reason to close a question, and my voting habits reflect that (at least my more recent habits do, not sure what I was doing a year ago). Nov 7, 2022 at 17:39
• There's also a problem I used to run into often: an ongoing confusion between the "expected value" of a population, the mean of a tightly-clustered Gaussian, and the performance of a subset of that population. Roughly, "this part will last about 1000 hours, because the MTBF for these is 1000 hours." Point here is that it doesn't matter how "D&D" is, it only matters what will happen at your table. Nov 14, 2022 at 1:14

### Statistics questions that involve player & DM decisions are (usually) opinion-based.

From Wikipedia:

Two main statistical methods are used in data analysis: descriptive statistics, which summarize data from a sample using indexes such as the mean or standard deviation, and inferential statistics, which draw conclusions from data that are subject to random variation.

Notice a major commonality between those two methods: they work on data. No data means no statistics.

Consider that question about average chance-to-hit per level. To give a proper statistical answer to that question, we would need a dataset with a large number of sample attacks, including at least the attacker's level & attack bonus, the target's AC, and whether the attack has advantage or disadvantage. Further, that dataset would need to actually be relevant to the asker.

Now, it's admittedly possible that an answerer could get a dataset relevant to the asker by either A) conducting a highly expensive worldwide survey of gaming groups to get a statistically useful representative sample of attacks over the course of each group's campaign, or B) stalking & spying on the asker's group to build a representative sample of that group's attacks over the course of a campaign. In practice, however, that's not what happens; instead, answerers generally just come up with their best guess (i.e. their opinion) for a dataset.

Some statistics questions are answerable purely from publicly available data (e.g. "What's the average Constitution save bonus by level of monsters in the Monster Manual?"), and those are perfectly acceptable.

• Some statistics questions can generate their own data, if the boundaries are well known - e.g. "What's the typical chance of success using roll and keep for various pool sizes" can be computed.. But that requires sufficiently known boundaries, in this case: pool sizes and difficulties. Nov 1, 2022 at 16:54
• @Trish I would technically consider that a probability question rather than a statistics question, but the difference is immaterial to a lot of people on the site, and you're right that it's a valid question type.
– Oblivious Sage Mod
Nov 1, 2022 at 17:17