So, I think there are at least three concerns dovetailing here.
First, I'm not sure who you're intending this to be addressed to. Only moderators can delete comments, but from what I've seen and heard the moderators are not going out of their way to do so without provocation. Moderators aren't that bored: they respond to issues that have been flagged for their attention (by non-moderator citizens such as ourselves) much more often than they go looking for moderation work.
This means that comments which get deleted usually have at least two people who think they ought to be deleted: the flagger, and the responding mod. Not a lot of unilateral deletion going on.
Second--and more specifically to this issue--there's often a fine line between explaining "why I don't like your question," and "how you can edit your answer to get me to upvote it." The former is not constructive, and the latter is only useful if the edit wouldn't change the answer entirely. We are encouraged to comment about downvotes only if we think the post could be improved.
In the case of your comment (I'm assuming this is reasonably close to your original deleted sentiment), I can't tell how that comment--nor the poster's responding comment--could be used to improve the answer. It's simply a conversation in which you say "I disagree," (your downvote expressed that just as effectively) and the answer responds, "Agree to disagree." Comments which boil down to "I disagree" are noise, and dilute this site's primary function as an efficient conveyor of good answers.
And yes, that is a rather hard line to take. It reduces the human interaction factor and that can make an environment less friendly--but then, the first time I ever downvoted someone, I left an explanatory comment which got an angry response that I "have a case of the doesn't-get-its." Explaining downvotes isn't a universal balm to ease the wounded hearts of the downvoted--often it's an invitation to comment battles.
If you want to propose changing the kind of critical comment which is accepted practice on rpg.se, that should be its own meta post. I'd be very interested in the discussion, and I'd be happy to help track down the existing debate about it both here and on meta.se so you can build on that instead of re-treading old ground. Until then, comments aren't for making people feel better, though I firmly believe we SHOULD be friendly and humanising in the comments we do make.
Taking these two points together, it looks like someone thought your comment wasn't going to help the answerer improve their post, so flagged it as chatty because--well, frankly, it kinda was. And a moderator saw the flag, agreed, and so the comment was removed.
And that brings me to the third point: the general deleting of comments. How often, how fast, what kind, who can, if it should happen at all... there's a lot of discussion about this already, both here on rpg.se and in the wider SE community. It's a rather contentious issue among citizens--but the moderators and site owners across SE have a LOT of experience which shows that tightly controlling comments is necessary for the Stack Exchange to retain its unique "answer-giving" position (as opposed to the "discussion-having" position of most online communities).
Again, if you would like to weigh in on changing the established policy, make a meta post here or on meta.se (depending on the scope of the change you propose), and base it on the existing discourse so we aren't re-inventing wheel. Unfortunately, simply saying "Doing this is bad" won't get any traction unless/until the community can be shown the ways in which the policy is harming the site more than it's helping.
