Most of the time the sort of comments I'd like to downvote are comment-answers that use the inability of people downvote comments to look highly supported. I flag these as comment answers and they get taken care of, though there is no specific flag for that complaint and I use the 'other' flag every time.
Sometimes, though, they are just suggestions or statements starting with "note that..." on an answer or question that I strongly disagree with. Because they can't be downvoted and tend either to make strong claims or be steeped in a strongly opinionated subset of RPG culture these comments often get a decent number of upvotes over time. The worst of these I can just flag as 'not constructive'. The better ones, however, I can't (since it's not clear with a cursory inspection why it's not constructive).
The options I see are:
Flag it as not-constructive anyways. Seems ok, except that puts me and one random mod in charge of deciding whether or not every comment suggestion or addendum is 'right' or not, regardless of how useful other people think it is as a suggestion for improvement. This also enables edit-wars where Alice posts a comment which Bob deletes but then Charlie posts almost the same comment again, which Dean deletes, and so on. Worst of all, you can't blame people for re-posting or flagging the comment since they have no way of knowing it was ever posted or deleted before. Sounds like a mess.
Leave another comment responding to the offending comment. This is what the community seems to do as the status-quo. It leads to lengthy comment 'discussions' that are really more like arguments (though one side is usually fairly clearly on superior footing academically speaking). These inevitably lead to either a chat room (rare) or Dr. Ballsun-Stanton nuking it from orbit (somehow more common. And it's pretty much always him, too. How three mods manage to manage the entire site I do not know). Existing policy explicitly discourages eliciting discussion in comments. I'm okay with ignoring that guideline if the discussion is strictly limited to how best to improve a question or answer and stays very short, but the kind of discussion this response elicits is definitely not okay.
Give up and just let it be. This results in misleading, erroneous, and/or insidiously offensive advice being disseminated via our platform as a sort of 'necessary casualty'. That is unacceptable for what I hope are obvious reasons.
Solicit meta.se for the ability to downvote comments, which I expect will be denied on the grounds we shouldn't be using comments that much anyways, what with them being 3rd class citizens. This does seem to have the least negative side effects though.