Quoting the official description of downvoting priveledge:
Use your downvotes whenever you encounter an egregiously sloppy, no-effort-expended post, or an answer that is clearly and perhaps dangerously incorrect.
There is nothing like "Because I have a different opinion" on the list, and all the possible reasons listed would also most likely require some other action, such as trying to improve the answer by commenting and/or editing, or, if it is so bad that it cannot be improved, flagging it so it is sooner or later deleted. In the last case things are obvious, downvoting is a temporary action so that the answer sinks in the bottom in the time before a mod can review it and delete.
In first two cases downvotes will remain even after the answer is improved, and make a negative impression for those who will read it later. As of my experience on other sites with voting system, if you see a vote before you vote yourself, score is very unlikely to change because of some strange form of hive-mind. If people see a negative score, they most likely vote down, often even before reading the post, and vice versa (vote up before reading the post) if the score is overwhelmingly high. This means that only the first couple of votes really tends to affect the overall public reaction to a post.
As I understand SE's policy, it is more important to improve existing answers than to delete them, so a downvote may possibly mean "Improve it". However, I have seen a lot of answers downvoted without any reason specified, and some of my generally positively rated answers downvoted without any negative feedback in the comments, I would assume that everyone around has experienced this kind of stuff. An upvote is a clear reaction of "This is a good way you follow!". A downvote is "You didn't do well!" without actually specifying what's wrong, so it is not really helping the author to improve his post writing skills or his post.
So, here is how I would see a possible new voting system.
Votes get hidden from public view, and you can only see vote count after you vote yourself. You can try to guess the approximate score by question's or answer's position, but nothing more. This will make users think individually instead of behaving as a hive. You can always see a vote count for your own posts.
New answers are shown at higher positions for some short period of time than they should be based on their score only, just as comments on YouTube or game reviews in Steam Store, so they also have a chance to get public attention. Otherwise, on average, comments already in the top tend to get even more attention and be upvoted by hivemind, and new posts just don't get the same amount of views to hijack high positions. It is possible sometimes that a post is not relevant anymore. For example, the author asked for a rule clarification and the current accepted and/or top answer is based on interpretations and even some assumptions, and a new answer appears after a year when this rule is clarified by game developers or someone finds a clarification elsewhere in official sources. Of course, it is often important to help the asker in-time, when he needs to use a rule in a game session rather than after a year, but I see the primary role of this web-site as an archive of answers that you can read at any time after the question was asked.
When you downvote, you are asked if you really want to do it by a popup, and have to confirm your choice. Then, if you do, you are forced to leave a special anonymous comment that describes your reason of downvoting, or vote for an existing one (to avoid "Me too!" comments), you can select and write several reasons. Such downvote comments are only visible if you can see the post's vote count (it is your own post or you have already voted for it) and needed to give the user knowledge of what he did wrongly without breaking the anonimity of comments.
There should appear a special priveledge similar to voting for closing questions which would allow declaring a downvote reason irrelevant for the post: if some given amount of users votes against a downvote reason. If a downvote has no more reasons marked as relevant, it is no longer counted. It should be used if a reason is irrelevant by the moment of removing it, such as if the post was edited or the reason was irrelevant from the beginning. For example, if users downvote a post saying that the question is about Vampire: The Masquerade and the answer quotes Vampire: The Requiem rules without even bothering to convert them, and the issue is solved by an edit, for example, by quoting relevant VtM rules, users can vote for this reason to be irrelevant and the downvotes will be removed. If, however, there is still another issue left, like "This question has too much of irrelevant info", those downvotes that have also been associated with it will stay.
I hope that my system isn't too complicated and it is clear what I mean.
This is not a duplicate because I am suggesting a system to fix things rather than just naming what's wrong.