If there are no other comments and you're going to down vote the question/answer, it's polite and useful to post a comment. The FAQ specifically asks you to do so. It's part of the etiquette of the site. Without the comment, the down vote is fairly worthless.
I believe someone asking for a comment to explain a down vote is the same as asking someone to follow the rules of the site.
For example my answer here, currently has two down votes. I have no idea why it's being down voted. How can I improve my answer if I don't know what's wrong? Without knowing how to improve my answer, what's the point of the down vote? I can only assume there's nothing wrong with the answer, and maybe some people just don't like me :)
Matsemann, to address your comments:
I didn't say the rules say you "always must" provide an explanation, I said they specifically "ask" you to do so, and they do: "Add comments indicating what, specifically, is wrong". While my statement in bold is a statement, it's a attitudinal statement (it says "I believe..."). Adding a comment for a down vote is as close to a rule as anything else is in the FAQ. It says to add comments saying what's wrong, I don't know how it can get clearer than that. Just because it's not enforced automatically through the system does not make it less of a rule. Being polite shouldn't be enforced. You should want to do that.
I assume you're referring to the "people don't like me" sentence. I don't take down votes personally. So don't worry, I'm not having a bad time. The smiley face at the end of that sentence indicates humor. See the FAQ: "Bring your sense of humor." ;)
That's your choice. It's a grey area deciding between a down vote and no vote. Since the down vote text says "This answer is not useful", those are typically the answers I reserve the down vote for. If an answer is simply not as good as another answer, I just don't vote on it at all. Though If I were to down vote, since the FAQ asks me to comment, I do. I don't think of it as "tailoring a huge reasoning", just a simple comment will usually do the trick. It doesn't take that much brain power to state why I think an answer is bad. It's also the polite thing to do. We don't have to be anonymous jerks just because the system allows it. Be a known jerk like me :)