The three downvoters look to be serial downvoters. One has a large history but seems to downvote about twice as much as he upvotes. The other two have almost no comments or posts and downvote almost as much (ratio) as the other guy.

Lemvotes.org is your friend so you can see who is being a jackass. It’s really only a few dickhead accounts that downvote so don’t worry too much about.