Why YSK? Comments you reply positively to, should definitely get your upvote. Comments you disagree with should be at your discretion. Trolls deserve downvotes (seriously, they live for those). Disagreeing with someone in the midst of a good discussion doesn’t necessarily warrant one, and might deserve an upvote. Even if you don’t reply, but you agree with the comment, give it your vote.
Also, this has nothing to do with propping up folks’ egos. Comments with more upvotes will likely be seen first the longer the post is up. Alternatively, downvoted posts are less likely to be seen unless users are looking for them.
Of course, this can lead to folks accusing communities of having leftist/right-wing bias, but I think overall it improves the usage of the site. Personally, the thing I liked most about Reddit was the conversations in the comments. Usually the ones with the most upvotes were more worth the read & engagement.
As a former lurker on Reddit, I’m forcing myself to give votes to everything I read here, comments or posts. It is getting easier and easier once I make it a habit. I’m also trying to comment a lot (like this one), even though I never did before. Commenting more actually is quite enjoyable as I finally feel like I’m more part of the discussions instead of an observer.