Evidence shows that the willingness of individuals to avenge punishment inflicted upon them for transgressions they committed constitutes a significant obstacle toward upholding social norms and cooperation. The drivers of this behavior, however, are not well understood. We hypothesize that ulterior motive attribution—the tendency to assign ulterior motives to punishers for their actions—increases the likelihood of counter-punishment. We exogenously manipulate the ability to attribute ulterior motives to punishers by having the punisher be either an unaffected third party or a second party who, as the victim of a transgression, may be driven to punish by a desire to take revenge. We show that survey respondents consider second-party punishment to be substantially more likely to be driven by ulterior motives than an identical, payoff-equalizing punishment meted out by a third party. In line with our hypothesis, we find that second-party punishment is 66.3% more likely to trigger counter-punishment than third-party punishment in a lab experiment. The loss in earnings due to counter-punishment is 64.6% higher for second-party punishers than third-party punishers, all else equal.