In this paper, we show that the presence of numbers in a problem tempts people to perform mathematical operations even when the correct answer requires no math, which we term “mindless math”. In three pre-registered studies across two survey platforms (total N = 3,193), we investigate how mindless math relates to perceived problem difficulty, problem representation, and accuracy. In Study 1, we show that increasing the numeric demands of problems leads to more mindless math (and fewer correct answers). Study 2 shows that this effect is not caused by people being wary of problems that seem too easy. In Study 3, we show that this effect is robust over a wider range of numeric demands, and in the discussion we offer two possible mechanisms that would explain this effect, and the caveat that at even harder levels of numeric demands the effect may invert such that much harder math increases accuracy relative to moderately hard math.