Open science is good for both epistemic and social reasons, but in nonobvious ways, it can have detrimental epistemic side effects. Drawing on case studies and the social epistemology of science, I show how practices intended to increase transparency, communication, and information sharing in science can backfire. We should not reject Open Science, just implement it carefully. I argue that we can do so by treating openness as a governing value in science, and thus, that our pursuit of openness needs to be balanced against our pursuit of the whole scheme of values that govern science.