Critical thinking in science and many other disciplines should encompass creative, analytical, practical, and wise thinking. Underlying it are both cognitive processes and dispositions–that is, what a person can do and what a person chooses to do. Critical thinking is both domain-general and domain-specific. The domain-specific elements of it cannot be well captured by general tests of critical thinking. We have found that critical thinking in STEM disciplines involves skills that are quite different from those involved in taking tests of cognitive and academic skills. Some of these skills are generating hypotheses, generating experiments, and drawing conclusions. In our tests of these skills, which we have administered to students at Cornell University, scores on the tests correlated not at all or even negatively with tests of academic preparation, such as the SAT and the ACT. Thus, universities that select future scientists and engineers on the basis of such standardized tests may be choosing the wrong people unless they can assure that those people are good scientific reasoners, not just good takers of analytically-oriented tests.