We study a general class of interacting particle systems called kinetically constrained models (KCM) in two dimensions tightly linked to the monotone cellular automata called bootstrap percolation. There are three classes of such models, the most studied being the critical one. In a recent series of works by Martinelli, Morris, Toninelli and the authors, it was shown that the KCM counterparts of critical bootstrap percolation models with the same properties split into two classes with different behaviour. Together with the companion paper by the first author, our work determines the logarithm of the infection time up to a constant factor for all critical KCM, which were previously known only up to logarithmic corrections. This improves all previous results except for the Duarte-KCM, for which we give a new proof of the best result known. We establish that on this level of precision critical KCM have to be classified into seven categories instead of the two in bootstrap percolation. In the present work, we establish lower bounds for critical KCM in a unified way, also recovering the universality result of Toninelli and the authors and the Duarte model result of Martinelli, Toninelli and the second author.