A key goal in phonology is to understand the factors that affect phonological learning. This article addresses the issue by examining how paradigms are reanalysed over time. Malagasy has a class of stems called weak stems, whose final consonants alternate under suffixation. Comparison of historical and modern Malagasy shows that weak stem paradigms have undergone extensive reanalysis in a way that cannot be predicted by the probabilistic distribution of alternants. This poses a problem for existing quantitative models of reanalysis, where reanalysis is always towards the most probable alternant. I argue instead that reanalysis in Malagasy is driven by both distributional factors and a markedness bias. To capture the Malagasy pattern, I propose a maximum entropy learning model, with a markedness bias implemented via the model’s prior probability distribution. This biased model successfully predicts the direction of reanalysis in Malagasy, outperforming purely distributional models.