Sign comprehension is critical for effective driving, responses to warnings, and way-finding. Signs that are poorly comprehended by older people increase accident risk and may compromise independence. This study sought to determine whether iconic sign comprehension suffers in healthy aging and in the presence of cognitive impairment. Additionally, we examined whether the addition of text to iconic signage would increase comprehension in older adults. In Experiment 1, young adults, healthy older adults, and older adults with varying levels of cognitive impairment were asked the meaning of 65 signs used for driving, warnings, and way-finding. Healthy older adults were generally good at sign comprehension but had difficulty with way-finding signs. Older adults with cognitive impairment had poorer sign comprehension overall and particular difficulty with way-finding icons and signs that had icons only. In Experiment 2, healthy older adults were asked the meaning of signs containing icons only, or icons and text. A significant improvement in comprehension was found when text was added. An important implication of this work is that the assessment of sign comprehension needs to involve a broad and heterogeneous sample of older adults reflecting the range of perceptual and cognitive abilities represented in the population.