Many elliptical galaxies possess an appreciable amount of X-ray-emitting hot plasma, providing a harsh interstellar environment for the survival of dust grains and polycyclic aromatic hydrocarbons (PAHs). Despite such a hostile environment, it has been found that a significant fraction of X-ray elliptical galaxies contain a considerable amount of dust, which cannot be explained solely from replenishment by old stars. Some of them even show the presence of PAHs. We present the results of AKARI and Spitzer observations of dust and PAHs in X-ray elliptical galaxies. We investigate their possible origins and discuss the implications of their presence for the evolution of elliptical galaxies.