We investigate the structural evolution of Er/Si nanoclusters obtained in co-implanted fused silica upon annealing via Raman spectroscopy and transmission electron microscopy. The effect of annealing temperature (900–1200 °C) on the nature and the relative fraction of the formed amorphous-Si, Si nanocrystals (Si-nc), and amorphous Er nanoparticles (Er-np) was determined in this ternary Er–Si–O system, showing a change of growth regime above 1100 °C due to the formation of mixed Er/O/Si aggregates. We observe that the nucleation and growth of amorphous Er-np and Si-nc are mutually affected. The 2-fold increase in the size of Er-np when no excess Si+ is present in the matrix indicates that the formation of Si-nc in the proximity of Er clusters hinders Er diffusivity above 1100 °C. This finding shows the importance of nanoclustering for improving the thermal stability of Er-doped silica systems.