In this paper we study the $\mathbb {C}^*$-fixed points in moduli spaces of Higgs bundles over a compact Riemann surface for a complex semisimple Lie group and its real forms. These fixed points are called Hodge bundles and correspond to complex variations of Hodge structure. We introduce a topological invariant for Hodge bundles that generalizes the Toledo invariant appearing for Hermitian Lie groups. An important result of this paper is a bound on this invariant which generalizes the Milnor–Wood inequality for a Hodge bundle in the Hermitian case, and is analogous to the Arakelov inequalities of classical variations of Hodge structure. When the generalized Toledo invariant is maximal, we establish rigidity results for the associated variations of Hodge structure which generalize known rigidity results for maximal Higgs bundles and their associated maximal representations in the Hermitian case.