Sustainability of cereal-based cropping systems remains crucial for food security in South Asia. However, productivity of cereal–cereal rotations has declined in the long run, demonstrating the need for a sustainable alternative. Base crop, that is, common crop in different crop rotations, productivity could be used as a sustainability indicator for the assessment of different long-term crop rotations. This study aimed to evaluate the impact of grain legume inclusion in lowland rice–wheat (R-W) and upland maize–wheat (M-W) rotation on system’s base crop (rice in lowland and wheat in upland crop rotations) productivity and sustainability and also in soil fertility. Mung bean (April–May) inclusion in R-W rotation increased rice grain yield by 10–14%. In upland, mung bean inclusion in M-W rotation increased wheat grain yield by 5–11%. Replacing wheat with chickpea in R-W rotation increased rice grain yield by 5–8%. Increased base crop productivity in legume inclusive rotations was attributed to significant improvement in panicle (rice) or spike (wheat) attributes. Increased soil organic carbon and available nitrogen and phosphorus in the legume inclusive rotations significantly influenced the base crop productivity in both the production systems. Among the crop rotations, R-W-Mb (in lowland) and M-W-Mb (in upland) rotations had the highest system productivity and net return. Therefore, intensification/diversification of cereal–cereal rotations with grain legume could improve soil fertility and sustain crop productivity.