Let $f:X\to Y$ be a surjective projective map, and let L be a holomorphic line bundle on X equipped with a (singular) semi-positive Hermitian metric h. In this article, by studying the canonical metric on the direct image sheaf of the twisted relative canonical bundles $K_{X/Y}\otimes L\otimes \mathscr {I}(h)$, we obtain that this metric has dual Nakano semi-positivity when h is smooth and there is no deformation by f and that this metric has locally Nakano semi-positivity in the singular sense when h is singular.