This paper presents a reliable in-motion alignment algorithm for a low cost Strapdown Inertial Navigation System/Global Positioning System (SINS/GPS) combination under random misalignment angles, which transforms attitude alignment into an attitude estimation problem. Based on Rodrigues parameters, an alignment model with a linear state-space equation and a second order nonlinear measurement equation is established. Furthermore, by employing a Taylor expansion on the nonlinear measurement equation, we implement a second order Extended Kalman Filter (EKF2). The proposed method uses a single filter that can not only determine the initial attitude, but also estimate the sensor errors. In addition, a scheme is given for avoiding singularity, which makes the algorithm more widely suitable for random misalignment angles. Experimental ground tests are performed with a low-cost Micro-Electromechanical System (MEMS) SINS, which validates the efficacy of the proposed method. The performance compared to the traditional alignment algorithm is also given.