We present an optimal gain scheduling control design for bipedal walking with minimum tracking error. We obtained a linear approximation by linearizing the nonlinear hybrid dynamic model about a nominal periodic trajectory. This linearization allows us to identify the linear model as a linear periodic system. An optimal feedback was designed using Bellman's dynamic programming. The linear periodic system allows us to determine a linear quadratic regulator (LQR) for a single period and to set the Hamilton-Jacobi-Bellman (HJB) function in a linear quadratic form. In this way, the dynamic programming yielded an admissible continuous gain scheduling that was designed with regard to the hybrid dynamics of the system. We tuned the optimization parameters such that the tracking error and the average energy consumption are minimized. Due to linearization, we were able to examine the stability of the approximated periodic system achieved by the periodic gain according to Floquet's theory, by calculating the monodromy matrix of the closed-loop hybrid system. In addition to determining stability, the eigenvalues of this approximated monodromy matrix allowed us to evaluate the settling time of the system. This approach presents a direct method for optimal solution of locomotion control according to a given reference trajectory.