This paper presents Hybrid Modified A* (HMA*) algorithm which is used to control an omnidirectional mecanum wheel automated guided vehicle (AGV). HMA* employs Modified A* and PSO to determine the best AGV path. The HMA* overcomes the A* technique’s drawbacks, including a large number of nodes, imprecise trajectories, long calculation times, and expensive path initialization. Repetitive point removal refines Modified A*’s path to locate more important nodes. Real-time hardware control experiments and extensive simulations using Matlab software prove the HMA* technique works well. To evaluate the practicability and efficiency of HMA* in route planning and control for AGVs, various algorithms are introduced like A*, Probabilistic Roadmap (PRM), Rapidly-exploring Random Tree (RRT), and bidirectional RRT (Bi-RRT). Simulations and real-time testing show that HMA* path planning algorithm reduces AGV running time and path length compared to the other algorithms. The HMA* algorithm shows promising results, providing an enhancement and outperforming A*, PRM, RRT, and Bi-RRT in the average length of the path by 12.08%, 10.26%, 7.82%, and 4.69%, and in average motion time by 21.88%, 14.84%, 12.62%, and 8.23%, respectively. With an average deviation of 4.34% in path length and 3% in motion time between simulation and experiments, HMA* closely approximates real-world conditions. Thus, the proposed HMA* algorithm is ideal for omnidirectional mecanum wheel AGV’s static as well as dynamic movements, making it a reliable and efficient alternative for sophisticated AGV control systems.