The need for better-performing algorithms to solve real-world power systems engineering problems has always been a challenging topic. Due to their stochastic nature, metaheuristic algorithms can provide better results. Thus, they have a rising trend in terms of investigation. This paper is a further attempt to offer a better optimizing structure, therefore, aims to provide a better-performing algorithm both for designing an appropriate proportional-integral-derivative (PID) controller to effectively operate an automatic voltage regulator (AVR) system and extracting the optimum parameters of a power system stabilizer (PSS) employed in a single-machine infinite-bus (SMIB) power system. Therefore, the paper discusses the development of the Lévy flight-based reptile search algorithm with local search capability and evaluates its potential against challenging power systems engineering optimization problems. The Lévy flight concept is used for better exploration capability in the proposed algorithm, whereas the Nelder-Mead simplex search algorithm is integrated for further exploitation. The latter case is confirmed through 23 benchmark functions with different features using statistical and nonparametric tests. The superiority of the proposed Lévy flight-based reptile search and Nelder-Mead (L-RSANM) algorithm-based PID controller for the AVR system is demonstrated comparatively using convergence, statistical and nonparametric tests along with transient and frequency responses. Besides, it is also assessed against previously reported and different methods, showing further superiority for AVR system control. Furthermore, the extraordinary ability of the L-RSANM algorithm to design an efficient PSS employed in the SMIB power system is demonstrated, as well. In conclusion, the proposed L-RSANM algorithm is shown to be more capable to solve the challenging power systems engineering design problems.