(108f) An Efficient Reinforcement Algorithm Approach to Optimal Control and Environmental Sustainability

In this paper, we present a new and efficient Reinforcement Learning approach to solving an optimal control problem, based on the Batch Q-learning algorithm. To improve on the convergence of the RL algorithm, we explore the use of k-dimensional uniformity of advanced sampling procedures, namely employing the Halton and Hamersley sequences (HSS). These sequences are used to randomly sample the discrete controls from the action space for the RL optimal control problem. Specifically, the Neural-fitted Q-iterative algorithm is applied to solve an optimal control problem for a first-order state dynamical system. We found that the HSS-based RL improves the convergence significantly as compared to the current RL practice. We implement the schemes in MATLAB and present the simulation results, comparing the optimal policy resulting from using the Pontryagin Maximum Principle with that from using the Neural fitted Q iteration approach. Our results indicate that the RL approach performs reasonably well, especially considering that it is model-free when compared to the objective function value for the deterministic case and error from the optimal trajectory for the stochastic case.

In an age of climate consciousness due to growing environmental concerns, biodiesel has shown enormous promise in replacing traditional fossil fuels like gasoline and diesel. Using optimal control, we aim to optimize the production of biodiesel and improve efficiency to increase its viability as a source of alternative energy. As such, in this paper, we also present a real-world case study involving optimal control problems for biodiesel production, employing the new HSS-RL algorithm.