reinforcement learning based adaptive pid controller for a continuous stirred tank heater process

The application of traditional controllers is restricted to the real-time analysis of the nonlinear process due to the need to linearize a nonlinear system. furthermore, tuning poses a significant challenge, especially when dealing with nonlinear systems, as traditional methods often require intricate manual computations to operate under various constraints. the continuous stirred tank heater (csth) process considered for the study has a wide range of operating points and is highly nonlinear. hence, this research aims to pioneer a new approach by leveraging reinforcement learning (rl) to streamline the traditional proportional integral derivative (pid) controller tuning process, adapting to real-time dynamic process demands. the study focuses mainly on temperature control of the csth process, which is renowned for its nonlinear and time-delay characteristics. by employing policy-based rl techniques, specifically twin delayed deep deterministic policy (td3) and soft actor-critic (sac) rl agents with suitable reward functions, the investigation evaluates their adaptability to various set points and resilience to disturbances. through rigorous experimentation and analysis, it is observed that td3 with gaussian reward function performs well compared to sac. the study seeks to demonstrate the performance of td3 rl-based methodologies in simplifying pid tuning by the reduction of performance metrics such as ise, iae, settling time, and overshoot as 47.6%, 26.5%, 3.8%, and 100% for servo response and ise and settling time as 37.7% and 4.7% for the regulatory response than traditional pid controller.