The escalating prevalence of diabetes mellitus presents a pressing global health challenge, with projections indicating a further rise in both type 1 and type 2 diabetes cases. While type 1 diabetes necessitates external insulin administration due to pancreatic insufficiency, type 2 diabetes may also require insulin supplementation, albeit in lesser amounts. Traditional insulin delivery methods, involving multiple daily injections, can be burdensome for patients, particularly those with type 1 diabetes, which requires higher insulin doses. To mitigate these challenges, researchers have developed artificial pancreas systems―a closed-loop approach capable of automatically measuring blood glucose levels and administering insulin. These systems integrate insulin pumps with continuous glucose monitors (CGMs) to maintain glycemic control. However, the effectiveness of these systems heavily depends on the control algorithm employed. While conventional control engineering algorithms have been predominant, recent advancements in artificial intelligence (AI) have led to the exploration of AI-based controllers, particularly those leveraging reinforcement learning (RL). RL-based algorithms, categorized as semi-supervised learning, demonstrate promise in adapting to individual patient dynamics without the need for labeled data. This review investigates the utilization of various mathematical models, such as UVA/PADOVA, Bergman's minimal, and Hovorka models, in conjunction with RL-based algorithms in diabetes management. By examining existing research findings, we elucidate the efficacy of RL-based approaches in patient-specific learning and offer insights into future directions for optimizing diabetes management strategies.