One of the most challenging area of diabetes research is to provide such automated insulin delivery systems – so called artificial pancreas systems – that have robust and adaptive capabilities in a highly sophysticated way. I.e. they are able to provide robust insulin delivery actions at the beginning of the therapy to satisfy the requirements of the patients without knowing the users daily lifestyle and preferences however adaptive on the short-term to learn these patient specifics to increase the quality of the therapy. One possible solution is the closed-loop systems that have self-learning features. In the present study, we have examined a glucose regulatory problem using direct reinforcement learning based controller. The approach represents the fully automatic insulin administration as the timepoint and the carbohydrate content of the meals were unknown and randomized. We constructed a virtual environment of the patient with type 1 diabetes by applying a mathematical model. Proximal policy optimization learning model with continuous action space was used. Furthermore, we evaluated the effect of different training lengths on the test scenario