Morzsák

Oldal címe

LLM evaluation on APR task part 2

Címlapos tartalom

Automatic Program Repair requires understanding the source code in order to fix bugs. Understanding the source code LLMs can fix bugs according to related works, although these works are mostly evaluated on either hand-crafted small data or on data which is possibly was available during the training / fine-tuning steps. In this project we aim to evaluate LLMs in real-life vulnerabilities, which require more understanding of the context. Part2 is about evaluating smaller models or quantized LLMs.