Research rather than a screen. This is the classification the study produced.
LLM Code Errors
Two studies at UNIST's Human-AI Interaction and Visualization Lab: where LLM code generation breaks, and how developers comprehend code.
- timeline
- jul 2024 to dec 2024
- role
- software research intern
- lab
- human-ai interaction and visualization lab, unist
- output
- findings published in a conference paper
Classifying how models fail
I analysed and classified LLM code generation errors from SWE-Bench, sorting them into bugs, feature requests, removals, and security issues to inform how the models were evaluated.
The interesting part wasn't the counting. It was that a benchmark score hides the shape of the failure. A model that quietly removes code and a model that writes an insecure patch score the same, and they are not the same problem.
How developers read code
I coded and synthesised 50 developer survey responses on code comprehension, producing findings that were published in a conference paper.
Qualitative coding is unglamorous and it is the same muscle as user research: take 50 people's messy accounts of what they did, and find the structure that is actually in them rather than the one you hoped for.
Why it's here
This is the research half of the same instinct that runs through the rest of my work. Understand the people using the thing before deciding what to build, whether the thing is a vineyard tablet, an analytics dashboard, or a language model writing patches.