Introduction
The Coding Tutor project explores the potential of Large Language Models (LLMs) as coding tutors. It introduces the Trace-and-Verify (Traver) agent workflow, which incorporates knowledge tracing and turn-by-turn verification to address challenges in coding tutoring. This work not only focuses on coding but also extends to other task-tutoring scenarios, adapting content to users' varying levels of background knowledge.
Key Features
- Traver Agent Workflow: An effective workflow that enhances coding tutoring through verification processes.
- Dialogue for Coding Tutoring (DICT): A novel evaluation protocol that combines student simulation and coding tests to assess tutor performance.
- Scalability and Cost-Effectiveness: The DICT protocol serves as a feasible proxy for human evaluation, making it easier to evaluate tutor agents systematically.
Benefits
- Automated Evaluation: Supports a systematic development and evaluation cycle for task-tutoring agents.
- Inference-Time Scaling: Demonstrates how the trained verifier can scale during inference for coding tutoring.
Highlights
- The repository includes released data and evaluation results, along with scripts for training and evaluation, making it a valuable resource for researchers and developers in the field of AI-driven tutoring.
