Coding Tutor

Introduction

The Coding Tutor project explores the potential of Large Language Models (LLMs) as coding tutors. It introduces the Trace-and-Verify (Traver) agent workflow, which incorporates knowledge tracing and turn-by-turn verification to address challenges in coding tutoring. This work not only focuses on coding but also extends to other task-tutoring scenarios, adapting content to users' varying levels of background knowledge.

Key Features

• Traver Agent Workflow: An effective workflow that enhances coding tutoring through verification processes.
• Dialogue for Coding Tutoring (DICT): A novel evaluation protocol that combines student simulation and coding tests to assess tutor performance.
• Scalability and Cost-Effectiveness: The DICT protocol serves as a feasible proxy for human evaluation, making it easier to evaluate tutor agents systematically.

Benefits

• Automated Evaluation: Supports a systematic development and evaluation cycle for task-tutoring agents.
• Inference-Time Scaling: Demonstrates how the trained verifier can scale during inference for coding tutoring.

Highlights

• The repository includes released data and evaluation results, along with scripts for training and evaluation, making it a valuable resource for researchers and developers in the field of AI-driven tutoring.