Can AI Code
A self-evaluating interview tool designed specifically for AI coding models. This tool leverages interviews to assess the capabilities of AI coders across various programming languages. Key features include:
- Multi-Language Support: Supports coding evaluations in Python and JavaScript.
- Custom Interview Framework: Allows for the creation of custom prompts and evaluation metrics.
- Quantization Evaluation: Tests the performance of AI models under different quantization schemes, helping developers understand performance impacts.
- Sandbox Environment: Incorporates a Docker-based sandbox for safe execution of evaluated code in real-time.
- User Input Integration: Engages users in contributing to interview questions and evaluation frameworks.
This project aims to accelerate the development of robust AI coders by providing comprehensive benchmarks and evaluations.