can-ai-code

Can AI Code

A self-evaluating interview tool designed specifically for AI coding models. This tool leverages interviews to assess the capabilities of AI coders across various programming languages. Key features include:

• Multi-Language Support: Supports coding evaluations in Python and JavaScript.
• Custom Interview Framework: Allows for the creation of custom prompts and evaluation metrics.
• Quantization Evaluation: Tests the performance of AI models under different quantization schemes, helping developers understand performance impacts.
• Sandbox Environment: Incorporates a Docker-based sandbox for safe execution of evaluated code in real-time.
• User Input Integration: Engages users in contributing to interview questions and evaluation frameworks.

This project aims to accelerate the development of robust AI coders by providing comprehensive benchmarks and evaluations.