LLM_Light_Testing

LLM_Light_Testing

This project proposes an automated testing framework based on Python for evaluating the inference effectiveness and performance of large language models (LLMs). Key features include:

• User-Friendly: Allows users to test multiple models and prompts with simple configurations.
• Extensibility: Modular design permits easy customization and support for various multimodal models.
• Efficiency and Reliability: Supports parallel processing of multiple prompts and models, enhancing testing speed while providing comprehensive error detection and reporting.
• GPU Monitoring: Integrated GPU utilization monitoring to analyze model performance in real time.

Benefits:

• Streamlined model testing process with standardized output formats.
• Automatic generation of summary tables to facilitate results analysis and comparison.
• Supports a wide variety of models and configurations for comprehensive testing scenarios.

Highlights:

• Easy cloning and setup with minimal requirements.
• Comprehensive documentation and example configurations provided for users.