LLM_Light_Testing
This project proposes an automated testing framework based on Python for evaluating the inference effectiveness and performance of large language models (LLMs). Key features include:
- User-Friendly: Allows users to test multiple models and prompts with simple configurations.
- Extensibility: Modular design permits easy customization and support for various multimodal models.
- Efficiency and Reliability: Supports parallel processing of multiple prompts and models, enhancing testing speed while providing comprehensive error detection and reporting.
- GPU Monitoring: Integrated GPU utilization monitoring to analyze model performance in real time.
Benefits:
- Streamlined model testing process with standardized output formats.
- Automatic generation of summary tables to facilitate results analysis and comparison.
- Supports a wide variety of models and configurations for comprehensive testing scenarios.
Highlights:
- Easy cloning and setup with minimal requirements.
- Comprehensive documentation and example configurations provided for users.
