JailBench

JailBench: A Comprehensive Chinese Security Assessment Benchmark for Large Language Models

JailBench is a large-scale dataset designed to evaluate the jailbreak attack risks of large language models in the Chinese context. It is aligned with the national cybersecurity standards and aims to provide a thorough assessment of the security vulnerabilities in AI-generated content.

Key Features:

• Extensive Dataset: Contains 10,800 test questions specifically designed to evaluate the jailbreak capabilities of large language models.
• Multi-Domain Coverage: The dataset encompasses five primary domains and 40 subdomains, ensuring comprehensive evaluation across various fields.
• Security Assessment: Provides a robust framework for assessing the security performance of large language models against jailbreak attacks.
• Research Contribution: Open-source access to the dataset encourages further research and development in AI security.

Benefits:

• Enhanced Security Testing: Facilitates the identification and mitigation of potential security risks in AI models.
• Standardized Evaluation: Aligns with national standards for cybersecurity, ensuring relevance and applicability in real-world scenarios.
• Community Collaboration: Encourages contributions and feedback from the research community to improve the dataset and its applications.

Highlights:

• Recognized at PAKDD 2025, showcasing its significance in the field of AI security research.
• Continuous updates and improvements based on community feedback and advancements in AI technology.