1. Small-scale datasets will be provided in the preliminary competition stage. Contestants will be considered qualified for the secondary competition stage only when the effect of the small-scale dataset they can achieve exceeds the official baseline.
Evaluation competitions hosted within China:
Challenge of AI in Law（CAIL）
Led by Department of Computer Science and Technology, and has been held four times so far
Initiator: Zhiyuan Liu, Yiqun Liu
............... CAIL 2022 Registration ...............
CAIL 2022 has set up a total of eight tasks, namely: judicial examination, event detection, judgment document proofreading, relevant legal case retrieval, judicial summary, argument understanding, information extraction, and interpretable similar case matching. At the same time, we will also provide dataset with a large amount of judicial document. CAIL 2022 is expected to be held in Beijing in January 2023 with an award ceremony and a workshop on legal intelligence technology. We sincerely invite researchers and developers from academia and industry to actively participate and provide support!
............... Description of the CAIL 2022 competition...............
Each part of the CAIL 2022 competition will be organized based on the three-stage competition system, with the three stages comprising the preliminary competition stage, the secondary competition stage, and the closed beta test stage, respectively. The competition system in the three stages is designed to prevent the overfitting of models that would result from excessive parameter tuning, so as to guarantee a higher score of the models with high generalization performance as much as possible. The final scores of the contestants will be given according to their comprehensive performance in the three stages.
-
-
2. Full-range training datasets will be provided in the second competition stage to help the contestants adjust and optimize the models in a better manner.
-
3. The contestants are required to provide their models in the closed beta test stage, during which the generalization of the models should be evaluated using the new test sets.
............... CAIL 2022 Evaluation Tasks ...............
Judicial
Examination
Examination
Judicial examination: The models are required to output correct answers based on the questions and options listed in the judicial examination
Event
detection
detection
The models are required to identify event triggers and their corresponding event types in legal cases
Document
proofreading
proofreading
The models are required to deliver assistance to judicial practitioners for automatic detection and correction of errors in legal instruments.
Similar case search
For Chinese criminal cases, similar cases related to them are screened from the pool of candidate cases according to the query cases.
Judicial
summarization
summarization
The models are required to integrate multi-source and multi-document information for summarization and simplification to improve the level of document summarization
Debate comprehension
It is targeted at the debate analysis of the texts expressed by both the charge and the defense in judgment documents, during the process of which the models are required to automatically identify the controversial viewpoints of both the charge and the defense in judgment documents
Information extraction
The models are required to identify key judicial entities and their relationships in texts based on case descriptions and to return the results in a specified format
Interpretable similar case matching
The models are required to judge whether a case is matched with the requirements and to provide supporting evidence.