Proceedings of the 2019 EMNLP and the 9th IJCNLP (System Demonstrations), pages 97–102Hong Kong, China, November 3 – 7, 2019. c©2019 Association for Computational Linguistics

97

IFlyLegal: A Chinese Legal System for Consultation, Law Searching, andDocument Analysis

Ziyue Wang†, Baoxin Wang†, Xingyi Duan†, Dayong Wu†, Shijin Wang†§, Guoping Hu†, Ting liu‡

†State Key Laboratory of Cognitive Intelligence, iFLYTEK Research, Beijing, China‡Research Center for Social Computing and Information Retrieval,

Harbin Institute of Technology, Harbin, China§iFLYTEK AI Research (Hebei), LangFang, China

†{zywang27,bxwang2,xyduan,dywu2,sjwang3,gphu}@iflytek.com‡tliu@ir.hit.edu.cn

Abstract

Legal Tech is developed to help people with le-gal services and solve legal problems via ma-chines. To achieve this, one of the key require-ments for machines is to utilize legal knowl-edge and comprehend legal context. Thiscan be fulfilled by natural language processing(NLP) techniques, for instance, text represen-tation, text categorization, question answering(QA) and natural language inference, etc. Tothis end, we introduce a freely available Chi-nese Legal Tech system (IFlyLegal) that bene-fits from multiple NLP tasks. It is an integratedsystem that performs legal consulting, multi-way law searching, and legal document analy-sis by exploiting techniques such as deep con-textual representations and various attentionmechanisms. To our knowledge, IFlyLegal isthe first Chinese legal system that employs up-to-date NLP techniques and caters for needs ofdifferent user groups, such as lawyers, judges,procurators, and clients. Since Jan, 2019, wehave gathered 2,349 users and 28,238 pageviews (till June, 23, 2019).

1 Introduction

The term Legal Tech refers to legal technologiesthat apply computer technologies to legal services,such as legal consultation and judicial documentanalysis. Such techniques are able to ease the loadof legal workers and provide easily accessible ser-vices for clients. Recently, researchers are concen-trating on enhancing Legal Tech with NLP tech-niques (e.g., named entity recognition (Yin et al.,2018), sequence labeling (Yan et al., 2018)). Stud-ies have proven that NLP techniques are markedlyeffective regarding several legal tasks, for in-stance, charge prediction (Hu et al., 2018) and lawarea classification (Sulea et al., 2017).

In industry, the majority of legal consultationproducts are merely platforms that redirect users toactual lawyers other than solving problems with a

Figure 1: Architecture of the legal system

compact and efficient intelligent system. The par-tially automated ones, however, tend to have singlefunctionality, either document analysis like casedescription analysis, or information acquiring likelaw searching and legal consultancy. Despite ofplentiful advances in NLP, most of industrial ap-plications remain constructed in conventional in-formation retrieval (IR) manner or highly rely onhand-crafted responses. They do not take advan-tage of the most advanced algorithms in NLP anddeep learning and fail to cater for the followingneeds: (i) flexibility: being flexible to compre-hend various forms of questions or queries; (ii)diversity: being able to generate different and cus-tomized replies according to slight changes of in-put queries; (iii) accuracy: being able to deliver re-sponses that correctly answer the input questions.

In this paper, we present an integrated systemthat adapts functionalities such as consultation anddocument analysis to legal context. Taking ad-vantage of recent advances in NLP, the legal sys-tem can act as an artificial lawyer for clients andas an assistant for legal workers. It is a practi-cal system for answering legal questions, perform-ing law searching in multiple modes, and analyz-ing case descriptions. First, by coupling question-answering and scoring models with external le-gal knowledge such as statutes and legal com-monsense, we design an architecture especially for

98

Figure 2: An illustration of the legal system

providing professional solutions to legal issues.Second, inspired by natural language inferencetask, we build a dedicated module to solving prob-lems with statutes only, namely natural languagearticle inference. Last but not least, we constructan analysis module to comprehend cases, predictjudicial sentences, and retrieve similar cases.

The main contributions of this work lie in thefollowings: 1. It integrates the multiple legal ser-vices, consultation, law searching and documentanalysis, into a single application; 2. We propose anew task, natural language article inference, whichreplies to legal questions with a sets of possiblearticles of law only; 3. All the utilized modelsachieve practicable results.

2 Related Work

The current automated legal consultation applica-tions usually rely on retrieving relevant text in-formation from pre-constructed database contain-ing legal question-answer pairs using text featuressuch as TF-IDF and bag-of-words (BoW). Doet al. (2017) proposed QA models for legal con-sultation, and Hang (2017) preformed legal ques-tion classification with deep convolutional neuralnetwork trained in multi-task manner.

Law searching is an unneglectable demand oflegal workers such as lawyers and procurators,as they need to support their views with suffi-cient articles. In industry, article retrieval applica-tions simply parse inputs into phrases and adoptcommon IR approaches. Academically, Zhanget al. (2017) built a Chinese legal consultation sys-

tem to improve the precision of retrieving articlesand predicting sentences by exploiting legal prece-dents when performing logical reasoning.

Legal document analysis is frequently viewedas text representation and classification task. Huet al. (2018) introduces few-shot attributes to en-rich the information of mapping from case de-scriptions to charges, and Sulea et al. (2017) usedmultiple SVM classifiers as an ensemble to per-form law area classification.

3 Chinese Legal-tech System

The presented system consists of the followingblocks: consultation, law searching and case anal-ysis. We will go into these blocks at length in thissection. Throughout the system, the LTP toolkit(Che et al., 2010) is employed for Chinese wordsegmentation, named entity recognition and se-mantic parsing tasks. The overall system archi-tecture is depicted in Figure 1. As this paper fo-cuses on legal context processing, we will providedetails for principal modules related to legal ser-vices and omit the others like chit-chat. For betterunderstanding, we will discuss experiments, userstudies and use cases in the following sections.

3.1 Consultation Block

This block is responsible for legal QA. It con-tains four modules, one of which is called in-tention recognition and natural language under-standing (NLU) module that filters out chit-chats,recognizes intentions of inputs, and analyzes thequeries. The other three, namely general le-

99

gal consulting, legal term explaining, and lawyerrecommending, are subsequent modules that re-sponse to legal consultation concerning the rec-ognized intention. This block accepts short ques-tions appealing for legal support and outputs lit-eral replies to the questions. We extract textcontaining legal issues from public corpus (e.g.,Wikipedia), and automatically collect web textfrom Chinese legal forums and online communi-ties for pre-training language models.

Intention Recognition and NLU module is thebasis of consultation block. It functions as thecombination of a gate and a converter. As we fo-cus on legal scope, this module only admits legalrelated content, meanwhile it rejects and redirectsthe rest to an external chit-chat module. This isachieved by a binary classifier that assigns label“chat” or “legal” to each input. Then, the admittedinputs are analyzed and rewritten for consultationusing pre-trained models and predefined features.

General Legal Consulting is an indispensablemodule for a legal aid system. Following the ideaof general QA system (Quaresma and Rodrigues,2005), we trained an end-to-end QA model es-pecially for legal consulting using data collectedfrom online forums and communities. Note thata legal consultation system should give neutralreplies, biased comments are thoroughly removed.This module roles as a virtual lawyer who analyzequeries and response with appropriate answers.

Furthermore, the consultation block has twocomplementary modules called legal Term expla-nation and lawyer recommendation that compen-sate the general legal consulting module with ex-tra useful information. If a legal term appears byitself as a query, it will be detected by intentionrecognition module and fed into legal term expla-nation module for detail descriptions. Lawyer rec-ommendation module is personalized with regardsto users’ preferred features, for example, lawyer’slocation and statistical winning percentage.

3.2 Law Searching Block

Aiming to work as a legal assistant, this block per-forms law searching in three manners. One isquery-based law searching that follows the ideaof standard IR approaches. Another is document-based law searching that reads long documentsand retrieves applicable articles. The last is a noveltask, natural language based law searching. Wewill discuss it in detail in the next paragraph.

0

500

1000

1500

2000

04/22-04/28 05/06-05/12 05/20-05/26 06/03-06/09 06/17-06/23

counts

(a) Total Page Views (per week).

0

100

200

300

400

500

04/22-04/28 05/06-05/12 05/20-05/26 06/03-06/09 06/17-06/23

seconds

(b) Average Viewing Time (per week).

Figure 3: User studies on cumulative page views (a)and average viewing time (b) for the recent two months(from April 22 to June 23)

Article inference is a natural language basedlaw searching approach without formatting thequeries. It is a challenging task on account thatthe number of related statutes to a given ques-tion sometimes remains unclear even if answeredby experienced lawyers, and that there are differ-ent laws and regulations stating the same fact butfocusing on different aspects. Undoubtedly, wecan simply pair up the input question with all thestatutes and perform one-way sentence matching,i.e. deducing articles from a question. Neverthe-less, this will result in tremendous amount of sen-tence pairs due to countless articles of various lev-els and from different provinces and cities, whichmakes it impossible to response immediately.

To address this issue, we need to generate a setof candidate statutes to narrow down the searchingspace and improve the predicting accuracy. Wepart the article inference task into two consecutivephases. Firstly, we train a law-level classifier thatcategorizes inputs into 267 in-force Chinese lawsand extract articles from the top 3 resulting lawsas a coarse candidate set. The total number of thisset could go beyond a thousand since some lawscontain over 400 articles. So far, the candidatespace is still too large to achieve instant response.We use a bidirectional LSTM architecture employ-ing four perspective features from (Wang et al.,2017) as an intermediate article inference model inarticle-level to obtain a fine-grained candidate set.Secondly, we slightly adapt the BERT model (De-vlin et al., 2018) so that it becomes more sensitive

100

Metrics p@1(%) p@3(%)Automatic 42.67 23.56Human (Partial) 90.00 75.56Human (Exact) 76.67 52.22

Table 1: The final test results of article inference task.Human evaluation is contains three level, not match,partially match and exact match.

Models p% r% f%vanilla RNN 80.11 80.81 80.64BiMPM 87.05 84.82 85.92BERT 90.57 87.76 89.14

Table 2: The results of article inference (Test set posi-tive:negative=1:4). BiMPM uses 4 perspective featuresfrom Wang et al. (2017). BERT here is adapted fromDevlin et al. (2018).

to legal context. More concretely, the BERT-baseChinese model is trained for another 300,000 stepswith 20% of all the question-article pairs. Then,the model is fine-tuned for final article inferencetask with the training data that has 1 correct articleout of 5 on average. In practice, we feed the fine-grained candidate set obtained from phase one tothe fine-tuned BERT and get the inference results.Eventually, 3 most probable articles are displayedand mapped to the names of laws accordingly.

Query-based law searching provides retrievalsto queries asking for certain laws or as detailedas certain statutes, which is the basic functionfor a law searching application. It is achievedby retrieving a set of articles with IR system andweighting and ranking by multiple features liketextual similarity, priority and validity of laws.Document-based law searching returns articleswith respect to a piece of case description. It readsand analyzes case description and returns relatedarticles ordered by relevance to the description. Inpractice, it can act as a fast candidate article poolfor a particular case in court trials. The differencebetween this module and article inference lies inthe inputs where document-based searching dealswith massive formal text written by law expertswhile article inference tackles arbitrary short oralquestions coming from daily life.

3.3 Case Analysis Block

Case description is an essential component of ajudicial document. It states the facts involved ina case, including sequence of events, people pre-senting during the event, consequences, etc. Thisblock reads and comprehends case descriptions

Functional Modules Accuracy%Civil+criminal article prediction 78.0Civil/criminal classification 95.5Criminal accusation prediction 98.2Civil cause prediction 84.3

Table 3: The results of analysis modules.

and reports the analysis results in different formatsincluding statistical graphs of sentences, similarcases, relevant statutes and recommended lawyers.

Civil/Criminal Classification is a preliminarytask of case analyzing, since the judging crite-ria and sentences varies with categories of cases.Cases are generally divided into two classes, civiland criminal. A civil case happens between cit-izens and the penalty usually excludes the termof imprisonment, while in a criminal case the de-fendant is prosecuted by public prosecution organand would be sent to jail if proven guilty. This isa binary classification task fulfilled by pre-trainedword vectors and 1-layer convolutional neural net-work using case description data.

Case Analysis involves civil case analysis andcriminal case analysis. The results of both cat-egories contains lists of similar cases, relevantarticles and recommended lawyers together withtheir professional history. For sentences predic-tion, civil case analysis outputs the possibilities ofaccuser/defendant winning the lawsuit. The pre-diction section of criminal case is also known asautomatic sentencing that models the sentencingresults given a paragraph of criminal case descrip-tion. The sentences contain the predicted accusa-tion, term of imprisonment and the legal grounds,i.e. the articles of criminal law. We adopt the dis-connected recurrent neural network described inWang (2018) for automatic sentencing task.

4 Experiments and User Study

We conduct experiments on all modules and willreport some important results in this section. Userstudies on cumulative page views and averageviewing times are presented in figure 3. For theconsultation block, the testset contains 2000 ques-tions with varying legal topics. The correspondingreplies are automatically obtained by the system.The retrievals are manually scored between 1 to 5,where 1 represents irrelevant answers and 5 standsfor the best matches. The system achieves an av-erage precision of 80% for the top 1 retrievals.

We compare the results of article inference task

101

Figure 4: Use case 1: use case of consultation block.

regarding different models and list them in Ta-ble 2. Statistics show that the vanilla RNN is ableto properly predict the article from the given can-didates, but is much inferior to the other compli-cated models. Yet, it is not the final result forthe task in practice due to the inconsistencies be-tween the distributions of experimental and realdata. The manually created dataset has one correctarticle out of 5 candidates on average. In reality,surprisingly, it almost equals to pick up the onlyone correct answer out of 600, making it an ex-tremely challenging task. We evaluate the resultsof 200 arbitrary questions by precision@n in twoways, automatic and manual, and report in Table1. Regardless of disappointing automatically eval-uated results, manual evaluation reveals the practi-cability of the models and proves that they are ableto figure out most of the answers.

The testset for the analysis block is automati-cally extracted from the original court documents,where categories are indicated in the titles and in-formation like the relevant articles and sentencesare always listed at the end. We evaluate the per-formance of analysis block with respects to thefollowing tasks: (i) civil and criminal article pre-diction; (ii) binary classification of civil and crim-inal cases; (iii) accusation prediction for crimi-nal cases and (iv) cause prediction for civil cases,which is the counterpart of accusation as to crimi-nal cases. The results are listed in Table 3.

5 Use Cases

In this section, we will present cases illustratingthe three blocks, consultation, law searching and

Figure 5: Use case 2: use case of law searching block.

case analysis. Also, we encourage readers to tryIFlyLegal via scanning the QR code in figure 7.

Figure 4 is a use case of the consultation block.The inputs can be arbitrary questions as long asthey contain legal issues, such as “How to dividecommon property after divorce?” and “Is temporalworker a legal labor relation?”. A proper answerwill be delivered along with a set of applicablestatutes. Figure 5 displays part of the law search-ing results for the tested input “Criminal Law Arti-cle 200”. It will be recognized as “Criminal Law”plus a further restriction on the number of article,“Article 200”. Figure 6 demonstrates the resultsof case analysis block with a screen-shot of thepredicted sentences. The other information can befound in the screencast.

6 Conclusion and Future works

We present a system called iFlyLegal for auto-mated legal QA, multi-way law searching, andmulti-perspective legal document analysis. Thesystem is built upon a combination of classicaltext features and deep learning techniques in NLP.We conduct sufficient experiments and report im-portant results in this paper. To help understandour system, we illustrate several use cases withsnapshots and necessary literal explanation. Thesecases also prove that our system is capable to ful-fill user demands.

Taking into consideration the need for easeof access, the system is demonstrated in formof WeChat Mini Program, which is compact,portable and freely available. Yet we will developa web-based version for those who prefer access-

102

(a) Input Interface. (b) Predicted Sentences.

Figure 6: Use Case 3: use case of analysis block. (a)is the case description. (b) present the predicted sen-tences.

Figure 7: Mini program QR code of IFlyLegal (scan-ning via WeChat). We also provide a screencast forreaders who do not have a Wechat.

ing via computers. During maintaining, the mod-els and topology behind would be improved alongwith our researches. Although neural network isregarded as “black box”, we are currently work-ing on the explainability of our models and try-ing to present users with evidences to the model-generated outputs so as to be convincing. IFlyLe-gal is an integrated and multi-functional system,whose build-in modules are complex and can beseparated and adapted for different tasks such astext categorization and natural language inference.We intend to turn iFlyLegal into an NLP plat-form for legal AI research to cater for the boostingneeds of NLP techniques in legal industrial and ex-plore other valuable research topics of legal NLP.In the future, we will investigate into adapting theNLP research platform to other languages.

Acknowledgments

The authors would like to thank all the review-ers for their insight reviews. This paper isfunded by National Key R&D Program of China

(No.2018YFC0807701).

ReferencesWanxiang Che, Zhenghua Li, and Ting Liu. 2010. Ltp:

A chinese language technology platform. In Pro-ceedings of the 23rd International Conference onComputational Linguistics: Demonstrations, pages13–16. Association for Computational Linguistics.

Jacob Devlin, Ming-Wei Chang, Kenton Lee, andKristina Toutanova. 2018. Bert: Pre-training of deepbidirectional transformers for language understand-ing. arXiv preprint arXiv:1810.04805.

Phong-Khac Do, Huy-Tien Nguyen, Chien-Xuan Tran,Minh-Tien Nguyen, and Minh-Le Nguyen. 2017.Legal question answering using ranking svm anddeep convolutional neural network. arXiv preprintarXiv:1703.05320.

Nga Tran Anh Hang. 2017. Applying deep neural net-work to retrieve relevant civil law articles. In Pro-ceedings of the Student Research Workshop associ-ated with RANLP, pages 46–48.

Zikun Hu, Xiang Li, Cunchao Tu, Zhiyuan Liu, andMaosong Sun. 2018. Few-shot charge predictionwith discriminative legal attributes. In Proceedingsof the 27th International Conference on Computa-tional Linguistics, pages 487–498.

Paulo Quaresma and Irene Pimenta Rodrigues. 2005.A question answer system for legal information re-trieval. In JURIX, pages 91–100.

Octavia-Maria Sulea, Marcos Zampieri, Shervin Mal-masi, Mihaela Vela, Liviu P Dinu, and Josef vanGenabith. 2017. Exploring the use of text clas-sification in the legal domain. arXiv preprintarXiv:1710.09306.

Baoxin Wang. 2018. Disconnected recurrent neuralnetworks for text categorization. In Proceedings ofthe 56th Annual Meeting of the Association for Com-putational Linguistics (Volume 1: Long Papers),volume 1, pages 2311–2320.

Zhiguo Wang, Wael Hamza, and Radu Florian. 2017.Bilateral multi-perspective matching for natural lan-guage sentences. arXiv preprint arXiv:1702.03814.

Yukun Yan, Daqi Zheng, Zhengdong Lu, and SenSong. 2018. Zooming network. arXiv preprintarXiv:1810.02114.

Xiaoxiao Yin, Daqi Zheng, Zhengdong Lu, andRuifang Liu. 2018. Neural entity reasonerfor global consistency in ner. arXiv preprintarXiv:1810.00347.

Ni Zhang, Yi-Fei Pu, Sui-Quan Yang, Ji-Liu Zhou, andJin-Kang Gao. 2017. An ontological chinese legalconsultation system. IEEE Access, 5:18250–18261.