ChatLaw: Chinese Legal Model

ChatLaw: Chinese Legal Model

Under the wave of ChatGPT, the continuous expansion and development of artificial intelligence have provided fertile soil for the spread of LLM. Currently, the fields of healthcare, education, and finance have gradually developed their own models, but there has been no significant progress in the legal field. In order to promote open research on the application of LLM in law and other vertical fields, this project has open-source the Chinese legal model and provided a reasonable solution for the combination of LLM and knowledge base in legal scenarios. The current open source versions of ChatLaw legal model for academic reference are Jiangziya-13B and Anima-33B. We use a large amount of original texts such as legal news, legal forums, laws, judicial interpretations, legal consultations, legal exam questions, and judgment documents to construct dialogue data. The model based on Jiangziya-13B is the first version of the model. Thanks to Jiang Ziya's excellent Chinese language ability and our strict requirements for data cleaning and data augmentation processes, we perform well in logically simple legal tasks, but often perform poorly in complex logical legal reasoning tasks. Subsequently, based on Anima-33B, we added training data and created ChatLaw-33B, which showed a significant improvement in logical reasoning ability. Therefore, it can be seen that large parameter Chinese LLM is crucial. Our technical report is here: arXiv: ChatLaw The version trained based on commercially available models will be used as the internal integration version for our subsequent products and is not open source to the outside world. You can try out the open source version of the model here

Tag: law

Reading: 55 2023-07-23

Chineasy -Simple learning Chinese

Chineasy -Simple learning Chinese

What is chineasy?Author: Xue Xiaolan (transliteration) As a daughter of a calligrapher who grew up in Taiwan, my earliest and most precious memory was my mother showing me the beauty, shape and form of Chinese characters to me.Since then, I have been deeply attracted by the structure of this incredible language. But in the eyes of outsiders, it must be as indestructible as the Great Wall of China.Over the years, I often want to know if I can find a way to break this wall, so anyone who wants to understand and appreciate this charming language can do this through their own eyes. Twelve years ago, I moved to the United Kingdom and registered for enrollment at the University of Cambridge.Two years later, I had a degree and two children.When I started a new life, I observed how popular China is, how much people are eager to accept this culture -but they are struggling in language.Even my own children think this is awesome. Since then, I have begun to think about how a new and simpler way to read Chinese is useful.During the day, I am an Internet entrepreneur and venture capitalist. In the evening, I am busy creating a system that makes Chinese learning Chinese simple. Methodology I invented a way to learn written language by understanding their basic shapes and intentions.Once they can identify the main characters of several groups, they can combine them and easily learn dozens of additional characters.By repeating the process of "identification and combination", students can quickly learn hundreds of Chinese characters in a very short period of time. A person who studies Chinese will master amazing 20,000 Chinese characters, but to reach the basic level of literacy, you only need to know about 1,000.But the most interesting thing is that if you can learn the top 200 Chinese characters, it is enough to read about 40 % of Chinese popular literature.This is enough for many practical applications, such as reading road signs and restaurant menus, as well as the main points of mastering newspapers and websites. In the past two years, I have deconstructed nearly two thousand most commonly used Chinese characters and found the top 100 "components".These components are similar to different shapes and sizes.In order to create more characters, you only need to combine the two or three (sometimes four or five) of these components.For non -Chinese readers, this is a really powerful tool for quickly and effective understanding of the meaning of Chinese characters.The memory process is interesting, fast and lasting because it uses a vivid and interesting display method.I hope you will find that learning Chinese characters is as interesting as the methodology behind I create Chinese.

Reading: 45 2019-03-27

Anima: The first open-source 33B Chinese language model based on QLoRA

Anima: The first open-source 33B Chinese language model based on QLoRA

The first open-source 33B Chinese language model based on QLoRA. The AI community has always been very open, and the development of AI today cannot be separated from many important open source works, open shared papers, or open source data and code. We believe that the future of AI will also be open. I hope to make some contributions to the open source community. Why is the 33B model important? Is QLoRA a Game Changer? Previously, most of the open-source models that could be finetuned were relatively small models 7B or 13b, although they could perform well through finetune training on some simple chatbot evaluation sets. However, due to the limited scale of these models, the reasoning ability of LLM core is relatively weak. That's why many of these small-scale models behave like toys in practical application scenarios. As discussed in this work, the chatbot evaluation set is relatively simple, and there is still a clear gap between small and large models in complex logical reasoning and mathematical problems that truly test the model's ability. Therefore, we believe that the work of QLoRA is very important, to the point where it could be a Game Changer. Through the optimization method of QLoRA, for the first time, a 33B scale model can be trained with a more democratic and low-cost finetune, and widely used. We believe that the 33B model can not only leverage the strong reasoning ability of large-scale models, but also flexibly fine tune training for private business domain data to enhance control over LLM.

Tag: QLoRA

Reading: 21 2023-07-23

1
Recommend