Shusheng Pu Yu

InternLM is a large-scale multilingual language model developed by Shanghai Artificial Intelligence Laboratory and SenseTime (with equal contributions) in collaboration with The Chinese University of Hong Kong, Fudan University, and Shanghai Jiao Tong University.

InternLM is a multilingual base language model with 104B parameters. InternLM undergoes a multi-stage progressive process of pre training on a large corpus with 1.6T tokens, followed by fine-tuning to fit human preferences. We have also developed a training system called Uniscale LLM for efficient training of large-scale language models.

The evaluation of multiple benchmarks shows that InternLM has achieved state-of-the-art performance in various aspects such as knowledge understanding, reading comprehension, mathematics, and coding. With such comprehensive capabilities, InternLM achieved excellent results in comprehensive exams including MMLU, AGIEval, C-Eval, and GAOKAO Bench without the need for external tools.

In these benchmark tests, InternLM not only outperforms open source models significantly, but also achieves superior performance compared to ChatGPT. In addition, InternLM demonstrates excellent understanding of the Chinese language and culture, making it a suitable foundational model for supporting language applications targeting Chinese.