I started my Ph.D. in April 2023, at the Center for Language and Cognition (CLCG), University of Groningen.

My research mainly focuses on low-resource conversational generation, the generalization of factual knowledge across languages, and prompt-based learning for classification. The ultimate goal for me is to achieve Artificial General Intelligence (AGI) someday in the future.

Some of my papers were published at international NLP conferences. See for details.

🔥 News

📝 Publications

EMNLP 2023

Cross-Lingual Consistency of Factual Knowledge in Multilingual Language Models
Jirui Qi, Raquel Fernández, Arianna Bisazza

Multilingual large-scale Pretrained Language Models (PLMs) have been shown to store considerable amounts of factual knowledge, but large variations are observed across languages. With the ultimate goal of ensuring that users with different language backgrounds obtain consistent feedback from the same model, we study the cross-lingual consistency (CLC) of factual knowledge in various multilingual PLMs. To this end, we propose a Ranking-based Consistency (RankC) metric to evaluate knowledge consistency across languages independently from accuracy. Using this metric, we conduct an in-depth analysis of the determining factors for CLC, both at model level and at language-pair level. Among other results, we find that increasing model size leads to higher factual probing accuracy in most languages, but does not improve cross-lingual consistency. Finally, we conduct a case study on CLC when new factual associations are inserted in the PLMs via model editing. Results on a small sample of facts inserted in English reveal a clear pattern whereby the new piece of knowledge transfers only to languages with which English has a high RankC score.

🎮 Demos

  • Interested in having a competition against LMs? Try our demo here and see if you can beat them!

📖 Educations

  • 2023.04 - Current: Ph.D. Candidate, Center for Language and Cognition, University of Groningen
  • 2020.09 - 2023.01: Master, Beijing Advanced Innovation Center for Big Data and Brain Computing, Beihang University
  • 2016.09 - 2020.06: Undergraduate, Beijing Jiaotong University
  • 2013.09 - 2016.06: Senior High, The High School Affiliated to Renmin University of China (RDFZ)