Quantitative Linguistics in the Large Language Models era: the study of semanticity in Catalan

Enfoque

In the era of Large Language Models (like ChatGPT or Google BARD), the field of computational linguistics faces a pressing challenge: bridging the gap between theoretical linguistic models and the transformative capabilities of network models, particularly transformer-based models. A long-standing understanding within linguistics is that the frequency of words within semantic networks is closely linked to their meanings and syntactic functions, a concept dating back to the earliest models in quantitative linguistics and connectionist models.

However, the training data for large language models diverges significantly from the conventional acquisition of human languages. This disparity underscores the importance of delving into new models and quantitative linguistic principles and the so-called “Linguistic laws”: these are statistical patterns that hold true in human languages and other communication systems, analogous to the statistical laws of physics. For example, G. K. Zipf formulated a couple of statistical laws on the relationship between the frequency of a word with its number of meanings: the law of meaning distribution, relating the frequency of a word and its frequency rank, and the meaning-frequency law, relating the frequency of a word with its number of meanings. In light of this, we recently introduce a novel concept called ”semanticity” which establishes a connection between a word’s potential meanings and its position within the linguistic network.

To explore this notion, we conduct a comprehensive analysis of Catalan using extensive oral and written corpora, leveraging the resources of the official dictionary (DIEC2). Our findings reveal that the semanticity of words provides a straightforward classification for content and function words and for various word types in Catalan, allowing for the integration of both their semantic and syntactic attributes within this single parameter.

Ultimately, we present the potential and limitations of this linguistic property and advocate for the examination of semanticity in other languages. This endeavor aims to forge new connections between the realms of computational and theoretical linguistics, ushering in a new era of linguistic exploration and understanding.