LMLM: Linguistically Motivated Language Models
Navn på bevillingshaver
Rob van der Goot
Titel
Associate Professor
Institution
IT University of Denmark
Beløb
DKK 6,988,496
År
2025
Bevillingstype
Semper Ardens: Accelerate
Hvad?
Design LLMs inspired by human language processing to increase robustness. More specifically, we will incorporate the principle of compositionality; the idea that meaning of language is obtained from the meanings of its parts and how they are put together.
Hvorfor?
Current language models have a homogeneous design, which leads to inherit limitations. To increase their robustness, we need to rethink their design and training procedure. Compositional processing will avoid language models to learn shortcuts, which will lead to more robust outputs.
Hvordan?
To mimic the hierarchical processing of human language processing, we will train sequential deep learning layers, each trained to represent a different granularity of language, starting with sub-parts of words, then words, followed by phrases. Finally, we will add more general layers to add reasoning, knowledge, and generation capabilities.