LMLM: Linguistically Motivated Language Models
Name of applicant
Rob van der Goot
Title
Associate Professor
Institution
IT University of Denmark
Amount
DKK 6,988,496
Year
2025
Type of grant
Semper Ardens: Accelerate
What?
Design LLMs inspired by human language processing to increase robustness. More specifically, we will incorporate the principle of compositionality; the idea that meaning of language is obtained from the meanings of its parts and how they are put together.
Why?
Current language models have a homogeneous design, which leads to inherit limitations. To increase their robustness, we need to rethink their design and training procedure. Compositional processing will avoid language models to learn shortcuts, which will lead to more robust outputs.
How?
To mimic the hierarchical processing of human language processing, we will train sequential deep learning layers, each trained to represent a different granularity of language, starting with sub-parts of words, then words, followed by phrases. Finally, we will add more general layers to add reasoning, knowledge, and generation capabilities.