Fast Inference Methods in Natural Language Processing

/medias/photo/seminaire-di_1630676501273-jpg

Résumé :
Modern natural language processing models are base on very large neural networks, meaning that inference is usually slow, even using modern GPUs.

In this talk, I will quickly overview several research topics that aim to leverage the architecture of modern GPUs to develop fast inference methods.
I will then quickly present two of my recent works on the topic.

References :

• Bregman Conditional Random Fields: Sequence Labeling with Parallelizable Inference Algorithms (Caio Corro, Mathieu Lacroix, Joseph Le Roux) https://arxiv.org/abs/2506.00732
• KAD: A Framework for Proxy-based Test-time Alignment with Knapsack Approximation Deferral (Ayoub Hammal, Pierre Zweigenbaum, Caio Corro)