ICML 2008

Semantics: relating to the (hidden) meaning in the language or logic.

Multi-task learning (MTL) is the procedure of learning several tasks at the same time witht the aim of mutual benefit.

In this paper the authors study training multiple NLP tasks jointly in MTL manner.

Most notable task is training a LM in an unsupervised manner using the ranking type cost. The task is to classify if the word is in the right company or not using the ranking type cost. In loose terms if the input sentence makes sense to humans or no?

The lm learns better representation of the words compared to the task of predicting what would be the next word. Reason being the ranking cost takes full advantage of before and after (context) rather than just the previous words context.

They show joint training is helpful.

[A Discriminative Language Model with Pseudo-Negative Samples](https://jointphd.notion.site/A-Discriminative-Language-Model-with-Pseudo-Negative-Samples-5d36719fa71441dd91a8c011ccd0d302)