Using language models to solve the NLP tasks is getting more popular each day. It has been proven that language models can give us state of the art results in most of the NLP tasks such as classification.

In this talk we go over the important factors that can help us to choose the best LM for our task and also improve the final results by applying preprocessing and post processing steps.

We discuss model selection, as well as re-training a language model for our task. Then we discuss how preprocessing can help us improve the results more.

Finally, we will discuss two different approaches that we applied on the classification layers to get better results as well as how to use ensemble language models.