What are Elmo based embeddings ?

ELMO embeddings are deep contextual embedding which takes into account all hidden layers to generate embeddings :  

  • They combine a linear combination of all layers of a deep pre-trained neural network instead of just the last layer to get superior embeddings that model syntactic and semantic characteristics of the word use and polysemy – variation across linguistic concepts.
  • They are character based and can take morphological cues and can handle out of vocabulary words better.  A deep bidirectional language model (biLM), is used to generated ELMO embeddings.

Leave a Reply

Your email address will not be published. Required fields are marked *