What is perplexity ? Where do you typically use perplexity ?

Perplexity is a measure used in probabilistic modeling. In NLP it is used to measure  how well the probabilistic model explains the observed data. It is closely related to likelihood, which is the value of the joint probability of the observed data.

Suppose the model generates data x_1, x_2, \hdots, x_N, then the perplexity can be computed as:

    \[perplexity = e^{\frac{- \sum_{i=1}{N} p(x_i)}{N}}\]

When each observation is an independent word, for NLP tasks, this is called  per word perplexity. If each x_i corresponds to a document, it is called per document perplexity.

Leave a Reply

Your email address will not be published. Required fields are marked *