做似然估计时，Log likelihood为负是否正常？ [推广有奖]

31楼

白日梦小姐 发表于 2013-11-7 11:24:02

楼主，我也刚刚弄懂这个问题。不知道现在回复对你而言算不算晚，但是至少对以后看到这帖子并且有同样疑问的人应该有用。

赞同9楼的看法。具体的原因可进一步阅读：http://wenku.baidu.com/view/77a63505eff9aef8941e0691.html

32楼

隆末客 发表于 2014-2-23 21:04:33

1. loglikelihood 通常应该为负。因为累积likelihood值通常为小数，取自然对数后……必然为负。
2.但是也有例外，如假设sample为正态分布时，如果variance过小，会导致loglikelihood为正。
原因如下，
在连续变量的情况下
我们用maximum likelihood method时，用的是probability density function，得到的并不是‘真正’的概率，而是单位概率。如，f（s）在x=0，取值为0.325，并不意味着此点发生概率为此。
所以loglikelihood有可能为证，在normal distribution时，尤其sample是daily return of stock

33楼

Lisrelchen 发表于 2014-2-23 22:27:22

The likelihood is proportional to the probability of observing the data given the parameter estimates and your model. To ensure that they are comparable across models the models have to be nested, i.e. you can get from the more complex model to the less complex model by imposing a set of constraints on the more complex model. If the models are nested than a larger likelihood function means a larger probability of observing the data, which is good.

A positive log likelihood means that the likelihood is larger than 1. This is possible because the likelihood is not itself the probability of
observing the data, but just proportional to it.

The likelihood is hardly ever interpreted in its own right (though see (Edwards 1992[1972]) for an exception), but rather as a test-statistic, or as a means of estimating parameters. There are a number of goodness of fit statistics based on the likelihood: many of the pseudo-Rsquares, the AIC, and the BIC.

34楼

Lisrelchen 发表于 2014-2-23 22:33:13

There's nothing inherently wrong with positive log likelihoods, because likelihoods aren't strictly speaking probabilities, they're densities. When they occur, it is typically in cases with very few variables and very small variances. For raw data, we define the log likelihood of a model as the density of the model-implied multivariate normal distribution for each observed data raw. If we had three values (0, 1, 2) and fit a model with a mean of 1 and variance of 2/3, we'd get densities of .231, .487 and .231. If we use 0, .01 and .02 and fit mean .01 variance 2/300 instead, those densities become 2.31, 4.87 and 2.31. The likelihoods change with the different scaling, and one yields a positive log-likelihood and one a negative, but they're the same model.

The issue with low variance items is not about how weird positive likelihoods look, but that variances can't go below zero, and very low variances run an increased risk of the optimizer picking a negative variance or your model bumping up against whatever bound you enforce.