Questions tagged [likelihood]

Given a random variable $X$ which arise from a parameterized distribution $F(X; \theta)$, the likelihood is defined as proportional to the probability of observed data as a function of $\theta$: $\operatorname{L}\left(\theta \mid x \right)=\operatorname{P} \left(X=x \mid \theta \right)$.

In statistics, a likelihood is a function of the parameters of a statistical model evaluated for a fixed sample of observations, defined as follows:

Likelihood — The likelihood that any parameter (or set of parameters) should have any assigned value (or set of values) is proportional to the probability that if this were so, the totality of observations should be that observed.

Fisher, Ronald A. "On the mathematical foundations of theoretical statistics." Philosophical transactions of the Royal Society of London. Series A, containing papers of a mathematical or physical character 222.594-604 (1922): 309-368.

Likelihood functions play a key role in statistical inference, especially methods of estimating a parameter using a statistic (a function of the data).

Excerpt reference: @ars's answer on What is the difference between “likelihood” and “probability”?

1594 questions
691
votes
12 answers

What is the difference between "likelihood" and "probability"?

The wikipedia page claims that likelihood and probability are distinct concepts. In non-technical parlance, "likelihood" is usually a synonym for "probability," but in statistical usage there is a clear distinction in perspective: the number that is…
Douglas S. Stones
  • 7,781
107
votes
5 answers

What is the reason that a likelihood function is not a pdf?

What is the reason that a likelihood function is not a pdf (probability density function)?
John Doe
  • 1,596
  • 3
  • 18
  • 25
99
votes
7 answers

Why to optimize max log probability instead of probability

In most machine learning tasks where you can formulate some probability $p$ which should be maximised, we would actually optimize the log probability $\log p$ instead of the probability for some parameters $\theta$. E.g. in maximum likelihood…
Albert
  • 1,275
74
votes
5 answers

Why do we minimize the negative likelihood if it is equivalent to maximization of the likelihood?

This question has puzzled me for a long time. I understand the use of 'log' in maximizing the likelihood so I am not asking about 'log'. My question is, since maximizing log likelihood is equivalent to minimizing "negative log likelihood" (NLL), why…
Tony
  • 1,863
73
votes
2 answers

What is the difference between a partial likelihood, profile likelihood and marginal likelihood?

I see these terms being used and I keep getting them mixed up. Is there a simple explanation of the differences between them?
Rob Hyndman
  • 60,133
67
votes
5 answers

How to calculate pseudo-$R^2$ from R's logistic regression?

Christopher Manning's writeup on logistic regression in R shows a logistic regression in R as follows: ced.logr <- glm(ced.del ~ cat + follows + factor(class), family=binomial) Some output: > summary(ced.logr) Call: glm(formula = ced.del ~ cat +…
dfrankow
  • 3,506
62
votes
3 answers

What kind of information is Fisher information?

Suppose we have a random variable $X \sim f(x|\theta)$ . If $\theta_0$ were the true parameter, the the likelihood function should be maximized and the derivative equal to zero. This is the basic principle behind the maximum likelihood estimator. As…
Stan Shunpike
  • 4,543
54
votes
7 answers

Why would someone use a Bayesian approach with a 'noninformative' improper prior instead of the classical approach?

If the interest is merely estimating the parameters of a model (pointwise and/or interval estimation) and the prior information is not reliable, weak, (I know this is a bit vague but I am trying to establish an scenario where the choice of a prior…
user10525
49
votes
10 answers

Why do people use p-values instead of computing probability of the model given data?

Roughly speaking a p-value gives a probability of the observed outcome of an experiment given the hypothesis (model). Having this probability (p-value) we want to judge our hypothesis (how likely it is). But wouldn't it be more natural to calculate…
Roman
  • 784
42
votes
5 answers

How to derive the likelihood function for binomial distribution for parameter estimation?

According to Miller and Freund's Probability and Statistics for Engineers, 8ed (pp.217-218), the likelihood function to be maximised for binomial distribution (Bernoulli trials) is given as $L(p) = \prod_{i=1}^np^{x_i}(1-p)^{1-x_i}$ How to arrive at…
Ébe Isaac
  • 1,132
38
votes
4 answers

Maximum likelihood function for mixed type distribution

In general we maximize a function $$ L(\theta; x_1, \ldots, x_n) = \prod_{i=1}^n f(x_i \mid \theta) $$ where $f$ is probability density function if the underlying distribution is continuous, and a probability mass function (with summation instead of…
bonifaz
  • 1,205
36
votes
5 answers

Wikipedia entry on likelihood seems ambiguous

I have a simple question regarding "conditional probability" and "Likelihood". (I have already surveyed this question here but to no avail.) It starts from the Wikipedia page on likelihood . They say this: The likelihood of a set of parameter…
Creatron
  • 1,705
34
votes
3 answers

How to rigorously define the likelihood?

The likelihood could be defined by several ways, for instance : the function $L$ from $\Theta\times{\cal X}$ which maps $(\theta,x)$ to $L(\theta \mid x)$ i.e. $L:\Theta\times{\cal X} \rightarrow \mathbb{R} $. the random function $L(\cdot \mid X)$…
Stéphane Laurent
  • 21,014
34
votes
4 answers

Theoretical motivation for using log-likelihood vs likelihood

I'm trying to understand at a deeper level the ubiquity of log-likelihood (and perhaps more generally log-probability) in statistics and probability theory. Log-probabilities show up all over the place: we usually work with the log-likelihood for…
ratsalad
  • 713
33
votes
8 answers

Bayes' Theorem Intuition

I've been trying to develop an intuition based understanding of Bayes' theorem in terms of the prior , posterior , likelihood and marginal probability. For that I use the following equation: $$P(B|A) = \frac{P(A|B)P(B)}{P(A)}$$ where $A$ represents…
Anas Ayubi
  • 431
1
2 3
…
99 100