maximum likelihood estimation derivation

\mathrm{avar}(\sqrt{n}(\hat{\theta}_{mle}-\theta))-\mathrm{avar}(\sqrt{n}(\tilde{\theta}-\theta))\leq0 I know my solution is probably not correct, but I don't know what else I should try. given the parameter vector $\theta.$ The joint density satisfies56 For most statisticians, it's like the sine . \vdots\\ \end{align*}\], The likelihood function is defined as the joint density treated \ln L(\theta|\mathbf{r})=-\frac{T}{2}\ln(2\pi)-\frac{T}{2}\ln(\sigma^{2})-\frac{1}{2\sigma^{2}}\sum_{t=1}^{T}(r_{t}-\mu)^{2},\tag{10.25} 0 stream f(r_{t};\theta)=(2\pi\sigma^{2})^{-1/2}\exp\left(-\frac{1}{2\sigma^{2}}(r_{t}-\mu)^{2}\right),\text{ }-\infty<\mu<\infty,\text{ }\sigma^{2}>0,\text{ }-\infty> \end{equation}\] which can be solved for $\hat{\theta}_{2}$: & +\frac{1}{2}(\theta-\hat{\theta}_{1})^{\prime}\frac{\partial^{2}\ln L(\hat{\theta}_{1}|\mathbf{x})}{\partial\theta\partial\theta^{\prime}}(\theta-\hat{\theta}_{1})+error.\nonumber What are the weather minimums in order to take off under IFR conditions? \end{align}\], \[ 1= {'Drni%|x7I B+#AJD`LKPA>%Zd|U\9M7*xtK A{XiObbz5O2!MeT!2p4WA#(.T/V(3Gz}ahadN "8v It so happens that the data you collected were outputs from a distribution with a specific set of inputs. Many of the concepts are exactly the same. 26 0 obj Merely finding that the derivative is $0$ at a certain point is not enough to prove there is a global maximum at that point, but the two inequalities above do it. function is typically negative (being the log of a number less than to estimate the ARCH-GARCH model parameters. \[\begin{equation} But generally youll find maximization of the log likelihood more common. variables $X,\,S_{X}=\{x:f(x;\theta)>0\},$ does not depend on $\theta$; \hat{\mu}_{mle}=\frac{1}{T}\sum_{i=1}^{T}r_{t}=\bar{r}.\tag{10.26} L(\theta|x_{1},\ldots,x_{T})\approx\left(\prod_{t=p+1}^{T}f(x_{t}|I_{t-1};\theta\right).\tag{10.21} Numerical optimization methods generally have the following structure: The most common method for numerically maximizing () is . Author of bobbywlindsey.com. \end{equation}\] The sample score is a $(2\times1)$ vector given by Patterson & Thompson (1971) introduced residual maximum likelihood estimation in the case of unbalanced incomplete block designs. \end{equation}\] Space - falling faster than light? To determine these two parameters we use the Maximum-Likelihood Estimate method. As a result, numerical Remember, youre solving this theoretically so dont need to actually get data as the values of your sample data wont matter in the following derivation. can be factored as the product of the conditional density of $X_{2}$ It is the statistical method of estimating the parameters of the probability distribution by maximizing the likelihood function. then $\hat{\alpha}_{mle}=h(\hat{\theta}_{mle})$ is the MLE for $\alpha.$, In the CER model, the log-likelihood is parametrized in terms of $\mu$ According to Miller and Freund's Probability and Statistics for Engineers, 8ed (pp.217-218), the likelihood function to be maximised for binomial distribution (Bernoulli trials) is given as L ( p) = i = 1 n p x i ( 1 p) 1 x i How to arrive at this equation? \[ densities of $\theta$ given the data $x_{1},\ldots,x_{T}.$ It is important This type of derivation suffers from two problems: it is rather cumbersome and, in fact, it is incomplete as it does not include a proof that the so-obtained estimate is indeed a global maximum . does not work because the random variables $\{X_{t}\}_{t=1}^{T}$ $$ \epsilon_{t} & = & \sigma_{t}z_{t},\\ Views are my own. Here, the factorization of the likelihood function given in (10.16) (ML) is typically used to estimate the ARCH-GARCH parameters. \sigma_{2}^{2}=\omega+\alpha_{1}\epsilon_{1}^{2}+\beta_{1}\sigma_{1}^{2}=\omega+\alpha_{1}(r_{1}-\mu)^{2}+\beta_{1}\sigma_{1}^{2}, In maximum likelihood estimation we want to maximise the total probability of the data. It is usually much easier to maximize the log-likelihood function \sigma_{t}^{2} & = & \omega+\alpha_{1}\epsilon_{t-1}^{2}+\beta_{1}\sigma_{t-1}^{2},\\ Hence, the values for $\sigma_{t}^{2}$ in (10.23) The best answers are voted up and rise to the top, Not the answer you're looking for? of equations $S(\hat{\theta}_{mle}|\mathbf{x})=\mathbf{0}.$ However, with the CER model estimates presented in Chapter 7. \ln L(\theta|\mathbf{x})=\ln\left(\prod_{t=1}^{T}f(x_{t}|I_{t-1};\theta)\right)=\sum_{t=1}^{T}\ln f(x_{t}|I_{t-1};\theta). and so increasing function the value of the $\theta$ that maximizes $\ln L(\theta|\mathbf{x})$ (Hessian) \[ First, assume the distribution of your data. %PDF-1.5 the log-likelihood function is regardless of the value of $\theta$. Youve done quite well so far and have collected some data. at time $t$, $f(x_{t}|I_{t-1};\theta)$ is the pdf of $x_{t}$ conditional sample variance. Maximumlikelihood estimate (MLE) The maximum likelihood estimator is dened as: b . For these models, the marginal joint pdf is often ignored in (10.20) The elements of $S(\theta|\mathbf{r})$, unfortunately, do not have I(\theta|\mathbf{x})=-E[H(\theta|\mathbf{x})]. ), upon maximizing the likelihood function with respect to , that the maximum likelihood estimator of is: ^ = 1 n i = 1 n X i = X . \frac{\partial\ln L(\hat{\theta}_{mle}|\mathbf{x})}{\partial\theta}=\left(\begin{array}{c} and $\theta=(\mu,\sigma^{2})^{\prime}.$ The joint density Now ask, what is the likelihood of getting the sample you got? \hat{\sigma}_{mle}^{2}=\frac{1}{T}\sum_{i=1}^{T}(r_{t}-\bar{r})^{2}.\tag{10.27} It seems pretty clear to me regarding the other distributions, Poisson and Gaussian; \[\begin{equation} So. \] are complicated nonlinear functions of the elements of $\theta$ and endobj The MLE can be found by calculating the derivative of the log-likelihood with respect to each parameter. We have some sample data points represented by Y. To subscribe to this RSS feed, copy and paste this URL into your RSS reader. Iteration stops when $S(\hat{\theta}_{n}|\mathbf{x})\approx0$. << /Pages 67 0 R /Type /Catalog >> . <> Maximizing the Likelihood. This method estimates the parameters of a model given some data. stream Maximum likelihood estimation (MLE) is a technique used for estimating the parameters of a given distribution, using some observed data. And assuming each sample is independent from each other, we can define the likelihood function as: Now that you have your likelihood function, you want to find the value of the distributions parameter that maximizes the likelihood. \end{array}\right). \frac{\partial\ln L(\hat{\theta}_{mle}|\mathbf{x})}{\partial\theta_{1}}\\ (clarification of a documentary), Cannot Delete Files As sudo: Permission Denied. If $X_{1},\ldots,X_{T}$ are discrete random variables, then $f(x_{1},\ldots,x_{T};\theta)=\Pr(X_{1}=x_{1},\ldots,X_{T}=x_{T})$ \[\begin{equation} L(\theta|x_{1},\ldots,x_{T})=f(x_{1},\ldots,x_{T};\theta)=\prod_{t=1}^{T}f(x_{t};\theta).\tag{10.17} Here, we see that the MLE for $\mu$ is equal to the The expected amount of information in the sample about the In some cases, it is possible to find analytic solutions to the set So, we have the data, what we are looking for. the ML estimator of $\theta$ has the following asymptotic properties of the joint pdf $f(x_{1},\ldots,x_{T};\theta)$ has the form As a result, numerical optimization methods true value of $\theta$ because every value of $\theta$ produces the log-likelihood function is 7 0 obj \frac{\partial\ln L(\hat{\theta}_{mle}|\mathbf{r})}{\partial\mu} & =\frac{1}{\hat{\sigma}_{mle}^{2}}\sum_{t=1}^{T}(r_{t}-\hat{\mu}_{mle})=0\\ \frac{\partial\ln L(\theta|\mathbf{r})}{\partial\sigma^{2}} we have a lot of information about $\theta.$ On the other \[\begin{equation} Let this cost function be represented as P(Y ; z). \frac{\partial\ln L(\theta|\mathbf{r})}{\partial\sigma^{2}} & =-\frac{T}{2}(\sigma^{2})^{-1}+\frac{1}{2}(\sigma^{2})^{-2}\sum_{i=1}^{T}(r_{t}-\mu)^{2}. In this L(\theta|x_{1},\ldots,x_{T})=f(x_{1},\ldots,x_{T};\theta)=Pr(X_{1}=x_{1},\ldots,X_{T}=x_{T}). \[ }$$ \ln L(\theta|\mathbf{x})=\ln\left(\prod_{t=1}^{T}f(x_{t};\theta)\right)=\sum_{t=1}^{T}\ln f(x_{t};\theta). By definition, the MLE satisfies $S(\hat{\theta}_{mle}|\mathbf{x})=0.$. Thanks for contributing an answer to Mathematics Stack Exchange! The derivative of the logarithm with respect to $\theta$ is With random sampling, the log-likelihood is additive in the log of to $\theta$; (3) the true value of $\theta$ lies in a compact set . (2) $f(x;\theta)$ is at least three times differentiable with respect \end{equation}\], \[\begin{eqnarray*} Viewed 174 times 1 I recently learned about the multivariate Gaussian distribution, and I saw a formula derivation in the literature where I do not know how to simplify the log-likelihood from K 2 log | | 1 2 i = 1 K ( y i ) T 1 ( y i ) to K 2 ( log | | + tr ( 1 ) + ( ) T 1 ( )), Maximum likelihood estimation In statistics, maximum likelihood estimation ( MLE) is a method of estimating the parameters of an assumed probability distribution, given some observed data. \hat{\theta}=\arg\max_{\theta}\text{ }\ln L(\theta|\mathbf{x}) ^ MLE = max P(Dj ) = max logP(Dj ) = max log ( k(1 )n k) = max klog +(n k)log(1 ) This is something that can be solved analytically, so we take the derivative with . simple closed form expressions and no analytic formulas are available . Hence, the sample average is the MLE for $\mu.$ Using $\hat{\mu}_{mle}=\bar{r}$ \], It is often quite difficult to directly maximize $L(\theta|\mathbf{x}).$ Let $R_{t}$ denote the daily return on an asset and assume that $\{R_{t}\}_{t=1}^{T}$ Concealing One's Identity from the Public When Purchasing a Home, QGIS - approach for automatically rotating layout window. $k$, potentially nonlinear, equations in $k$ unknown values: \end{equation}\], \[\begin{equation} \ln L(\theta|\mathbf{x})=\ln\left(\prod_{t=1}^{T}f(x_{t}|I_{t-1};\theta)\right)=\sum_{t=1}^{T}\ln f(x_{t}|I_{t-1};\theta). As we shall see in the next sub-section, the information matrix is In maximum likelihood estimation, the parameters are chosen to maximize the likelihood that the assumed model results in the observed data. To recap, you just need to: Originally published at https://www.bobbywlindsey.com on November 6, 2019. To find the maximum value, we take the partial derivative of our expression with respect to the parameters and set it equal to zero. f(r_{t};\theta)=(2\pi\sigma^{2})^{-1/2}\exp\left(-\frac{1}{2\sigma^{2}}(r_{t}-\mu)^{2}\right),\text{ }-\infty<\mu<\infty,\text{ }\sigma^{2}>0,\text{ }-\infty> Stack Overflow for Teams is moving to its own domain! \], $\hat{\alpha}_{mle}=h(\hat{\theta}_{mle})$, $\sigma=h(\sigma^{2})=(\sigma^{2})^{1/2},$, \[ f(x_{1},x_{2},x_{3};\theta)=f(x_{3}|x_{2},x_{1};\theta)f(x_{2}|x_{1};\theta)f(x_{1};\theta). It can be shown (we'll do so in the next example! Linear Regression Complete Derivation With Mathematics Explained! 25 0 obj %PDF-1.4 \hat{\sigma}_{mle}=(\hat{\sigma}_{mle}^{2})^{1/2}=\left(\frac{1}{T}\sum_{i=1}^{T}(r_{t}-\hat{\mu}_{mle})^{2}\right)^{1/2}. The log-likelihood is given by log L = 3 log 3 x log ( e) = 3 log 3 x. \], \[ Since $\theta$ is $(k\times1)$ the first order conditions define \end{align}\] $$\ell^{\prime\prime}(\theta)=\dfrac{-3}{\theta^2}<0$$ In reality, you dont actually sample data to estimate the parameter but rather solve for it theoretically; each parameter of the distribution will have its own function which will be the estimated value for the parameter. Its a bit like reverse engineering where your data came from. f(x_{1},\ldots,x_{T};\theta)=f(x_{1};\theta)\cdots f(x_{T};\theta)=\prod_{t=1}^{T}f(x_{t};\theta).\tag{10.16} f(x_{1},\ldots,x_{T};\theta) & \geq0,\\ \sigma_{1}^{2}=\omega+\alpha_{1}\epsilon_{0}^{2}+\beta_{1}\sigma_{0}^{2}.\tag{10.24} xcbdg`b`8 $X@,c |o3 !$$ Hp~N7H ff&o G K The maximum likelihood estimate is a method for fitting failure models to lifetime data. This R_{t} & = & \mu+\epsilon_{t},\\ L(\theta|\mathbf{r}) & =\prod_{i=1}^{n}(2\pi\sigma_{t}^{2})^{-1/2}\exp\left(-\frac{1}{2\sigma_{t}^{2}}(r_{t}-\mu)^{2}\right).\tag{10.23} \end{array}\right), \], \[ \end{equation}\] Recall, the \[ \[\begin{equation} \[\begin{equation} $f(x_{1},\ldots,x_{T};\theta)$ is not a joint probability but represents is, $\hat{\theta}_{mle}$ solves the optimization problem Assumptions Our sample is made up of the first terms of an IID sequence of normal random variables having mean and variance . \end{equation}\] f(x_{1},\ldots,x_{T};\theta) & =\left(\prod_{t=p+1}^{T}f(x_{t}|I_{t-1};\theta)\right)\cdot f(x_{1},\ldots,x_{p};\theta),\tag{10.19} \[ It estimates the model parameter by finding the parameter value that maximises the likelihood function. z_{t} & \sim & iid\,N(0,1). Take the derivative and set it equal to 0 and I get ^ = 1 x. The value of $\theta$ Formally, $\theta$ is identified L(\theta|x_{1},\ldots,x_{T})=f(x_{1},\ldots,x_{T};\theta)=Pr(X_{1}=x_{1},\ldots,X_{T}=x_{T}). \frac{\partial\ln L(\hat{\theta}_{mle}|\mathbf{r})}{\partial\sigma^{2}} & =-\frac{T}{2}(\hat{\sigma}_{mle}^{2})^{-1}+\frac{1}{2}(\hat{\sigma}_{mle}^{2})^{-2}\sum_{i=1}^{T}(r_{t}-\hat{\mu}_{mle})^{2}=0 if for all $\theta_{1}\neq\theta_{2}$ there exists a sample $\mathbf{x}$ This lecture deals with maximum likelihood estimation of the parameters of the normal distribution . Asking for help, clarification, or responding to other answers. of $\theta$ and $\alpha=h(\theta)$ is a one-to-one function of $\theta$ L(\theta|\mathbf{r}) & =\prod_{t=1}^{T}(2\pi\sigma^{2})^{-1/2}\exp\left(-\frac{1}{2\sigma^{2}}(r_{t}-\mu)^{2}\right)=(2\pi\sigma^{2})^{-T/2}\exp\left(-\frac{1}{2\sigma^{2}}\sum_{t=1}^{T}(r_{t}-\mu)^{2}\right),\tag{10.18} for $\sigma_{1}^{2}$ is determined from (10.24). Who is "Mar" ("The Master") in the Bavli? \[ to emphasize that $\sigma_{t}^{2}$ is a function of $\theta$. than the estimation of the CER model parameters. \vdots\\ and solving the second equation for $\hat{\sigma}_{mle}^{2}$ gives Similar to this method is that of rank regression or least squares, which essentially "automates" the probability plotting method mathematically. \], \[ Show that the MLE is unbiased. Now mathematically, maximizing the log likelihood is the same as minimizing the negative log likelihood. ", Handling unprepared students as a Teaching Assistant, Movie about scientist trying to find evidence of soul. \], $\mathbf{x}=(x_{1},\ldots,x_{T})^{\prime}$, \[ The likelihood function is always positive \] probability density function (pdf) $f(x_{t};I_{t-1};\theta),$ where plug-in principle estimators for the conditional variance parameters. Update the starting values in a direction that increases. \end{equation}\] \ln L(\theta|\mathbf{x})=\ln\left(\prod_{t=1}^{T}f(x_{t};\theta)\right)=\sum_{t=1}^{T}\ln f(x_{t};\theta). The likelihood, log-likelihood and score functions for a typical model \], \[ \], \[ & =\hat{\theta}_{1}-H(\hat{\theta}_{1}|\mathbf{x})^{-1}S(\hat{\theta}_{1}|\mathbf{x}) Then the joint pdf and likelihood function \], \[\begin{align*} Maximum-likelihood estimator resulting in complex estimator, Maximum Likelihood Estimator for a Random Sample from Bernoulli distribution. \[ Now the maximum likelihood estimation can be treated as an optimization problem. where $\mathbf{r}=(r_{1},\ldots,r_{T})^{\prime}$ denotes the observed \end{eqnarray*}\], $R_{t}|I_{t-1}\sim N(\mu,\sigma_{t}^{2})$, \[\begin{equation} The maximum likelihood estimate in factor analysis is typically obtained as the solution of the stationary point equation of the likelihood function. Let $R_{t}$ denote the daily return on an asset and assume that $\{R_{t}\}_{t=1}^{T}$ My question: We can extract the values of these parameters using maximum likelihood estimation (MLE). Thus, this is essentially a method of fitting the parameters to the observed data. denote the observed sample. which uses a divisor of $T-1$ instead of $T$. 09 80 58 18 69 contact@sharewood.team Notice that the likelihood function is a $k$ dimensional function The log-likelihood is given by $\log L = 3\log\theta - 3\theta x \log(e) = 3\log\theta - 3\theta x.$ Take the derivative and set it equal to $0$ and I get $\hat{\theta} = \frac{1}{x}$. endstream are determined recursively using the GARCH(1,1) variance equation << /Filter /FlateDecode /Length 3490 >> \] $$L(\theta)=\prod_{i=1}^{3}f(X_i \mid \theta)=\prod_{i=1}^{3}\theta e^{-3\theta X_i} = \theta^3 e^{-\theta \sum_{i=1}^{3}X_i}$$ 27 0 obj f(x_{1},x_{2};\theta)=f(x_{2}|x_{1};\theta)f(x_{1};\theta). In the Poisson distribution, the parameter is . \frac{\partial\ln L(\theta|\mathbf{r})}{\partial\alpha_{1}}\\ \], $S(\hat{\theta}_{n}|\mathbf{x})\approx0$, $f(x_{1},\ldots,x_{T};\theta)=\Pr(X_{1}=x_{1},\ldots,X_{T}=x_{T})$, Introduction to Computational Finance and Financial Econometrics with R. Write this section in a similar way as the asymptotic section in the chapter on Estimating the CER model. f(x_{1},x_{2};\theta)=f(x_{2}|x_{1};\theta)f(x_{1};\theta). Now use algebra to solve for : = (1/n) xi . \] Once you differentiate the log likelihood, just solve for the parameter. \[ for a fixed value of $\theta.$, Standard regularity conditions are: (1) the support of the random It is the value of $\theta$ that makes the observed data most likely. \end{array}\right). \[\begin{equation} You want to know the probability of at least x visitors to your channel given some time period. I(\theta|\mathbf{x})=-E[H(\theta|\mathbf{x})]. \], Since the MLE is defined as a maximization problem, under suitable \], \[ \], \[\begin{align} where $\theta=(\mu,\omega,\alpha_{1},\beta_{1})^{\prime}$. & +\frac{1}{2}(\theta-\hat{\theta}_{1})^{\prime}\frac{\partial^{2}\ln L(\hat{\theta}_{1}|\mathbf{x})}{\partial\theta\partial\theta^{\prime}}(\theta-\hat{\theta}_{1})+error.\nonumber pdf $f(x_{1},\ldots,x_{T};\theta)$ and likelihood function $L(\theta|x_{1},\ldots,x_{T})$. \] \[\begin{align*} \end{equation}\], \[ Use MathJax to format equations. \int\cdots\int f(x_{1},\ldots,x_{T};\theta)dx_{1}\cdots dx_{T} & =1. for ARCH and GARCH models the set of equations $S(\hat{\theta}_{mle}|\mathbf{x})=0$ The conditional From the properties of the GARCH(1,1) model we know that $R_{t}|I_{t-1}\sim N(\mu,\sigma_{t}^{2})$ \frac{\partial\ln L(\hat{\theta}_{mle}|\mathbf{x})}{\partial\theta}=\mathbf{0}. L(\theta|x_{1},\ldots,x_{T})=f(x_{1},\ldots,x_{T};\theta)=\left(\prod_{t=p+1}^{T}f(x_{t}|I_{t-1};\theta\right)\cdot f(x_{1},\ldots,x_{p};\theta).\tag{10.20} directly related to the precision of the MLE. These include maximum likelihood estimation, maximum a posterior probability (MAP) estimation, simulating the sampling from the posterior using Markov Chain Monte Carlo (MCMC) methods such as Gibbs sampling, and so on. The joint density is a $T$ dimensional function of the data $x_{1},\ldots,x_{T}$ A Gaussian distribution will have two, etc. This is achieved by maximizing a likelihood function so that, under the assumed statistical model, the observed data is most probable. f(r_{t}|I_{t-1};\theta)=(2\pi\sigma_{t}^{2})^{-1/2}\exp\left(-\frac{1}{2\sigma_{t}^{2}}(r_{t}-\mu)^{2}\right),\tag{10.22} Before continuing, you might want to revise the basics of maximum likelihood estimation (MLE). Maximum Likelihood Estimation Eric Zivot May 14, 2001 This version: November 15, 2009 1 Maximum Likelihood Estimation 1.1 The Likelihood Function Let X1,.,Xn be an iid sample with probability density function (pdf) f(xi;), where is a (k 1) vector of parameters that characterize f(xi;).For example, if XiN(,2) then f(xi;)=(22)1/2 exp(1 How do I derive the maximum likelihood estimator based on $X = (X_1, X_2, X_3)$? Estimation of parameter of Bernoulli distribution using maximum likelihood approach $\Theta$. define $\hat{\theta}_{mle}$ as the value of $\theta$ that solves To learn more, see our tips on writing great answers. to the MLEs. \frac 3 \theta - \sum_{i=1}^3 x_i \quad \begin{cases} >0 & \text{if } 0<\theta< \frac 1 3 \sum_{i=1}^3 x_i, \\ \\ <0 & \text{if } \theta> \frac 1 3 \sum_{i=1}^3 x_i. Now you know how to use Maximum Likelihood Estimation! Solving the first equation for $\hat{\mu}_{mle}$ gives When the migration is complete, you will access your Teams at stackoverflowteams.com, and they will no longer appear in the left sidebar on stackoverflow.com. L(\theta|x_{1},\ldots,x_{T})=f(x_{1},\ldots,x_{T};\theta)=\left(\prod_{t=p+1}^{T}f(x_{t}|I_{t-1};\theta\right)\cdot f(x_{1},\ldots,x_{p};\theta).\tag{10.20} (10.27). conditional likelihood function (10.21). parameter $\theta$ is the information matrix \max_{\theta}L(\theta|\mathbf{x}). To find the maxima of the log likelihood function LL (; x), we can: Take first derivative of LL (; x) function w.r.t and equate it to 0. \frac{\partial\ln L(\theta|\mathbf{r})}{\partial\beta_{1}} \] are illustrated in figure xxx. Stop updating when the FOCs are satisfied. is described by the CER model. \end{align}\], \[\begin{equation} We start with the likelihood function for the Poisson distribution: Then differentiate it and set the whole thing equal to zero: Now that you have a function for , just plug in your data and youll get an actual value. \hat{\theta}=\arg\max_{\theta}\text{ }\ln L(\theta|\mathbf{x}) By clicking Post Your Answer, you agree to our terms of service, privacy policy and cookie policy. That is, if $\hat{\theta}_{mle}$ is the MLE \hat{\theta}_{mle}\sim N\left(\theta,\frac{1}{n}I(\theta|x_{t})^{-1}\right)=N(\theta,I(\theta|\mathbf{x})^{-1}) \], \[ It's a bit like reverse engineering where your data came from. \ln L(\theta|\mathbf{r})=-\frac{T}{2}\ln(2\pi)-\frac{T}{2}\ln(\sigma^{2})-\frac{1}{2\sigma^{2}}\sum_{t=1}^{T}(r_{t}-\mu)^{2},\tag{10.25}