MLE of a discrete random variable

Be aware that, when doing MLE (in general, when doing parametric estimation) you are computing (estimating) a parameter of a probability function (pmf). If the variable is discrete, it means (roughly) that its probability function takes discrete values (in this case, $k=1,2,3$), but the parameter itself can be continuous (it can take any real value, in some domain). So, the first thing you need to make clear is that:

  • what is the parameter of my pmf that I want to estimate? in this case, it's $\theta$

  • it's continuous? what's its domain? in this case, looking at the pmf, we see that $\theta$ must be in the range $[-1,1]$. In this range, and only in this range the probability function is valid (takes non-negative values). Then the parameter is continous and its domain is $-1 \le\theta \le 1$

Once you have that establlished, you try to write the likelihood. If you are not sure, start by some simple example. Assume you have only two samples, say, $x_1=2$, $x_2=0$. The likelihood of this realization is

$$L(\theta)=p(x_1=2;\theta) \times p(x_2=0;\theta) = \frac{1+\theta}{3} \frac{1-\theta}{3} $$

To write this in general, suppose you have $n_0$ samples that take value $x=0$, $n_1$ that take value $x=1$ etc. Then

$$L(\theta)=p(x=0;\theta)^{n_0}p(x=1;\theta)^{n_1}p(x=2;\theta)^{n_2}$$

Write that expression down, and take its logarithm if you think this simplifies things (it does). Then ask yourself: for given $n_0,n_1,n_2$, this is a (continous) function of $\theta$, what is the value of $\theta$ that maximizes this function, in the given domain?


Update: given that you've done your homework, here's my solution $$\log L(\theta)= n_0 \log(1+\theta) +n_2 \log(1-\theta) +\alpha $$ where $\alpha $ is a term that does not depend on $\theta$ (we can leave it out). This function is differentiable in $(-1,1)$, so we can look for critical points (candidate extrema) as:

$$\frac{d\log L(\theta)}{d \theta}= \frac{n_0}{1+\theta}-\frac{n_2}{1-\theta} $$ Equalling this to zero, we get $\theta_0=(n_0-n_2)/(n_0+n_2)$

Have we already found then the MLE? Not really. We have only found a critical point of $L(\theta)$. To assert that a critical point is a global maximum we need to 1) check that it's a local maximum (it could be a local minimum or neither) 2) check that the local maximum is really a global maximum (what about the non-differentiable or boundary points?).

We can usually check that with the second derivative. But in this case it's simpler. We see that at the boundary ($\theta = \pm 1$) the likelihood tends to $-\infty$. Hence, given that the function is differentiable inside the interval, and it has a single critical point, it must be a (local and global) maximum.