How to know whether Lagrange multipliers gives maximum or minimum?

As Om(nom)$^3$ said in the comments, if you're working on a closed and bounded region then it's not possible to get only one critical point.

If you're not on a closed and bounded region then it's no longer guaranteed that you'll have more than one critical point. If you only have one critical point then you can use the Bordered Hessian technique. (Thanks to ziggurism for clearing that up.)

On a closed bounded region a continuous function achieves a maximum and minimum. If you use Lagrange multipliers on a sufficiently smooth function and find only one critical point, then your function is constant because the theory of Lagrange multipliers tells you that the largest value at a critical point is the max of your function, and the smallest value at a critical point is the min of your function. Thus max = min, i.e. the function is constant. Also note that "critical point" should probably be called something else, like "point of interest" because usually critical points are defined as points where the gradient is zero.

In fact the normal second derivative test doesn't apply to constrained extremum problems. You should instead use the Bordered Hessian method. In brief, instead of computing the positive-definiteness of the Hessian matrix of second partial derivatives of $f$, you instead compute the Hessian of $f-\lambda g$, including derivatives with respect to $\lambda$