Hamiltonian from a Lagrangian with constraints?
Comments to the question (v2):
To go from the Lagrangian to the Hamiltonian formalism, one should perform a (possible singular) Legendre transformation. Traditionally this is done via the Dirac-Bergmann recipe/cookbook, see e.g. Refs. 1-2. Note in particular, that the constraint $f$ may generate a secondary constraint
$$g ~:=~ \{f,H^{\prime}\}_{PB} +\frac{\partial f}{\partial t}~\approx~\frac{d f}{d t}~\approx~0.$$
[Here the $\approx$ symbol means equality modulo eqs. of motion or constraints.]
References:
P.A.M. Dirac, Lectures on QM, (1964).
M. Henneaux and C. Teitelboim, Quantization of Gauge Systems, 1994.
The Hamiltonian is defined by $$ H = \sum_{i=1}^n \left( \frac{\partial L}{\partial \dot q_i} \dot q_i \right) - L $$ So in your case: $ H' = H - \lambda f $