For constrained optimisation ideally you should be using Lagrange multipliers. But even then there are `abnormal` cases where the Lagrange multiplier fails to give you an optimal solution. Roughly your constraint does not have enough information to give you a unique solution (dg insufficient rank to find solutions in df = \lambda dg).
In infinite dimensions (minimising a functional) a penalty metric is an even worse an idea. My favorite example is sub-Riemannian geometry [0]. People wanted to find sub-Riemannian geodesics - special curves that could only move in restricted directions. Early on people tried to use the penalty metric idea (making it really expensive to move in certain directions), but Montgomery [0] proved that this does not necessarily give optimal solutions. The limit of an equation is not necessarily the same as the limit of the solutions. In a sense they used Lagrange multipliers to examine the situation.
Looking at abnormal optimisation problems is an interesting area of research [1]. However if a local solution is "good enough" maybe you can get away with a penalty metric.
[0] https://www.ams.org/books/surv/091/ [1] https://books.google.com.au/books?isbn=9401594384