As we witness a surge of interest in exploring the potential of machine learning in scientific and engineering applications where there are often well established first principle models, curious questions regarding the interplay between machine learning and physics models become important both at conceptual level and for practical purposes, such as what are the connections and differences, the interactions of machine learning and physics in a collated framework, and the comparative advantage and limitations of each method. In this talk I will present several electromagnetic (EM) inversion cases where conventional inversion, machine learning based regression, and machine learning with physics constraints are applied and compared. In the case of machine learning with physics constraints, I will show how the prior and posterior density can be effectively shaped by alternating between minimizing the data and parameter space losses, respectively, to improve inversion resolution and accuracy.