Light traveling in the 3D world interacts with the scene through intricate processes before being captured by a camera. Physics based vision aims to invert the processes to recover the scene properties, such as shape, reflectance, light distribution, medium properties, etc., from images. In recent years, deep learning shows promising improvement for various vision tasks.
When physics based vision meets deep learning, there must be mutual benefits. On one hand, classic physics based vision tasks can be implemented in a data-fashion way to handle complex scenes. This is because, physically more accurate optical models can be too complex to be solved (usually too many unknown parameters in one model). These intrinsic physical properties potentially can be learned through deep learning. On the other hand, deep learning methods should consider physics principles in the modeling and computation, since the models can provide strong constraints and rich knowledge about the real world.
Therefore, we believe when physics based vision meets deep learning, many vision algorithms can get the benefits.