As XR technologies are gaining more traction, estimating detailed geometry and appearance of scenes and humans is becoming a crucial task in order to have realistic and interactive virtual experiences. The majority of today's methods focus only on reconstructing geometry and simple textures. However, estimation of underlying physical properties like reflectance and illumination is required to enable applications like modification of lighting conditions and materials, and photo-realistic re-rendering of the observed scene in novel views. Solving this highly ill-posed inverse rendering problem requires densely sampling the input space of the functions modeling the underlying appearance properties. This is usually not feasible outside of controlled lab frameworks. For this reason, I am interested in how learning algorithms, synthetic datasets, and different capturing systems can aid in solving the geometry and appearance reconstruction of scenes and humans in ordinary everyday settings.everyday settings.