Hi Avi, thanks so much for detailed explanation.
I tried your solution. Problem with black highlights is solved now but mapping of reflections is a bit off when using scene tab overrides.
I solved this by modifying panorama photo and making a proper spherical map from it.
Thank you.