Is this solved ?Or just noticed ? I might have scene with issue like this.
I had the same problem with the scenes where I wanted to lit scene with Corona sky and reflections taken from HDRi map. For my case I found pretty simple solution: I've just saved hdr image as jpg, and then used it to override Direct visibility/Reflections/Refractions - it makes reflections to clean muuuuch faster.