I've just tried the same setup (with the refract override) on the legacy material and the problem disappears, it renders as I'd expect it to.
Basic glass setup, legacy material, diffuse 0, reflect 1, refract 1, thin ticked
If anything, the legacy glass looks better when working as expected, which makes me think, even a black override in refraction is affecting it with the Physical material.
So to summarize, Physical material, reflect override only works if not overriding refract at the same time, Legacy material - allows you to do both