When I need to mask sides out (and can't use poly selections for some reason), i like to use a vertex map as a mask.
I don't see much difference to using a poly selection here, though, nor was I missing the refraction.
Anyway, maybe this method could be of some use for you (replace the missing normal map with one of your choice in the file).
Top glass: normal on both sides, you are right, far too busy.
Left glass: the vertex map solution
Right glass: the selection set solution
PS
As the file name says, there will be another crash with IR I need to report ;)
CU
Michael