hm, maybe i not undertand what he wants. but i use it that way for more than one hdri.
overall i use it the way that the "rest" really is only the "leftovers" withut any light in it, so all light and light materials have their own layer, then the "rest layer" has nothing but an BG maybe, and every thing is tunable in seperate layers.
that said i not use envrionments only dome lights(sky)