That can be done with per light object include/excludes.
If you want white light for specific objects, create white light (sun, coronalight etc.) and include only these objects for the white light.
Environment from "colorful" HDRI will still illuminate the facades, but if you play around with the "white" light (it can be slightly blueish or with another tint to cancel out the yellow/other color tint from HDRI), you can get expected results.