Hi, switching to linear color space seems to help somewhat (I could swear I tried this before). With a layer shader and a (127/127/255) color shader above the normal map image you can controle the normal strenght to a certain extend.
And sorry if I sounded a bit impolite, it was late at night ;-)