I know that any Noctua cooler is not felt noisy even under full load. If it is really noisy (you can try recording a video, if possible, preferably with open setup, hold the microphone as close as possible to the case, and also do mouse clicks for reference), then probably there are mechanical flows. Maybe the fan is not well mounted? Maybe it is not quite the Noctua fan(s) but the case fans which kick in based on CPU temperature? For my last build, I have noticed that the case fans were as noisy as the pump, and have set up a custom profile for all of them, in the BIOS, with Asus's "FanControl" something. And that IS the recommended place to set up the fan curves.
Given that you are sure of the underclocking and stability, you can even set it to not be used with 100 load even on high temps.