There should be a Corona Displacement modifier that you can adjust to get the correct displacement height. In my experience, it is always too high, and I'm not sure what is the correct value besides just using my eye.
Also, I'm not sure if this is an issue for anyone else, but the default bump value is set to 99 on Quixel materials...I usually adjust that back down to 1.
I've also noticed that the format of displacement maps from Quixel isn't great because they don't use full values from white to black, so I often plug them into a gradient ramp to adjust the levels so I can get the full range. This gives me much better control of the displacement.
Another issue I have, and I'm just throwing this out there to see if anyone else has any input, is that I often find Quixel metal material to be wildly inaccurate when compared to their shader ball example.