Well, I remember I've red something about displace map and IR, but I'm not sure, so I'm posting this.
Perspective view, zoom on the top left of the wall hole, start IR. Than zoomed out and this is the result (attached), displace is working only on a part of the surface.
To solve, just stop and restart IR.
Done on Max 2015 with Apr 8 Corona build