So this is probably the cause. In low resolution rendering, the displacement should work fine. If the resolution is so high, the mesh becomes subdivided multiple times. You can for example try lowering displacement quality settings (either bigger screen size or lower max subdiv per poly, or both).