Displacement relies on the underlying geometry and only displaces in the direction of the normal. If you have polygons that are at right angles to each other, then the direction of the normals are at right angles which creates what you are seeing. One solution is to add a bevel to the edge with a few edge loops to allow enough angles between the faces to appear like it is smooth.