if you mean the "Display AA", then this option only affects displaying custom VFB during rendering, not how final image looks like.
Higher internal res. can indeed cause some overhead if set too high, or when the scene is simple or resolution high. But it should not interfere with Display AA.
Box filtering wont help you with the jagged edges - you need to render the image in higher resolution, tone map it, and then downscale (which is what the internal res. does automatically)