Aha, the very reason that checkbox (disable caustics adaptivity) was added. Adaptivity, as the name suggests, picks which areas need caustics and which don't, and how much to focus on the bits that do. This speeds things up, but in some cases it can't pick correctly, and artifacts can appear. Disabling it avoids those, but will render slower. However, slower and without artifacts is the better choice in such scenarios!