This is correct behavior. When caustics are disabled, transparent shadows are used instead. Similar to checking "Affect shadows" checkbox in VrayMTL. When Caustics are on, shadows are opaque and caustics are traced in them. Progressive renderer is quite inefficient at resolving caustics though, that's why there is almost no caustics yet, just a few fireflies. You may try Bidir renderer with VCM enabled to render caustics faster.