If it helps knowing I'm running a dual Epyc build that has 8 memory slots per CPU (16 total). Just for the fun of it I tried running the benchmarks (and one rather simple interior) in a 8 sticks vs 16 sticks scenario.
The results were for all intents and purposes identical between the two configs.
I do think that going with less than 8 sticks though would result in slower rendering - I haven't tested it out though.
Do note that each platform can be a bit different so the above might work for my 7B13 Epycs but might not for your dual Xeons.