In meantime, imma go upgrade to 10gbe because all those 8k textures aren't being friendly to 1gbit network..
As I'm currently planning my studio implementation I wonder if i have to do this aswell. I'm starting from scratch so I need to build a futur proof installation. But it cost a lot. Network adapter, NAS, cables, rooter.... What's more, it looks like SSDs are a must have to enjoy real capabilities of 10Gbe (in raid 1 for security purpose). Let us know your findings !
I will post a guide how I went about it, it's not that expensive actually if you choose specific products and I stayed conservative, no SPF optics or Infiniband stupidity. Traditional copper wiring with RJ45, CAT6a cables, and Netgear prosafe series. You can do it under 1.5k for small office. I think it's almost the same price as regular link-aggregated setup, but much more useful.
Actually it's not even so much textures I realized :- ) It's Distributed .exr dumps. If you render high-res (like 8k), that's almost 40 Mpx, you need to set the pixel transfer to atleast 1/10 of this to even see any distributed benefit in real-time, otherwise it will just keep sending like 50 strips forever....not very useful visually. And sometimes distributed malfunction and doesn't even finish sending the remaining data outside of those strips in the end. Uh.
So 8k render, with tons of passes like Lightmixer, that amount to heavy, heavy amount of bandwidth. Each node will be sending like 1GB .exr every minute, that takes 12 seconds over 1gbit network, so 5 nodes will totally saturate it (when reaching the master workstation).
People with backburner animation rendering are fine, you can have even 40 PCs over 1gbit in such case, it will just take longer time for them to start-up the first-time they need to cache textures, but then it's breeze. But damn, I like distributed to be instant and useful in real-time.
Having a good synced back-up instead of ssd raid is more useful these days. There is very small chance SSDs will fail, if ever, but if they do it's usually because of reasons that would flail the other ones in raid also.