Chaos Corona Forum

Chaos Corona for Cinema 4D => [C4D] I need help! => Topic started by: BigAl3D on 2019-10-16, 23:57:27

Title: More Team Render Woes
Post by: BigAl3D on 2019-10-16, 23:57:27

So for some reason, we started getting this strange error when using Team Render. We were rendering a still frame at 4k, with some Depth of Field effect. Previously, we tried an 8k still and could not get it to work at all. Not sure if it has anything to do with Maximum size of packets settings or not, but I've never fully understood those numbers. Although, after we changed to to Interval to 1 and Packet Size to 30, it seems to work better. I just revisited an old thread where TomG talked about large size images and manually setting those numbers to 20s and 256MB.

(Complete Console text attached)

2019/10/16 17:26:55  [Corona4D]

Sending chunk 26/30 to the server 2019/10/16 17:26:56  [Corona4D] Frame synchronization failed: Communication Error 2019/10/16 17:26:57  [Corona4D] Sending chunk 26/30 to the server 2019/10/16 17:26:57  [Corona4D] Frame synchronization failed: Communication Error 2019/10/16 17:26:59  [Corona4D] Sending chunk 26/30 to the server 2019/10/16 17:26:59  [Corona4D] Frame synchronization failed: Communication Error 2019/10/16 17:27:00  [Corona4D] Sending chunk 26/30 to the server 2019/10/16 17:27:00  [Corona4D] Frame synchronization failed: Communication Error 2019/10/16 17:27:02  [Corona4D] Sending chunk 26/30 to the server 2019/10/16 17:27:02  [Corona4D] Frame synchronization failed: Communication Error 2019/10/16 17:27:03  [Corona4D] Sending chunk 26/30 to the server 2019/10/16 17:27:04  [Corona4D] Frame synchronization failed: Communication Error Then the clients start dropping off the render job. Didn't matter if we launched the render from the Mac Pro or the iMac. We have a couple of Windows 10 clients too, but only render clients. Full-speed non-debug version Build timestamp: Sep 26 2019 12:25:41 Version: 4 hotfix 3 (core 4 (Hotfix 3)) Cinema version: R20.059 CINEMA 4D Studio (single-license) MacOS 10.12.4 (16E195) Mac Pro (late 2013)

Title: Re: More Team Render Woes
Post by: TomG on 2019-10-17, 01:27:04

If you mean and interval of 1, that would be 1 second - which would mean data tries to be sent every second, which would most likely lead to network congestion.

The first thing to try is under the Corona Team Render settings, swap to Arbitrated (experimental), if you haven't already. You can start with default values of 5 for Interval and 64 for packet size. If that doesn't work, I would try the numbers I mentioned below - this sends data much less often (only every 20s rather than every 5 seconds), but in larger chunks when it does send data (256MB rather than 64MB). Sending data too often (low interval) can cause your network to overload and data will be lost. Sending chunks that are too small can cause TR to be pretty pointless, as it will take so long to send the data every x seconds in y chunks that it will be slower than not using TR.

Along with that, useful info is:
How many nodes are you using? (the more nodes, the more likely this is to be network congestion)
What is the speed of your network? (the slower the network, the more likely this is to be network congestion)

Cheers!

Title: Re: More Team Render Woes
Post by: BigAl3D on 2019-10-17, 19:45:56

Thanks for the reply. Yes that makes a lot of sense as far as not overloading the network. No point in sending tiny chunks every second, when you don't need to. Some info on our LAN. Seems to be fast to my little brain.

10 packets transmitted, 10 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.153/0.186/0.220/0.023 ms

10 Gbit/s


The nodes can range from one to eight. Some scenes are too slow on certain nodes. So if we do use all eight nodes, it would make sense to increase the time interval AND the size of the chunks, correct?

Title: Re: More Team Render Woes
Post by: TomG on 2019-10-17, 20:11:33

Good on the network speed, that's a plus!

That sounds like what I would try, for more nodes, longer interval, but larger chunks. If you still have problems, do let us know (I'd be interested to know if you don't have problems, what settings you settled on, as I am sure there will be some variance based on number of nodes and network speed, but we aren't able to test all combinations here to see what works best, so real world facts and figures would be good to see).

Title: Re: More Team Render Woes
Post by: HFPatzi on 2019-10-21, 16:04:32

After long time of testing, my go to settings are 50s / 50MB in manual mode. Most of the time 11 Machines are beeing used. Network is 1Gbit/s.

Maybe it helps.

Title: Re: More Team Render Woes
Post by: BigAl3D on 2019-10-23, 20:32:04

Forgot to revisit this thread. We think at some point we may have reduced the interval down to 1 sec when we had problems. The funny thing is, when we got it to work, the number was increased back to the original 5 sec.

Title: Re: More Team Render Woes
Post by: HFPatzi on 2019-10-23, 21:50:39

I guess it also depends on the size of the image