Chaos Corona Forum
Chaos Corona for 3ds Max => [Max] General Discussion => Topic started by: simtub on 2022-10-25, 08:46:28
-
We're considering leasing a few units of Dual Epyc Racks for our inhouse rendering needs. I'm wondering whether Corona 9 has any limitations on the number of cores per cpu when rendering.
Does anyone have any experience and knowledge on this? We'd be using backburner to render individual frames for animations but for still images, maybe all of the racks will have Corona distributed rendering turned on. So say 4 x Dual Epyc Racks (8 x 64 core CPU's) all rendering 1 single image on DR. Would that work? Would W11 handle this load? I know we had issues with W10 on the 12900K's not being able to read all the cores and had to upgrade to W11 to resolve the issue.
-
No issues.
-
So say 4 x Dual Epyc Racks (8 x 64 core CPU's) all rendering 1 single image on DR. Would that work?
To answer this part, it would but would benefit highly from 10GBe network, but Epyc motherboards usually all come with it. And there are some cheap-er/cheap-ish 10GBe switches already.
-
We finally got our Dual Epyc systems and did a test on the Corona Benchmark and it seems like the hardware is not running at full speed.
Results for our system were:
00:00:18.86 25,763,100 AMD EPYC 7773X 64-Core Processor (×2) Win 2.98 128 / 256 256 2023-02-03
We saw a similar spec system on the database and it was twice as fast!
00:00:09.12 53,289,000 AMD EPYC 7773X 64-Core Processor (×8) Win 2.79 128 / 256 512 2022-05-16
Any idea why that could be?
-
(x2) vs (x8)? I have no idea if 8 CPUs on a single board is possible at all, or if it's just a benchmarks failure to properly detect specs :]
-
(x2) vs (x8)? I have no idea if 8 CPUs on a single board is possible at all, or if it's just a benchmarks failure to properly detect specs :]
Yes I saw that too and I think it's inaccurate. For the Threadripper (Which are single socket only) CPU's on the database it also lists (x2)
-
What version of Windows are you running, and are you sure that can access all the threads of all the processors? https://www.amd.com/en/processors/epyc-minimum-operating-system#amd-epyc-7003-series would suggest it has to be Windows Server?
-
What version of Windows are you running, and are you sure that can access all the threads of all the processors? https://www.amd.com/en/processors/epyc-minimum-operating-system#amd-epyc-7003-series would suggest it has to be Windows Server?
We're running Windows 11 Pro. All threads of the processors are working and showing on the Task Manager
-
Have a look at the Task Manager during the rendering if it's utilising all the threads. Alternatively, see if you can change the graph to see NUMA nodes (right click onto graph).
You can try disabling virtualization in bios, setting power profile to High or Ultimate (from stock balanced) and run the benchmark multiple times.
-
We had Windows 11 Pro installed and apparently this tops out at 2 CPU's and 64 Cores per CPU. So we upgraded to Windows 11 Pro Workstation Edition that can take up to 4 CPU's and 6TB of Ram. Still have the same issues. It's utilizing all threads in the task manager but the CPU fluctuates up and down constantly during rendering from 100% to 40-50%.
Virtualization is disabled in bios and power profile is on high.
We noticed that in Cinebench and Vray benchmarks the results are consistent to all the hardware out there.
Have a look at the Task Manager during the rendering if it's utilising all the threads. Alternatively, see if you can change the graph to see NUMA nodes (right click onto graph).
You can try disabling virtualization in bios, setting power profile to High or Ultimate (from stock balanced) and run the benchmark multiple times.
-
I've seen similar issues when some collegues bought eng. samples of same high capacity and they eventually got it to run at full performance. Sadly I can't remember what they have done, hopefully they will chime in.
-
Some updates and positive news.
On our Dual EPYC node, we realised that 4 x 64Gb Dimm configuration was the issue. Supposedly on a Single CPU, filling 4 Dimm slots would enable the system to run in Quad Channel but since we have 2 CPU's then 4 Dimms is just a Dual Channel Memory set up and we would need 8 Dimms for Quad Channel or 16 Dimms for 8-Channel Memory set ups. (Our school boy error when setting out the original specifications).
So it seems Corona is much more bandwidth dependent on the number of memory channels much more so than Vray and Cinebench. When we ran benchmarks for Vray and Cinebench in Dual Channel, the scores were very good and matched to other systems of similar spec. It was only Corona that had issues, Dual Channel Ram literally halved the performance on the Corona Benchmark giving us around 20seconds where as Quad Channel gave a respectable 11 seconds. I suspect 8-Channel may push the Corona benchmark to 8-9 seconds but we don't have enough Ram to test this.
Other things we considered to resolve this speed issue before switch the RAM were:
- Upgrading to Windows Pro 11 Workstation Edition (This version can run more CPU's and RAM than Windows 11 Pro but it didn't affect our case even with 128 Cores and 256 Threads)
- Turning off Virtualization (Didnt really do much)
- Turning off VBS, Core Isolation and Memory Integrity on Windows (Didn't really do much)
- Power Plan on High Performance (Both in Windows and BIOS - Not sure how this affects the system performance as yet)
- NUMA settings on BIOS (Not yet looked into it)
- Windows Updates and latest BIOS
Hope this can help others out there configuring Dual Socket Systems. I would be interested to know from the Corona Development team why there is such a dramatic performance difference between DUAL and QUAD/8 Channel RAM configs as Vray doesn't have this issue at all.
Cheers.
-
Corona doesn't really care about bandwidth, I tested some latest dual-Xeon systems (8280, 8380, etc. and will soon test upcoming Saphire Rapids next week) and there was almost zero noticeable difference between single to 8-channel.
I've replicated same results with all Threadripper gens. Comparison of 4-channel Threadripper and 8-channel Threadripper Pro showed zero difference in 99perc. of work tasts across almost all SPEC benchmarks. Latency on other hand was very important, but less so with latest AMD gens with IO on die (and Epycs had also IO on die).
What can be happening with Epycs is more complicated inter-core communication with the 64+ core versions esp. in multiple sockets. Dataset on one CPU will need to access the data-set on second one. Populating channels is thus not increasing the performance, but just making sure a bottleneck isn't happening.
With Sapphire Rapids Xeons and next Epycs this will be simplified since DDR5 can run two memory channels on single DIMM.
Happy for you to get all the performance back though :- ).
-
What can be happening with Epycs is more complicated inter-core communication with the 64+ core versions esp. in multiple sockets. Dataset on one CPU will need to access the data-set on second one. Populating channels is thus not increasing the performance, but just making sure a bottleneck isn't happening.
Yes I think the 64 Core versions of the EPYCS running in multiple sockets is the issue here with Corona and not every dual CPU system. That's why it was so hard to find out what the problem was as there wasn't many user cases out there.
Thanks again for the advices given.
-
Maybe a silly question, but is it worth the price of such a system ?...
-
Maybe a silly question, but is it worth the price of such a system ?...
I guess it depends on your rendering needs. We had to render a lot of animation work lately and using an online renderfarm was too costly especially since we have to develop the animation throughout the process so a lot of draft rendering was required for interim meetings before final rendering.
We had considered buying a large number of desktops based on either Intel 12900K’s at the time or Ryzen 7950x’s but we calculated that we needed around 10 systems and it just wasn’t feasible space wise to hook up 10+ PC’s together with 10 separate power supplies and network connections. On top of that you have to deal with 10 installs and multiple software licenses. The hardware would have cost much less than the dual Epycs though.
Having the dual epycs on racks allowed us to store them in the server room which is temp controlled and saves a lot of space. In the long run I think the return on investment is justifiable. We will be using these for the next several years.
-
Maybe a silly question, but is it worth the price of such a system ?...
If you are feeling a bit adventurous you can try snagging used EPYC CPUs for a lot less :) Doesn't come without risks though obviously.
-
If you don't render animation, powerful multi-threaded system is no longer needed or worth the price.
If you do render animation, no amount of systems will ever be enough, and managing fewer but more powerful machines is absolutely worth the cost of human and mental labor.
Installing (even cloning), Licencing, etc.. is such a damn hassle. Few big machines > More lesser machines.