Render Farm Build

[superfluido]

Hello Everyone, and thanks for your help.

I am now finally convinced to build my own render farm. I need more render power and online render farms are not an option due to long waiting lines (many still renders, so quick to process, but I need them fast), lack of control and random issues I always seem to come by.

I am happily rendering on Corona for C4D on a Mac Studio M1 Ultra 128 GB ram with no complaints: the machine is powerful yet super silent and sits quietly on my desk. Still, I need to speed up the workflow, thus the render farm.

A few dumb questions, to whom it has more experience in the field that I do:
_Do render nodes need to be as powerful as the main machine? (Probably not)
_Sometimes I find to be short on ram due to very high-poly-count scenes, does that mean that all the render nodes should also have 128GB of ram? (It seems quite unreasonable)
_Is it better to buy a big and powerful (and expensive) machine like a TR 5995 with lots of ram, or is it fine to have many cheaper render nodes?
_It seems that the new M2 Mac Mini has quite a good price to performance ratio and packs everything I need to set up the render farm. Is it a stupid idea to buy 10 Mac Minis instead of one big machine? Or do I need more ram on every unit?
_Do I need to move all my resources and assets to a dedicated server?
_What would you do with a budget of 5 to 10k?

Any suggestions? Thank you All :)

[TomG]

Some answers (but not all :) )

_Do render nodes need to be as powerful as the main machine? (Probably not)
They can be as powerful, more powerful, less powerful - it's all good.

_Sometimes I find to be short on ram due to very high-poly-count scenes, does that mean that all the render nodes should also have 128GB of ram? (It seems quite unreasonable)
Yes, the RAM on each machine will need to be able to handle the scene. While there may technically be some savings from not having the full GUI of the DCC open, having less other software running, etc. it is safest to assume that if your scene wouldn't work on 64GB on your main machine and needs 128GB, then the render nodes will also need 128GB (else they won't be able to render the scene).

_Is it better to buy a big and powerful (and expensive) machine like a TR 5995 with lots of ram, or is it fine to have many cheaper render nodes?
Again, either works. It may affect how things are "delivered" - e.g. say Machine A, or two Machine Bs which are half the power. If Machine A would deliver one frame in 1 minute for an animation, then the other option will see you waiting for 2 minutes before you see any final result, BUT it will be 2 frames and not one. It's never entirely accurate, but you can as a general guideline "add up" the overall power of all the machines combined. That said note that if you get into a lot of machines, you may start running into network issues with all the traffic, and would need to look into that as another aspect :)

[CambridgeCreative]

Hi Superfluido

I would add to TomG's resonse that you also need to consider licencing costs for both Cinema 4D and Corona and also power consumption. Cinema has 5 node licences as standard but Corona comes with one, then extra costs start at one node licence and then jumps to a package for 5...... I worked in an office 3 years ago with 5 render nodes that were old and not very powerful but you got a sense of burning through lots of frames, but actually the rendering took a long time. Now in my current office we have two PCs, one a Ryzen 9 5800X 64GB RAM (that i use to model on) and the other a Threadripper 3880X 128GB RAM (used just for rendering) and I would rate this setup superior.

[CambridgeCreative]

One other thing, I also find that the delay in the threadripper downloading assets and then starting rendering can offset its speed that it can render frames in small/uncomplicated animations compared to the speed of the server. My PCs mentioned above are hardwired together with crossover cables, so asset transfers are as quick as I can get. I haven't tried connecting another Ryzen 9 as a render client to see if that starts rendering quicker, but it might be something to consider.

As a note also, I have noticed the Threadripper is massively quicker than the Ryzen 9 as a client on animations with polycounts over 5 million.

[superfluido]

Wow, super! Thanks a lot for the replies – I'll probably go with one powerful build (Threadripper 3990 or 5995 with lots of ram, just to be sure) so that in case I need more I can just (with time) side another machine in parallel. It seems also easier to maintain, plus, as you said, it counts as one node only and maybe it could function as a server too. Let's see!

[jamesdowling]

Just so it's on your radar there have been a few leaks that the new threadrippers will be announced this month and may be released shortly after, and may have 96cores at the top end. 
The Threadrippers that are currently out are powerful but are really overpriced for quite old tech now (especially compared to a Ryzen 9 7950x). So before spending 5k+ on a threadripper build, if you are not in a super rush to upgrade it might be worth holding off and seeing what AMD reveals.

[Nejc Kilar]

Just so it's on your radar there have been a few leaks that the new threadrippers will be announced this month and may be released shortly after, and may have 96cores at the top end.
The Threadrippers that are currently out are powerful but are really overpriced for quite old tech now (especially compared to a Ryzen 9 7950x). So before spending 5k+ on a threadripper build, if you are not in a super rush to upgrade it might be worth holding off and seeing what AMD reveals.

I definitely agree with the above but it might also be worth mentioning that today's leaks from some of the more "famous" leakers suggest that the Oct 19th launch of the 7xxx TRs will be mainly for OEMs with very low DIY (if any) availability. Only one way to find out what they are planning :)

[superfluido]

Thanks for the update on the new Threadripper! I'll sure wait to check the prices and performance before making a new purchase.

Meanwhile, just to help me decide, I created a spreadsheet with some possible combinations of workstations – jpg attached. Prices are what I could find in my region for a finish build after discounts, I considered the cost of render node licences on a lifespan of 5 years. When the price to performance ratio was good, I tried adding more machines to the render farm. I assumed the render power would multiply linearly, which might not be the case, but I did not know how else to calculate it otherwise.

The results were not what I was expecting. It seems the best render farm could come from 5xMac Studios... I did not take electricity cost into consideration, but it would probably favour the Mac Studios, as it is very energy efficient (and silent, which in my office is a bug plus). Am I missing something?

[Nejc Kilar]

That is quite an interesting list you've got there, thank you for sharing it! What a cool way to go about things :)

I haven't looked at it in depth but I do think it might be worthwhile to check whether 7950x oriented builds could be of help to you here as well. The 7950x CPUs do really well from a price / performance POV imho and could slot in there rather nicely.

I'm just thinking out loud here but a 7950x build would probably cost you around 1800€-2500€ and I'd expect the two of them to perform kinda sorta close to a 5995wx to make it all viable. Again, just thinking out loud.

That said one thing I noticed on my end here when I was setting up my own little render farm is that physical space eventually becomes a bit of a limiting factor - obviously it depends on the amount of room you have in your office but I'm currently very much mindful of that here :)

[jamesdowling]

I can't really comment on using the Mac studio as I have not used one but I recommend going for more than 64gb of ram, especially as this can't be upgraded at a later date with the mac. I usually sit below 64gb in projects, but there's always one that uses more. Every node will need to load the file and assets so if you are hitting the RAM limit on your current machine you will need at least that amount for each node. This will probably throw off your price/performance calculation for the mac.
If you wanted to go the old threadripper route it only makes sense to go for the 64core, as the platform is generally more expensive to build overall (motherboards etc).

The AMD Ryzen 9 7950X is probably the sweet spot at the moment for cost/performance and because you won't need a GPU you can build them in small form factor cases. It's scoring about 14,000 000 on corona bench.

I have gone down the small farm route in the past and hated it, sorting out networking, sharing drives, troubleshooting and licence costs was a real pain so I have opted for one powerful machine instead. Although I do a lot more gpu rendering now so thats played a big part in being able to move to one.

[superfluido]

Alright, thanks for the heads-up!

I added them to the list, and it went hands down straight to the top! It seems to be by far the most cost-efficient build and with 5 machines, it would make quite a big powerhouse for the price.
Also, I updated the prices with the ones I could find in another online shop configurator, and now Mac Studios do not seem so convenient any more but rather on-par with the Threadripper machines.

Here's the new list – I'll wait for the new 96 cores chips and update the list when they come out.
Ps. Was it so painful to keep a small render farm running? And do you do now GPU rendering with Corona??

[jamesdowling]

I found setting up the network really confusing, I'm solo and have built PC's for many years but that was one step too far for me lol. I had to get a network guy to come and help figure it out, the main issue was not having access to the router (I was in a shared office space). And sometimes machines would lose the network which became frustrating trying to troubleshoot when you have no idea about these things, Some people love networking though. No GPU in Corona, unfortunately (hopefully one day though!) I have been working for motion graphic studios a bit more so they are heavily C4D, redshift and I play around with different renderers in Max so I'm not CPU bound.