Author Topic: In-house Render Farm  (Read 1119 times)

2024-09-19, 14:07:38

acrobaticpoly

  • Active Users
  • **
  • Posts: 33
    • View Profile
    • AcrobaticPoly
Hey Guys,

I'm looking into creating some sort of renderfarm to speed up our process in our studio.

We are a team of 3 based in the UK with one collegue working remotely overseas in Europe - we all have very capable machines.
We're finding ourselves doing more animation and are currently relying on outsourcing the rendering workload to an external renderfarm, but I would like to know what the cost would likely be if we where to build one in house.
We use 3ds Max and Corona and would most likely use Pulzes Render managment system to send jobs out. I'm now not sure if it's better to just add 2 more machines to the mix with Threadrippers and 128gb each or go down the server side of things.

Any help would be much appreciated!
Website: www.acrobaticpoly.com
Photos: https://vsco.co/egyptianblue/gallery
"I am out with lanterns, looking for myself."

2024-09-19, 14:47:59
Reply #1

Nejc Kilar

  • Corona Team
  • Active Users
  • ****
  • Posts: 1300
    • View Profile
    • My personal website
Probably the most often asked question for any studio embarking on the animation journey, I feel ya.

My mini render farm is made up of server gear because two reasons - it is the most powerful stuff you can get (if you go 2 socket style) + if you buy used server CPUs it is also relatively affordable. There are however potentially a ton of gotchas with them and unless you're into hardware I would suggest either hiring someone to do it for you or opt for pre-builts or Threadrippers.

When it comes to server gear there are tiny gotchas all over the places potentially - such as in my case where I've bought 2x Epyc 7B13s that run on a single motherboard but because it is a server motherboard the VRMs were overheating pretty much constantly. That is because server gear is designed for server like airflows which is often directed using shrouds and so putting that kind of gear into a normal case (even an airflow focused one like the Fractal Torrent)... Might not be optimal.

That said I managed to get it working by simply wedging a fan between the two CPU coolers and have it point downwards towards the VRMs. That solved the issue in its entirety.

Point is, its a gotcha.

The other gotcha is running regular Windows on these types of configurations - one of my LAN ports doesn't seem to work properly and I'm assuming its the drivers. All the motherboards I've seen for dual Epycs are "rated" for Windows Server and the like.

On top of that you get no fan curves or anything so its often a choice between "tornado " mode and "kinda ish silent but do keep the AC on" mode.

Oh and as a further gotcha, I'm running Noctua fans on the thing and the motherboard constantly thinks they are dying because of the "low" RPM they run at. 1500 RPM is nothing in server world (stuff gets to ~7000 RPM over there) but for us normal folks its already rather loud. So the only way I found to resolve that in my situation is to run them full speed all the time or have a separate fan hub installed (the built in one in Torrent or Meshify or Define works fine, just don't plug the fan readout to the motherboard). Otherwise the fans continuously ramped up to full speed and go back to 0. I think the motherboard was doing CPR to them or something :)

Anyhow, hopefully that illustrates some of the "difficulties" working with server gear.

On the flip side however, a used 96 core 9654 will cost you around 2500€ (around 5000€ if you buy new). You get two of those and you've got some solid power on your hands - and you'll still be below 10k for the whole build whereas a single 96 core 7995wx will cost you 11k.

Building a Threadripper seems easier to me because you've got all the consumer gear available to you and the motherboards tend to have wider compatibility with stuff. As long as you chose compatible components and you don't break them when putting the system together... You should be good. It is a little bit more advanced than consumer stuff in terms of setting it up but imho if you can build and setup a Ryzen / i9 system, you'll be able to do it.

Then there are the prebuilts, probably the most reliable option because yes, you'll pay more (sometimes a lot more) but if something is wrong you typically can call the company that built them and they'll fix it all up for you. Unless you run into one of "those" companies.

I opted for used Epycs and I plan on continue buying them. I have a relatively small office space and with 5 systems I'm already kind of running out of physical space so for me getting the most out of a system (in terms of speed) is paramount.
That might or might not work for you. Do note I'm also super nerdy about hardware so running into gotchas with desktop Epyc setups is ... A bit frustrating but also rewarding once I'm able to solve things.

Hope this wall of text is at least a little bit helpful.

Oh and also, something to consider is that currently some of the higher core count 2S (two CPUs in one motherboard) systems are not quite working as expected in Corona. It is being looked at but just so you know.
« Last Edit: 2024-09-20, 12:00:52 by Nejc Kilar »
Nejc Kilar | chaos-corona.com
Educational Content Creator | contact us

2024-10-01, 12:44:42
Reply #2

mase

  • Active Users
  • **
  • Posts: 97
    • View Profile
Since I'm in a similar position to OP and we're on the right thread already I would like to ask what's the best long term solution for storage?

It is no surprise that arch vis projects with all the assets, scenes, textures and CXRs take up a lot of disk space. And it's also important to keep these as archives in case something comes back in the future. So without getting too technical - what's the best storage solution you would recommend, that could also be expanded as time goes on?

2024-10-01, 13:12:21
Reply #3

clemens_at

  • Active Users
  • **
  • Posts: 146
    • View Profile
Since I'm in a similar position to OP and we're on the right thread already I would like to ask what's the best long term solution for storage?

A Nas. Whether its a prebuilt system from vendors like Qnap, Synology, TrueNas ect, or your own server running linux, unraid, TrueNas, ect.

2024-10-14, 11:44:13
Reply #4

acrobaticpoly

  • Active Users
  • **
  • Posts: 33
    • View Profile
    • AcrobaticPoly
Hope this wall of text is at least a little bit helpful.

Thank you so much for such a great answer, there is a lot here to digest and to consider!

We're currently looking at getting threadrippers and as we have a dedicated IT company looking into the matter too, they've also sugested putting some of the budget into storage too.

I'll keep you updated on the progress of this intervention! :D
Website: www.acrobaticpoly.com
Photos: https://vsco.co/egyptianblue/gallery
"I am out with lanterns, looking for myself."