Creating a render farm

[julecocq]

Hi Corona Community,

Just would like to talk about setting up a mini render farm, with Corona.

Right now I have the 3ds Max: FairSaaS licence, with 3 render nodes included.

I don't have much knowledges about hardware. So my first question would be, what would you suggest in terms of hardware, to be able to use these 3 render nodes ? Do I need 3 complete machines (with graphic cards, RAM and everything) ? or is there other solutions, like having 3 CPU somehow ?

Then, I know there is a topic about it at coronarenderer.freshdesk.com, but we don't really need to get 3dsMax installed on each node right ?

Also, we need a Operating System on each node ?

You see I'm really dummy regarding this topic...

Thx for your help guys !

[Juraj]

- Any hardware you can get :- )
- Every node needs to be a complete PC, with exception being rackmount with shared PSU. Not very common thing.
- There are dual and quad-socket server PCs utilizing multiple CPUs, if you want to cram as much performance into as little space as possible. A lot has been written about in HW section.
- Each PC needs Windows (because 3dsMax), 3dsMax (because Autodesk) and Corona. In future, there is possibility of 3dMax-less network rendering using Standalone Corona version.
  You don't need to licence the 3dsMax for network rendering (it can stay in trial mode, you won't be able to use it to work, but for DR and Backburner yes), but you do need Corona licence of which you have 3 for this.

[FrostKiwi]

I'd like to add, that you do not need a graphics card, just an integrated GPU will do it.
Also you can totally cheap out on storage.
You only need to start Max and backburner from the Hard drive. All textures are being downloaded into ram.
So technically you can use a USB drive if you are very short on money. It won't matter for performance.

[Juraj]

So technically you can use a USB drive if you are very short on money. It won't matter for performance.

Let's not get carried away ;- )

[TomG]

I'd suggest using some form of remote desktop to control the other PCs, then you don't need monitors and keyboards and can have access to them all from one central machine. "Pro" versions of Windows feature this built in if you enable it, but there are alternatives, e.g. Google has a free remote desktop system. You can move your one main monitor / keyboard / mouse to the machine for initial set up, then just leave it as a box on its own after that :)

For nodes, any hardware will do, personally I find refurbished dual Xeons to be good for price / performance, I went for older ones due to budget, but even a $500 machine can perform well. Graphics card is irrelevant, so you can go without if you get the option in the system you are buying, or go for the cheapest, as it has no role to play.

You will need Max installed on each node, but it doesn't have to be licensed - this means you can't run Max on the node (ie open it as a program and use it) but you won't need to. As noted, you will need an OS for each machine too of course.

Hope this helps!

[maru]

I have added a link to this thread to the general DR guide in our helpdesk. There are some basic facts listed here using plain words, so why not.

I would also add that you can network-render using:
a) Corona's DR only - this can be used for "all PCs are rendering one frame" scenario. Apart from 3ds Max, Autodesk Backburner is also required for this to work.
b) Backburner - this can be used for "each single PC is rendering one single frame" scenario, or for rendering one image as strips.
c) Other software

[julecocq]

Thx a lot guys !

Just one thing I don't understand.

How can I use Max if I don't have any licence (except one on my main workstation) ?

[TomG]

Just run the installer and there you go. If you run it, it will ask to be activated, but you never need to run it.

I usually download the installer on my main machine, then just copy it over to the other machines, run it, never run Max, and presto!

[julecocq]

aaaah ok !

:)

really nice

What about having Corona Stand Alone instead of Max ?

[maru]

Corona Standalone cannot open .max files and that is required. You are basically rendering with Max. You need Max. (or any other supported application in the future)
More specifically it's 3dsmaxcmd.exe - command line version of 3ds Max.

[TomG]

Never tried it myself - you would of course then have no access to any plugins that you used in Max for one thing. Beyond that, not sure what the difference would be. Harmless enough installing Max, and it is intended to be used that way by Autodesk (which is why they include Backburner with Max) so I never looked into any alternative approaches myself. Maybe someone else can add more on that.

EDIT - someone else has added more info just as I was posted, brilliant!

[julecocq]

thumb up guys

[jasond]

Great timing on this as we're looking to set up a farm as well!

What's your take on having many fast i7 nodes compared to a few nice Xeons? Our Autodesk reseller suggested this along with possibly using blade servers. We're currently running 10 Xeon E5-2620 machines but can only use them 5pm - 7am which isn't ideal.

[TomG]

Blade servers could be interesting, but I got cold feet when looking into the option as I wasn't sure of the technical aspects involved in setting that up.

For the i7 solution, just my take. I find that the i7s would be great if you are using the machine for something else too, so as a workstation and then a render node in its downtime. If the machine is only to be used for rendering, I suspect the dual Xeons are the way to go.

As a 'for example' my 3 year old workstation is an i7 3930K overclocked from 3.2 to 4.2, pricing on that processor still seems to be around $500 to $700 on its own (not looked into systems). I bought an older refurbished dual Xeon system (X5650s)for that price, about $550, and it is 1.5 times faster at rendering. Not sure if the same holds up on more recent i7s and Xeons, but I suspect so - a dual Xeon system if it is just a render node is most likely going to give better price / performance ratio.

Hope this helps some!

[maru]

What's your take on having many fast i7 nodes compared to a few nice Xeons? Our Autodesk reseller suggested this along with possibly using blade servers. We're currently running 10 Xeon E5-2620 machines but can only use them 5pm - 7am which isn't ideal.

This: https://forum.corona-renderer.com/index.php/topic,7722.0.html
Or start a new thread in that section. :)

[Juraj]

My 70 euro x2 for dual Xeon E5-2670 v1 should arrive tomorrow hopefully, and I already have the rest. Will post my build to see how it fares.

You can actually buy them instead of 3930k i7 and put them into single-socket motherboard :- ) You will then pay 70 euro instead of 500+ euro for the same performance :- )

For anything else, read the thread Maru linked. This question has been asked 50 times since this forum was created and there are simply enough answers by now.

[jasond]

Thanks very much for the tips, that thread is quite helpful as are the other posts in the Hardware forum!! I'm going through it closely with our IT guys as they'd be having all the fun installing :( They are mainly trying to make the business case for building a dedicated blade or a stack of PC's that we can use 24/7. Will post in the appropriate forum next time :)

[Juraj]

Blade servers are the thinnest units meant into Blade enclosure. They're extremely expensive, noisy and require external cooling. They usually can't hold the most performance oriented units which top out at 145W each CPU. They're just meant for massing huge density in clusters. Useless for anyone else.

Then then are simply rack-mount servers, which can host common 4U chassis which are almost as roomy as regular ATX cases ( "almost" being the key, you will be using 80-100mm fans inside, so they're gonna be MUCH louder :- ) ... ) This is good density oriented solution for medium-to-big sized company.

But you can just buy the top range v4 Xeons and simple 4x ATX-XL cases for example ('Big towers') will yield huge performance (like 25 000 R15 Cinebench points, so roughly 25x 6core i7s ). Rack is not such a necessity and it's loud...and hot. For smaller studios a stack of normal cases should suffice 100perc. time.

Unless you are single freelancer on very low budget don't buy multiple i7s...or some IKEA Helmer case nonsense :- ) It's not worth to have full room of boxes and managing it to save some 30perc. of budget in value/performance. If you can have 40 cores/80 threads in single silent tower, with single licence for software, that...is amazing.

[jasond]

Fantastic information Juraj, thank you very much! We have a large server room here with several racks that could hold either the blade enclosure or 4U servers but like you say, a stack of Xeon towers seems to be the best bang for the buck.

Thanks again.

[danielmn]

Hey guys,
thanks for all the great info.  Just wanted to add my 2 cents and add a question.

As far a GHZ speed between processors I have seen very little difference.   I have gone from a 6 core 3.4ghz to a 6 core 4.2 ghz and only got about 10% difference on the benchmarks..... After I saw this I throttled down the Overclock... there was no reason to run the risk of an OC and power consumption.


Quesiton:
What is your take on RAM consumption if I was too only rely on DR rendering...several machines working on one stil.... Can the main machine have all the ram and the rest just have bare minimum RAM.... just wondering, what your thoughts were.

background
I am building a dual 8 core cpu machine.  But I was thinking of buying some cheap $200 all ready built HP quad processor machines.  The only downside with these machine they have very little ram.

[cgifarm]

Hey Guys, having a small renderfarm is cool for small and fast projects. Problem is that if you would like to scale this up in the future, you will hit the following problems:

1. Provisioning your computers with OS's and installing applications + plugins takes a LOT of time, major software install must be performed on each server if you don't have a good provisioning solution which will make it easy to create an image of one of your machines and install that on the other machines.
2. Electricity upgrade, your computers will begin using lots of electricity when they are rendering, and you must have a good installation so you avoid any cable meltings or blown fuses.
3. You will begin doing server administration instead of focusing on your projects, which I believe will lead into loosing more money than actually getting by using your servers.
4. Heat issues - you will have to ventilate the room where you keep the servers properly, AC could also be needed.
5. Licensing, you will have to purchase or rent licenses for all your nodes.
6. Networking bottleneck - this happens if you have 20 servers plus I guess, or very big scenes and your 1GB connection won't be enough and you need to go on 10 GB Fiber Optics, which ad up to the costs a lot.
7. Your project assets must have proper paths so all your nodes can find the assets, otherwise you will have missing textures or not properly set outputs, this can be solved by only working on your network storage I guess and using network paths.
8. If rendering big res images, all your nodes must have enough ram, like 48 - 96 GB if you are doing 8-12 K res. Having multiple nodes with poor performance won't do anything on these monster res images.

It can be a fun project, but I would advise giving our farm a test. We offer $20 worth of credits, enough for you to make an impression. Here's a quick video on how our process works.

https://www.cgifarm.com/renderfarm-quick-start-guide


Good luck with your project guys! My project also begin like building something small for my studio, but I ended up focusing only on the renderfarm in the end, it had to be one or the other :).

[Tex3D]

Hey everyone, I’m super new to using Corona (I bought it today!) and I’ve already started setting up a small 3 node render farm. Can someone please explain to me the texture situation?

So I start max and load my scene and then send it to render via Corona. Do I have to have the texture maps on each computer (in a special folder) or does the main one send the textures and lighting info to each slave node assuming DR is set up correctly? I keep seeing people talk about correct paths and such so it sounds like all textures need to be on each machine? Is this right?

Thank you for any help!
Dave

[Frood]

Hi and welcome :)

Just some short notes:

1. You will be using DR and Backburner (and maybe deadline) for sure with 4 machines sooner or later (single node processing gives you better quality in less time generally). So besides drserver, you may consider also to install/use Backburner server on all computers.

2. Current Corona DR usually sends missing assets to the slaves but you may end up with a lot of files unneeded on your slaves and I consider it more as fallback than something to rely on (in case of 3 slaves, all assets would need to be copied 3 times around).

3. With no dedicated server, I personally would create a share on your main workstation where all assets can be accessed in your network and create scenes using UNC paths even when working locally. Means: If you are working on your main computer "mybox", when linking assets it would get "\\mybox\textures\shinywood_d.jpg" and not "D:\data\textures\shinywood_d.jpg" for example. "textures" would be shared in this case on "mybox". This way you ensure that scenes work everywhere in your network.

4. If you will be running drserver or Backburner as windows service, you would have to use UNC paths anyway (this is because usually the operating system as "user" has no access to mapped drives).

5. I would run corona license server in your network (preferable also as windows service) for convenience.

6. Don't forget to check firewall settings in case something goes wrong, disable firewalls temporarily to check if it's the cause of the problem.



Good Luck

[Tex3D]

Awesome reply! Thank you so much for your help!!

-Dave

[3dvizual]

If you where to build a small renderfarm today how would the best setup be?

What I can read is that the Rack solutions is loud, noisy and produces a lot of heat.

How would you create a small renderfarm in your studio (maybe 4 pcs)? What will be the specs on each render slave pc.

Will you use AMD Threadripper 2990wx 32 cores / 64 threads - 96GHz in total or a Intel setup?

What graphicscard? And how many RAM in each slave. And which motherboard