[solved] DR machines start and fail loop

[Vuk]

In middle of a deadline for some reason DR is in a loop now I tried it from my workstation, tried it from a node as a backburner server job with dr for other slaves same thing. DR machines are green as soon as the render starts and then simultaneously start going red and green and red so they won't start. The scene is heavy around 1.8 gigs but I rendered the same scene without any adjustments last night from my workstation and all was working fine until this morning when I tried again and the problem started. I even cleaned the scene up got it down to 1.2 gigs and the problem persists.

I am sure its not a network problem since all was working fine less then 12 hours ago. There was no Win update in the past days so that is excluded as well... Anyone experiencing the same problems?

[Frood]

I had DR loops back in v1.6.1 but never again since then. Which version is this?

Something in the logs? (slaves Max.log / DrLog.txt)

BTW: This sounds urgent, you may contact support through helpdesk:

https://coronarenderer.freshdesk.com/support/tickets/new


Good Luck

[Vuk]

Version 3.0 newest possible version. For some reason it just started working... Looking good so far fingers crossed got about 18 5k renders to render till the end of the day :) its gonna be tight!

[Frood]

Oh - glad to read! Favourable occasion for another


Good Luck ;]

[Vuk]

Ok so again the same problem in DR. Machines are parsing scene then starting then again parsing scene and go red connection failed.

Worked before without any issues in the same scene before I upgraded to 3.0. I had RC 4 or 5 of version 3.0 before and never had this issue. The scene is not even taking a 4th of total ram of the machines which are more or less all running with 64gb of ram. All the pats are UNC and from a Server. I had this issue less then 2 weeks ago I was restarting nodes, scene trying it for 2 hours and then all of a sudden it started working. Now I am again on a deadline for the same images and again same problem... The only solution now seems to just start render by render on each machine...

[Juraj]

Just like Frood mentioned, I used to suffer from this all the time, but it was some time ago, pre-v2 version.

I wonder if it's some particular feature in scene that sets it off, because I would have it happen only on every second scene, it's wasn't constant thing. Since all my scenes are kinda super heavy, it was impossible to pin-point by myself (or I never had the patience).

[Vuk]

Actually scene was kinda heavy the first time it happened and I got it down from 2gb to 1.2gb there is no plugin nothing in particular just floorgenerator, multitexture all is corona did find some vray displacement trough Forensic and deleted it but it still does the same thing. Funny thing is that this actually started the day 3.0 was released and I upgraded from RC to full version of 3.0...

[Frood]

Maybe rebooting slaves may help?

https://forum.corona-renderer.com/index.php?topic=22767.0

Not that I'm a fan of such kind of "solutions" but you take what you get when it's getting urgent.


Good Luck

[Vuk]

Tried that as well a few times actually even rebooting the OS. No help, tried rendering from different computer same thing. Rendered another scene which is the same project but a different scene with half of the same assets and it works. Apparently its something inside the scene that its bothering the DR. I guess that maybe merging all into a new scene would solve the problem but I was in such a rush that I just went with single machine rendering for each render...

[Jpjapers]

Ive experienced this recently with v2 and im certain its something to do with scene size as its only ever with a heavy scene and usually works the first time and then fails every time after.

[Vuk]

I am pretty sure it hasn't got anything to do with scene size. Its a simple house/interior scene the file is less then a gb... I have done several aerial/masterplans lately and worked with much much bigger and complex files and never had this issue.
I tried merging the complete scene in a new max file and the problem persists. I am now pretty sure that this is related to a single object/model or asset that I merged but since there are so many and the deadline is tight I have completely given up on it.

[Vlad_the_rant]

This is probably caused by a malformed object somewhere in the scene or a misbehaving plugin.
An option to try would be to convert as many objects to editable meshes as possible. Applying a "reset xform" before conversion to editable mesh could also help, the aim being to have all objects be just meshes, with no modifier stacks and at the correct scale since Max can and does sometimes freak out when something is wrong with the modifier stack.
Why, you may ask? Because when a Max file is loaded, what actually happens is Max rebuilds everything. This means taking the base object and applying all the modifiers to it with their parameters during loading, so when an object is just a mesh with no modifier stack, it will just load the mesh and there's less chance some modifier or something can get screwed up during the loading of the scene.

[Juraj]

That sort of searching is basically impossible with production scene with millions of objects.

Annoying culprit of this behavior is that those scenes render just fine themselves on both workstations or individual slaves. They only fail in DR mode on slaves.

I don't remember which scene of mine did it but maybe VUK could upload his (though I am sure I uploaded mine as well back then).

[Jpjapers]

That sort of searching is basically impossible with production scene with millions of objects.

Annoying culprit of this behavior is that those scenes render just fine themselves on both workstations or individual slaves. They only fail in DR mode on slaves.

I don't remember which scene of mine did it but maybe VUK could upload his (though I am sure I uploaded mine as well back then).

Agreed, it works fine on my workstation too.
I dont want to have to collapse my stack just to render something on the farm reliably. Its far too destructive a workflow.

The solution to the problem shouldn't be to change the way we work.

[Vlad_the_rant]

The solution to the problem shouldn't be to change the way we work.

Agreed, but if the problem is sporadic and there's a tight deadline, collapsing everything may be the only option, just for that particular job.