Author Topic: Deadline 10.1.17.4 3ds max SMTD jobs failing - anyone else?  (Read 9851 times)

2021-08-05, 11:07:42

alexyork

  • Active Users
  • **
  • Posts: 699
  • Partner at Recent Spaces
    • View Profile
    • RECENT SPACES
Hi folks,

Is anyone else having issues deploying Deadline 10.1.17.4? The 3ds max SMTD submitter is failing to send most but not all scenes, either via 2020 or 2022. Something about Python error, but not clear at all from the logs what's going on...

No joy on the Deadline forums yet, and there's currently no working version that supports 2022 we can roll back to! Seems like it's happening on the previous 16.9 version too:

https://forums.thinkboxsoftware.com/t/deadline-3ds-max-2022-python-exception/28039/21

Cheers,
Alex York
Partner
RECENT SPACES
recentspaces.com

2021-09-09, 09:19:37
Reply #1

Alex Abarca

  • Active Users
  • **
  • Posts: 416
  • Corona Certified Instructor
    • View Profile
    • Instagram
Just saw this.
I'm having the same issue with Deadline 10.1.18.4 + Corona 7
Is this the error you're getting?
Error: FailRenderException : RenderTask: Unexpected exception (Exception caught in 3ds max: -- Syntax error: at keyword parameter, expected <factor>


2021-09-09, 09:34:29
Reply #2

alexyork

  • Active Users
  • **
  • Posts: 699
  • Partner at Recent Spaces
    • View Profile
    • RECENT SPACES
Yep, same. As a result we jumped ship to Pulze and are never looking back!
Alex York
Partner
RECENT SPACES
recentspaces.com

2021-09-09, 09:41:08
Reply #3

johan belmans

  • Primary Certified Instructor
  • Active Users
  • ***
  • Posts: 761
    • View Profile
    • belly.be
Still running Deadline 10.1.16.9 + Max 2022 and Corona 7 without any issues.
edit: and in combination with RP Manager

2021-09-09, 09:49:03
Reply #4

Alex Abarca

  • Active Users
  • **
  • Posts: 416
  • Corona Certified Instructor
    • View Profile
    • Instagram
Hi Alex...
Yeah, Pulze is nice! But my only drawback is that it only works with 3dsmax. I need Blender in the pipeline.
I'm trying some things and hopefully, I'll get to the root of the problem.

Johan, that's great to hear! Hopefully, I can get my problem resolved, I would like to know what's causing it. I'm not sure if it's coming from 3dsmax, corona, or deadline.
« Last Edit: 2021-09-27, 20:17:17 by Alex Abarca »

2021-09-09, 10:08:09
Reply #5

johan belmans

  • Primary Certified Instructor
  • Active Users
  • ***
  • Posts: 761
    • View Profile
    • belly.be
Did you try to downgrade to Deadline  to version 10.1.16.9?
In this way you can determine if Deadline is causing the issue you have.

I am not sure if the following could be the problem, but in the realease notes of Deadline  10.1.17.4:
"...Application Plugin Improvements
3ds Max

Added Corona 3, 4, 5, and 6 parameters to 3dsMax.

Corona Standalone

Added support for Corona 6 Standalone.

...."

Corona 7 is not mentioned, nor in the realese notes of Deadline 10.1.18.5

2021-09-09, 20:21:15
Reply #6

Alex Abarca

  • Active Users
  • **
  • Posts: 416
  • Corona Certified Instructor
    • View Profile
    • Instagram
Hi Johan,
Not yet, but I will downgrade if the problem persists. I am testing a lot of scenarios at the moment, for example, I'm testing a file where I am slowing appending items like geometry, then camera, then render params, etc.

One of the nice things about Deadline is that it's really zippy/snappy and getting access to cloud rendering in conjunction with local machines is crucial. 

2021-09-10, 16:14:52
Reply #7

johan belmans

  • Primary Certified Instructor
  • Active Users
  • ***
  • Posts: 761
    • View Profile
    • belly.be
yeah, the cloud rendering service in Deadline is great. We had it running for a while. But in the end we didn't needed it that frequently and where paying Amazon every month a fee to keep the service setup. At that time it wasn't an easy setup.
But we recently discovered Pulse, a Deadline Client Application. Now we do not have to keep an eye on our "renderworkers". They will switch on automatically whenever the repository receives a new renderjob and workers will be shutdown if there are no active render jobs.

Hopefully you can narrow down your problem.

2021-09-24, 01:08:40
Reply #8

virtualvision

  • Users
  • *
  • Posts: 1
    • View Profile
I was having the same problem on Release 10.1.18.4.
As per Johan's suggestion - rolling back to 10.1.16.9 has fixed the problem.
Max 2022. Corona 7. RP Manager submission.
Spent a few hours on this and was completely stuck - thanks Johan for your suggestion.

2021-09-24, 10:59:05
Reply #9

johan belmans

  • Primary Certified Instructor
  • Active Users
  • ***
  • Posts: 761
    • View Profile
    • belly.be

2021-11-14, 13:43:13
Reply #10

johan belmans

  • Primary Certified Instructor
  • Active Users
  • ***
  • Posts: 761
    • View Profile
    • belly.be
FYI: We have updated Deadline to release 10.1.19.4 and all runs smoothly

edit:
not that smoothly ..... anyway, some jobs render fine other do not. But there is a workaround.
Jobs which do not render when submitted from inside 3ds Max, will render when submitted from the Monitor by Submit/3d/3ds Command or by Submit/3d/3dsmax

I reported this on Deadline's forum:
https://forums.thinkboxsoftware.com/t/deadline-10-1-19-4-3dsmax-2022-1-corona7h1-some-render-jobs-only-render-when-submitting-from-deadline-monitor/28597

« Last Edit: 2021-11-16, 17:30:26 by johan belmans »

2021-11-24, 15:20:01
Reply #11

Alex Darby

  • Active Users
  • **
  • Posts: 77
    • View Profile
    • Visoo
We moved over to Pulze as there was no official support for Corona 7 even after a year of being released.
I asked numerous times for help on the forums with no reply.
I saw a change with customer service once they where bought by Amazon. There seemed to be more interest in pushing amazon cloud rendering than anything else.

Pulze was a bit touch and go at the beginning of this year as they where still ironing stuff out, but now we are super happy and have been for months.
They are also very responsive and hands on with customers.



2021-11-30, 11:02:30
Reply #12

johan belmans

  • Primary Certified Instructor
  • Active Users
  • ***
  • Posts: 761
    • View Profile
    • belly.be
Hi Alex,

I agree with the poor support of Thinkbox or Amazon.
But still deadline is a powerful solution for render management in Max.

Update about fixing the issue in Deadline:

A better solution in the meanwhile is to reset the Corona Render settings.
I never thought we would make use of the "Reset Settings" button in the Render Setup menu of corona in the Scene tab.
But if you do so, you can send renderjobs to deadline from Max again.
All problematic files, I have tested so far, which did get errors in the past render fine now.

2021-11-30, 23:54:04
Reply #13

pavelhuerta

  • Active Users
  • **
  • Posts: 18
    • View Profile
Hi Guys,

I'm experiencing the same problem with deadline and Corona Renderer 7 using 3ds Max 2022. It seems that reset settings works and I noticed that with just change from UHD Cache to Path Tracing the render goes normally. Someone else can confirm if that works for you too?

Kind regards...

2021-12-02, 11:46:59
Reply #14

alexyork

  • Active Users
  • **
  • Posts: 699
  • Partner at Recent Spaces
    • View Profile
    • RECENT SPACES
Good to know! We have totally switched to Pulze, but it's good to know there's hopefully knowledge there about the issue now. I wonder if this comes down to Thinkbox or Corona devs to fix?

Cheers,
Alex York
Partner
RECENT SPACES
recentspaces.com

2021-12-03, 16:21:00
Reply #15

johan belmans

  • Primary Certified Instructor
  • Active Users
  • ***
  • Posts: 761
    • View Profile
    • belly.be
Next update about fixing the issue:

It seems that the file path for UHD caching is the trouble maker, even when UHD cache is not used.
If you remove the default path C:\HdCache.hdc, the file will render when submitted from MAX.


2021-12-06, 10:22:56
Reply #16

alexyork

  • Active Users
  • **
  • Posts: 699
  • Partner at Recent Spaces
    • View Profile
    • RECENT SPACES
Is this an issue caused by Corona therefore, rather than Deadline? Wondering who's responsibility it lies with to fix.
Alex York
Partner
RECENT SPACES
recentspaces.com

2021-12-07, 19:27:08
Reply #17

johan belmans

  • Primary Certified Instructor
  • Active Users
  • ***
  • Posts: 761
    • View Profile
    • belly.be
Is this an issue caused by Corona therefore, rather than Deadline? Wondering who's responsibility it lies with to fix.

Always "silent" happens when devs (no matter which company) have to answer such a question ;-)

2021-12-09, 15:54:01
Reply #18

maru

  • Corona Team
  • Active Users
  • ****
  • Posts: 12711
  • Marcin
    • View Profile
Is this an issue caused by Corona therefore, rather than Deadline? Wondering who's responsibility it lies with to fix.

Always "silent" happens when devs (no matter which company) have to answer such a question ;-)

We really don't want this. Sorry for the delay here.

To anyone who is having some Corona + Deadline issues:
Can you please submit a support ticket at https://corona-renderer.com/link/contact-support
Please include the following info:
- which exact version of Windows, 3ds Max, Corona, and Deadline you are using
- what exactly is happening (e.g. I am sending a job to the farm and rendering is not starting on a node)
- any logs you can connect (e.g. Deadline logs and/or Corona DR logs if you are using Corona DR)
- if you see any kind of error message / notification - please also include a screenshot

Once this is done we will reply, possibly with some further questions, and once we understand the issue, we can contact Deadline support about this to find out on whose side the culprit is.

These kinds of investigations are complex (similar like with 3rd party plugins), but we did it in the past and it can work out well.

Reporting on the forum is great, it makes us see that there is some issue, but we may easily miss some important information and following a thread with many replies is inconvenient.

Marcin Miodek | chaos-corona.com
3D Support Team Lead - Corona | contact us

2021-12-10, 10:53:40
Reply #19

alexyork

  • Active Users
  • **
  • Posts: 699
  • Partner at Recent Spaces
    • View Profile
    • RECENT SPACES
Hey Maru,

Good to know you guys are on the case. I would point you to the link in my original 1st post here in this thread, which links to the Thinkbox forums, as it goes into detail about specific errors and pointers to the issue, not just from me/RS but from other people/studios.

Cheers,
Alex York
Partner
RECENT SPACES
recentspaces.com

2021-12-11, 17:36:02
Reply #20

johan belmans

  • Primary Certified Instructor
  • Active Users
  • ***
  • Posts: 761
    • View Profile
    • belly.be
Is this an issue caused by Corona therefore, rather than Deadline? Wondering who's responsibility it lies with to fix.

Always "silent" happens when devs (no matter which company) have to answer such a question ;-)

We really don't want this. Sorry for the delay here.

To anyone who is having some Corona + Deadline issues:
Can you please submit a support ticket at https://corona-renderer.com/link/contact-support
Please include the following info:
- which exact version of Windows, 3ds Max, Corona, and Deadline you are using
- what exactly is happening (e.g. I am sending a job to the farm and rendering is not starting on a node)
- any logs you can connect (e.g. Deadline logs and/or Corona DR logs if you are using Corona DR)
- if you see any kind of error message / notification - please also include a screenshot

Once this is done we will reply, possibly with some further questions, and once we understand the issue, we can contact Deadline support about this to find out on whose side the culprit is.

These kinds of investigations are complex (similar like with 3rd party plugins), but we did it in the past and it can work out well.

Reporting on the forum is great, it makes us see that there is some issue, but we may easily miss some important information and following a thread with many replies is inconvenient.

hi Maru,

I will submit a ticket.

For the moment the issue is pinned down to the following:

WHEN a Corona 7hf1 render job (Max2022.3 in my case) is sent to Deadline (all newer releases after 10.1.16.9)
AND the default UHD cache path isn't touched (C:\HdCache.hdc).
THEN the job will appear in Deadline monitor, will get assigned to a worker. The worker will start the job, and depending on the heaviness of the scene will quit after a while. The Corona VFB does not pop up. In the end the job will not render and no error logs in Deadline will appear to pin point the issue.

Even if you send a corona render job which only make use of Path Tracing for primary and secondary solver, the job will not render as long the UHD cache path is  C:\HdCache.hd

If you reset all corona settings or remove the default UHD cache path, the job will render.

All the releases of Deadline prior to and 10.1.16.9 included didn't suffer from this issue.

I cannot verify if Corona DR is suffering from this issue, as we do not use DR.

EDIT: I tried to submit a "ticket", but can only submit a "request"....
« Last Edit: 2021-12-11, 17:50:54 by johan belmans »

2021-12-13, 13:16:28
Reply #21

maru

  • Corona Team
  • Active Users
  • ****
  • Posts: 12711
  • Marcin
    • View Profile
EDIT: I tried to submit a "ticket", but can only submit a "request"....

Yes, we had some changes done to our support system and it is now called "submitting a request". Request, ticket, question, message, case, it's the same thing. :)
You can do it here - https://support.corona-renderer.com/hc/en-us/requests/new
Or you can simply send an email to support@corona-renderer.com - the end result will be the same.

Thanks in advance!
Marcin Miodek | chaos-corona.com
3D Support Team Lead - Corona | contact us

2021-12-13, 14:29:15
Reply #22

Frood

  • Active Users
  • **
  • Posts: 1903
    • View Profile
    • Rakete GmbH
If you remove the default path C:\HdCache.hdc, the file will render (...)

Just for reference:

LUTs, Lightcache, IES files - even if not used/activated but only referenced somewhere in the UI - are required assets using Corona for some reason. It has already caused trouble earlier:

https://forum.corona-renderer.com/index.php?topic=22054.0

And it spams log files with ugly unnecessary warnings, which is worse ;]


Good Luck




Never underestimate the power of a well placed level one spell.

2021-12-13, 16:01:09
Reply #23

maru

  • Corona Team
  • Active Users
  • ****
  • Posts: 12711
  • Marcin
    • View Profile
(Report ID=CRMAX-720)
Marcin Miodek | chaos-corona.com
3D Support Team Lead - Corona | contact us

2021-12-13, 16:01:37
Reply #24

maru

  • Corona Team
  • Active Users
  • ****
  • Posts: 12711
  • Marcin
    • View Profile
Has anyone submitted a support ticket/request yet? If so, can you please share your ticket number with me?
Marcin Miodek | chaos-corona.com
3D Support Team Lead - Corona | contact us

2021-12-13, 17:48:24
Reply #25

johan belmans

  • Primary Certified Instructor
  • Active Users
  • ***
  • Posts: 761
    • View Profile
    • belly.be

2021-12-15, 13:10:12
Reply #26

maru

  • Corona Team
  • Active Users
  • ****
  • Posts: 12711
  • Marcin
    • View Profile
This is just initial info but we contacted Deadline and they identified a potential issue and are working on a fix now. Will be notifying you on the progress.
Marcin Miodek | chaos-corona.com
3D Support Team Lead - Corona | contact us

2021-12-16, 00:06:03
Reply #27

pavelhuerta

  • Active Users
  • **
  • Posts: 18
    • View Profile
Hi guys,

seems the staff of aws thinkbox have posted the fix for the customize.ms script in the repository, check in the next link:

https://forums.thinkboxsoftware.com/t/deadline-3ds-max-2022-python-exception/28039/65

I have not tested the fix yet, but soon I'll do.

Greetings!

2021-12-16, 10:49:09
Reply #28

alexyork

  • Active Users
  • **
  • Posts: 699
  • Partner at Recent Spaces
    • View Profile
    • RECENT SPACES
I have to say, it's truly pathetic how little the Deadline devs seem to care about issues their customers bring to their attention. I can't even remember how many weeks and months it's been since a bunch of us raised the issue with them and their lacklustre response on their support forum says it all. I even went through their official support channels first (of course) but received zero reply. You're forced to buy some kind of super premium extra AWS support yadayada which is no doubt a waste of money. It shouldn't take two devs speaking to eachother to get something like this looked at, although obviously we're really grateful to Corona devs for pushing them! It just shouldn't be necessary. This is why we jumped to Pulze. But Deadline offers so much more that we miss... so... what can you do.
Alex York
Partner
RECENT SPACES
recentspaces.com

2021-12-17, 09:21:25
Reply #29

maru

  • Corona Team
  • Active Users
  • ****
  • Posts: 12711
  • Marcin
    • View Profile
To everyone with Corona+Deadline issues:
As noted by Pavelhuerta, please visit this link and follow the instructions there: https://forums.thinkboxsoftware.com/t/deadline-3ds-max-2022-python-exception/28039/65

Please let us know in this thread if it's working fine for you or not.
Marcin Miodek | chaos-corona.com
3D Support Team Lead - Corona | contact us

2021-12-17, 17:38:07
Reply #30

johan belmans

  • Primary Certified Instructor
  • Active Users
  • ***
  • Posts: 761
    • View Profile
    • belly.be
Just tested the patch were Maru is referring to and I can confirm the issue (I had so far) is solved.

2021-12-23, 17:30:29
Reply #31

edwin.amsler

  • Users
  • *
  • Posts: 2
    • View Profile
Hey everyone. I'm Edwin. I've been Thinkbox's support department since December 10, 2010 and I've worked with the same team from before we founded the company. This problem has been rougher on the community than we realized and I think our difference in support methods has caused a delay and I want to make sure we're doing better.
 
At the moment, our forums are community driven and we rely instead on support tickets and phone calls as our eyes and ears as to what's happening in the community. I have initiatives to change this, but there are a few things to do before that happens.
 
That said, we should *never* drop a support case and we have mechanisms and tools to prevent that from happening. @alexyork if you could e-mail your ticket id to me at amsler@amazon.com I'll dig in and find out what happened. You may be enjoying Pulze now but I take this seriously and want to find out what happened. The same goes for anyone else here, and if anything is being missed our phone number is +1-866-419-0283 (5-9 American central time). It's on the website as well.
 
Now, this problem in particular was a hard one to narrow down. As you've all discovered (which helped us finally fix this problem) it was how we were parsing the "corona_gi_giCache_file" parameter that is sent along with the job, and everything works fine until that's set to a file path. The fact that a dev reached out was helpful but we need to congratulate johan on December 3rd who truly helped crack the case.
 
To dig a little deeper, we have code that pulls render settings at submission and re-applies them at render time. That drives a powerful feature to allow users to make renderer-specific tweaks to a scene without re-opening Max. The issue came up because we had the wrong data type mapping on whether a parameter should be parsed (for numbers) or not (for text, file paths, etc). When that code tries to parse a file path, it throws the entirely unhelpful "Syntax error: at keyword parameter, expected <factor>" error which had us confused for so long. Correcting the mapping fixed the bug that came in with the new Corona 6 properties.
 
I appreciate the help of both communities here and in our forums. It's time my team offer the same level of support there as we do in our ticket and phone systems.

2021-12-24, 14:53:12
Reply #32

johan belmans

  • Primary Certified Instructor
  • Active Users
  • ***
  • Posts: 761
    • View Profile
    • belly.be
Hi Edwin,

thanks for your reply in this thread. It would be great if you and your team could keep an eye on the Deadline forum to sort out this kind of issues.
Although I can image it will take a lot of your time browsing to all the posts over there.
But hey I am still a Deadline fan. Keep up the good work!

Merry Christmas

2021-12-28, 17:03:44
Reply #33

edwin.amsler

  • Users
  • *
  • Posts: 2
    • View Profile
Agreed. Again, still some things I need to sort out there. One dimension I'm struggling with is fair balance of time/issues on my team. We'll experiment as we do. :)