Chaos Corona Forum

Chaos Corona for 3ds Max => [Max] I need help! => Topic started by: Bormax on 2017-01-26, 12:59:15

Title: How to avoid long parsing time?
Post by: Bormax on 2017-01-26, 12:59:15
Hi

Mostly working with exterior scenes I have situations when files with about the same amount of objects, tree models, cars, etc., have quite different parsing time. And this difference not about few percent, for example one scene has Scene parsing time 9 sec, another one has 1min 20sec of parsing time. As I said they are about the same - trees, grass, cars... Was thinking, that problem is some object in scene, but excluding groups of objects from rendering (trees, then grass, then cars and so on) I made parsing time go down proportionally in both scenes, so I didn't find any particular object/s which can affect to parsing time so much.

So I'd like to know:
What the Scene parsing process is?
What determines the time for parsing?
How to avoid long parsing time?
And would be interesting to hear your ideas what can be wrong with the scene described above? You can find attached screenshot with this scene, parsing time here 1min 10sec - 1min 30sec

Thanks in advance!
Title: Re: How to avoid long parsing time?
Post by: maru on 2017-01-26, 15:17:18
Are you using some 3rd party plugins in this scene? Can you think of any differences between the scenes where parsing is fast (e.g. few seconds) and slow (e.g. one minute)?

I am also concerned about the "0" stats in your VFB... looks like something is wrong here.
Title: Re: How to avoid long parsing time?
Post by: Bormax on 2017-01-26, 20:09:36
In both scenes Forest pack used for grass and background trees scattering. As I said excluding those object from rendering reduce parsing time proportionally, it means if grass is not renderable, parsing time about 20% less in both scenes, the same with trees. Buildings in both cases are adopted archicad models, parsing time for rendering only them takes in both cases 1 second.
Experimenting today with slow scene I've got this statistic for rendering parts of the scene separately:
Building with forest - parsing time 2 sec
Building with trees on foreground and bushes - 4 sec
Building with cars - 2 sec
Building with grass - 11 sec
And - SURPRISE!! - after these tests I rendered again everything and got only 20 sec of parsing time. Saved this file with another name, opened old file again, and got 1min 50sec  when I rendered 1st time (as before when it was 1st render after opening), and next renders take now 20 sec for scene parsing... Do not have any idea what can be the reason for this. Last time I did tests 22.01, when I asked about CPU usage in daily section, then I got stable 1min 10sec of parsing time for whole scene (not 1st rendering), the same was before, when I was working with this scene - that was the reason to ask about it.
Title: Re: How to avoid long parsing time?
Post by: Ondra on 2017-01-27, 10:58:52
this sounds like some cached precomputation on the side of forest pack - probably nothing we can do about it. You could send us the scene so we could confirm it
Title: Re: How to avoid long parsing time?
Post by: Bormax on 2017-02-09, 07:37:20
Hi

It looks like I found the problem point. Quite often I have simple animation for the sun for different cameras, and I've noticed that if I don't change frame number I get few second of parsing, if I change frame, parsing time grows. Finally I found that the problem is in proxies, actually in Enable animation of them. Even if the proxy was created as not animated, this checkbox has influence on parsing time.

Example file uploaded with private uploader 1486622037_box-proxy.zip

Simple scene with only one proxy - box, subdivided to rise polycount. Open it and start render - parsing time is 0, change frame - parsing time 2 sec. Check Enable animation on, don't change frame - parsing time 0 sec, change frame - parsing time 2 sec again.
In big scenes it's also dramatically slows down feedback time of IR

I made the same tests with proxies of trees, there I've got difference like 2 sec - 21 sec (worst result). Probably more proxies in the scene - bigger difference of parsing time.
Is it a bug, or some problems on my side?
Title: Re: How to avoid long parsing time?
Post by: romullus on 2017-02-09, 14:49:52
Sounds like a bug to me. Could you make new report in the bug reporting board or Mantis?
Title: Re: How to avoid long parsing time?
Post by: Bormax on 2017-02-09, 16:28:52
Done
Title: Re: How to avoid long parsing time?
Post by: iancamarillo on 2017-11-28, 20:54:55
was this fixed in 1.7 hotfix1?
Title: Re: How to avoid long parsing time?
Post by: iancamarillo on 2017-11-28, 23:26:37
I haven't updated the machines to 1.7 hotfix1 but after I disabled animation on thousands of proxies, my parsing time for each frame dropped from 20min to 3min. Probably saving me days to complete. I'm still testing to confirm this though. I attached a script I made to disable animation from all selected proxies.
Title: Re: How to avoid long parsing time?
Post by: 3dboomerang on 2019-03-26, 12:44:13
guys is this comfirmed or not, i'm using animated proxies and i'm struggling with annoying parsing times. I'm also using forest pack

A follow up would be welcome?...
Title: Re: How to avoid long parsing time?
Post by: iancamarillo on 2019-08-12, 19:15:07
this sounds like some cached precomputation on the side of forest pack - probably nothing we can do about it. You could send us the scene so we could confirm it

Hi Ondra, I've uploaded an archive to your dropbox. When I send you a message it doesn't show in my sent folder. Can you let me know if you've received the files and message? Thanks
Title: Re: How to avoid long parsing time?
Post by: TomG on 2019-08-12, 19:23:04
Rather than message any individual, it's best to message support. The main thing about messaging an individual is that they may not be there when you message them, but there's always someone manning support. PS - Ondra isn't around just now :)
Title: Re: How to avoid long parsing time?
Post by: iancamarillo on 2019-08-12, 19:36:05
Got it. I'll resend the message to support now. Thanks
Title: Re: How to avoid long parsing time?
Post by: iancamarillo on 2019-08-12, 20:00:40
Hi Tom,
Is there a dropbox for files over 2.5gb that support has access to other than ondra? Also, what is supports PM name? Thanks
Title: Re: How to avoid long parsing time?
Post by: TomG on 2019-08-12, 20:17:48
Yes, info is at the bottom of https://corona-renderer.com/upload

There's no generic forum account for support, you should email them at support@corona-renderer.com (as then things are in the tracking system and won't get lost - can't track forum PMs :) )
Title: Re: How to avoid long parsing time?
Post by: 3dboomerang on 2019-09-17, 12:53:39
does anyone have a script to disable animation in all proxies immediately? I have 100+ proxies in this scene, I can't even begin to disable them all by hand? I'm running on 7,5 minutes of parsing time with animation enabled.

Using Forest Pack Pro, animated sun (target)
Title: Re: How to avoid long parsing time?
Post by: Frood on 2019-09-17, 13:20:52
Try to paste this in the maxscript listener + Return:

Code: [Select]
for o in objects where classof(o)==CProxy do o.animationEnabled=false

Good Luck



Title: Re: How to avoid long parsing time?
Post by: Ondra on 2019-09-17, 16:13:33
Current dailies have significant parsing speedups, have you tried them?
Title: Re: How to avoid long parsing time?
Post by: 3dboomerang on 2019-09-17, 16:16:55
no, to be honest, I usualy just download when corona releases a new version, is there a link?

grts
Title: Re: How to avoid long parsing time?
Post by: TomG on 2019-09-17, 16:49:47
Best link is this one (as the actual dropbox changes now and then, but this page gets updated to point to the current location - plus is has instructions!): https://coronarenderer.freshdesk.com/support/solutions/articles/5000570015-corona-renderer-for-3ds-max-daily-builds
Title: Re: How to avoid long parsing time?
Post by: Bormax on 2019-09-18, 11:17:53
does anyone have a script to disable animation in all proxies immediately? I have 100+ proxies in this scene, I can't even begin to disable them all by hand? I'm running on 7,5 minutes of parsing time with animation enabled.

Using Forest Pack Pro, animated sun (target)

Did you try script published by iancamarillo in this topic on first page?

I haven't updated the machines to 1.7 hotfix1 but after I disabled animation on thousands of proxies, my parsing time for each frame dropped from 20min to 3min. Probably saving me days to complete. I'm still testing to confirm this though. I attached a script I made to disable animation from all selected proxies.
Title: Re: How to avoid long parsing time?
Post by: 3dboomerang on 2019-09-18, 11:35:46
yes, it worked, all animations were disabled, as soon as i'm sending another render to the BB i'm going to compare. I saw yesterday it already made a difference but i'm still having 7m+ parsing times, I think I'm going to have to send all scene layers one by one and see which is taking so damn long to parse
Title: Re: How to avoid long parsing time?
Post by: 3dboomerang on 2019-09-22, 12:16:35
Attached you can find a binder of each layer rendered with the parsing time in the corner below, all layers combined give me about 64Gb of usage, with "conserve memory" enabled.

Parsing time atm for rendering a camera has an average of 10.7452667 minutes

I'd be happy to get some suggestions to lower this if possible. The scene is huge, so I'm not sure if there are any options left for me to explore and reduce the parsing time.

I already did the following:
All textures, except the HDRI, are reduced to jpg in 4K res
All vegitation used to spread with hand, or used in Forest Pack Pro, has been done with proxies at the base.
All proxies are set to point cloud
All proxies are set to "keep in memory"
All proxies have animation mode disabled
Some Forest Pack Objects still use "edge-mode" - I will remove this in a last resort to lower my parsing times, it has 5,5 minutes of parsing time, which is half of everything.

Scene:
The scene uses 1HDRI and 1Corona Sun to light + Artificial Lighting (see layer lighting)
There is a lightmix to generate day and night (in 1-HDRI, believe it or not) with in total 9-light-select groups


All buildings are proxies. The only thing not proxy is the terrain for distribution.

Max file is about 100Mb large with 5Gb+ in proxy material (to keep the file running and viewport friendly)

All files are on a server -

Grts

Title: Re: How to avoid long parsing time?
Post by: 3dboomerang on 2019-09-23, 03:48:53
I installed the daily build on my main workstation, a threadripper 1950x, and its parsing times are ~3minutes, compared to my node AMD WX2990 which has 9 minutes parsing times, on the last official corona version.

Whatever you put in there, it's magic.
Title: Re: How to avoid long parsing time?
Post by: arqrenderz on 2019-09-23, 14:54:55
Dont want to crash the party here, but is that some of the vray code  for the parsing speedup? the office use corona and cray, and i have seen very fast scene starts on vray side..
Anyway ondra, will you guys show more of the partership in the near future? like how are you collaborating with each other?
Title: Re: How to avoid long parsing time?
Post by: maru on 2019-09-23, 17:03:06
I installed the daily build on my main workstation, a threadripper 1950x, and its parsing times are ~3minutes, compared to my node AMD WX2990 which has 9 minutes parsing times, on the last official corona version.
Whatever you put in there, it's magic.
Hi, did you start a support ticket in the end? If so, could you share your ticket number here?

Title: Re: How to avoid long parsing time?
Post by: 3dboomerang on 2019-09-24, 08:46:27
No I didn't make a support ticket, do u see the need?
Title: Re: How to avoid long parsing time?
Post by: maru on 2019-09-24, 11:50:29
No I didn't make a support ticket, do u see the need?
Generally if you are having any issues with Corona, it is best to start a support ticket. This is the best way for the team to track the report, update it timely, and offer solutions.
Title: Re: How to avoid long parsing time?
Post by: 3dboomerang on 2019-09-26, 09:54:21
Ok I didn't know, I thought it was common to first see in the forums if there are solutions with your problem. Generally I found them?

Is it possible the daily doesn't work on a node-computer to render? The parsing times on my 2990WX is still longer than on my 1950x, both have daily builds installed.


1950 hits about 4 minutes of parsing time, 2990WX is ~ double that
Title: Re: How to avoid long parsing time?
Post by: arqrenderz on 2019-09-26, 16:53:26
The enable animation on the proxys should be lokked at, it really kills  like 40% of the parsing time on BIG scenes with lots of proxys.
Title: Re: How to avoid long parsing time?
Post by: GeorgeK on 2021-11-10, 08:10:45
Hi everyone, as of the latest public daily of 2021-11-05, this issue with parsing times is addressed and should no longer be reproducible.
Please feel free to test it out and share your feedback with us, the latest DB can be found here: https://forum.corona-renderer.com/index.php?topic=33839.msg191656#msg191656