Author Topic: Long Parsing scene - 50% ram and 50% processor usage  (Read 18599 times)

2019-08-09, 04:47:03

Jak

  • Active Users
  • **
  • Posts: 63
    • View Profile
Hello,

I am rendering a couple of animation sequences, the loading times for parsing scene is quite long, takes aproximately 2 - 3 minutes, with usage of half of my computers RAM, and oscilating between 50-60% of usage of processing power.

I read that parsing scenes takes time because of low RAM, however this is not the case here. Is there anything that I can do to remove parsing times? 2-3 minutes extra parsing times for each of the hundreds of frames is quite unfortunate.

EDIT: More information about my scene (as per information here: https://coronarenderer.freshdesk.com/support/solutions/articles/5000675854-system-is-running-low-on-ram-you-may-experience-excessive-parsing-times-and-or-crashes-you-will-pro)
[altough I know all of the points below are mainly to lower RAM usage, which is not necessary in my scene)

1) I have only 4 passes including Beauty pass; no atmospherics or AO pass - denoising 4 passes only.
2) No displacement
3) No super high resolution textures
4) Lots of instances and proxies
5) Resolution in HD only (not full HD)
6) Ram usage 50 percent, so I have enough RAM (47GB RAM usage, ~82GB RAM installed total)

Any info would be appreciated.
« Last Edit: 2019-08-09, 05:20:29 by Jak »

2019-08-09, 13:34:16
Reply #1

rowmanns

  • Corona Team
  • Active Users
  • ****
  • Posts: 1892
  • Corona for 3ds Max QA Team
    • View Profile
Hi,

Would you be able to send over a copy of your scene?

We will take a look to see if something can be optimised, either in your scene or in Corona.

Instructions on how to do this are in my signiture.

Thanks,

Rowan
Please read this before reporting bugs: How to report issues to us!
Send me your scene!

2019-08-09, 17:18:51
Reply #2

Jak

  • Active Users
  • **
  • Posts: 63
    • View Profile
Hi,

Would you be able to send over a copy of your scene?

We will take a look to see if something can be optimised, either in your scene or in Corona.

Instructions on how to do this are in my signiture.

Thanks,

Rowan


Well, this is still the same scene where I had problems saving any format with glare and bloom (posted problem in a thread earlier - doubled my RAM and I can save it now), so the file is still under NDA as it was before, which is why I am still unable to send the file for inspection.
Im curious if this has something to do with Corona scatter and proxies (as I am using Corona scatter and quite a lot of instances, plants, trees, cars etc).

I think I will power through this, but its a real shame, because I have one frame set to stop and save at the 10 minute mark, so every 500 frames I lose time for additional ~100/150 frames due to the precomputation ie parsing scene.

The scene is quite big might I add, I have managed to divide it into two chunks, however the precomputation is still there.

Once the precomputation is done, during the rendering the processor is at 100 percent usage (and at half RAM, same as precomputation), so there is no problem there. Its only during parsing scene Corona is not using full RAM (as per parsing scene instructions) or full processing power.





2019-08-14, 08:54:55
Reply #3

Jak

  • Active Users
  • **
  • Posts: 63
    • View Profile
Hi,

(...)

We will take a look to see if something can be optimised, either in your scene or in Corona.

(...)

Rowan


The problem is getting worse. During mid render of an animation sequence, the precoputation has increased from 2 minutes now to ~7 minutes. At this rate I am no way going to make it on schedule with rendering the animation. How is it possible that the precomputation increased? The camera has not moved that far, nor did I add any new objects/materials to the scene.

@Rowan, you mentioned scene optimisation. Could you specify what optimisations you have in mind so that I can try and fix this issue myself? What else is there apart from adding proxies in order to cut down on memory as far scene optimisation goes? I hid geometry that is not being rendered during the camera movement, but cannot hide the objects that are going to be in view eventually. Can you please either link me to an article or help file, or could you state what I can do in this thread?

Also regarding Corona optimisation, what did you have in mind? What settings could help to lower the precomputation, per you post in terms of Corona optimisation? Could you either post a link to such information here, or post some of this information yourself?

EDIT: I am using 3dsmax and RAM usage is still at ~50% during the precomputation scene.

« Last Edit: 2019-08-14, 09:11:37 by Jak »

2019-08-14, 11:25:38
Reply #4

Ink Visual

  • Active Users
  • **
  • Posts: 159
    • View Profile
Hi, I have similar problem with scene parsing rising over time as animation frames are being rendered. Last week it would grow from 3mins to almost 7mins during 48 hours of rendering.
The scene is indeed very heavy as it's a huge masterplan in mountainous environment but it was optimised in numerous ways already. Standard stuff, there are no big textures used, heavy polys are proxies, instances used wherever it was possible.
Last step that I've done yesterday that helped reducing parsing times was going through all the proxies once again, converting them back to mesh and running prooptimizer on them. On some sequences it reduced parsing from 3mins to 1.5mins, although the problem with parsing rising over time is still there.

2019-08-14, 11:30:26
Reply #5

Vlad_the_rant

  • Former Corona Team Member
  • Active Users
  • **
  • Posts: 107
  • Vladimir
    • View Profile
Corona proxies DO NOT save RAM! Their only purpose is speeding up viewport performance, not RAM usage reduction. Therefore lots of proxies are, at best, the same as no proxies at all and at worst detrimental in very large scenes (since they all need to be loaded into RAM before rendering starts). If they are stored on a slow drive or on a network path, loading them will take time. Can you try removing temporarily at least the largest proxies or half the proxies to see whether that speeds up the pre-render preparation? Thanks!

2019-08-18, 22:38:17
Reply #6

Jak

  • Active Users
  • **
  • Posts: 63
    • View Profile
Hi, I have similar problem with scene parsing rising over time as animation frames are being rendered. Last week it would grow from 3mins to almost 7mins during 48 hours of rendering.
The scene is indeed very heavy as it's a huge masterplan in mountainous environment but it was optimised in numerous ways already. Standard stuff, there are no big textures used, heavy polys are proxies, instances used wherever it was possible.
Last step that I've done yesterday that helped reducing parsing times was going through all the proxies once again, converting them back to mesh and running prooptimizer on them. On some sequences it reduced parsing from 3mins to 1.5mins, although the problem with parsing rising over time is still there.

Good to know its not only me that is having these issues.

Quote
Corona proxies DO NOT save RAM! Their only purpose is speeding up viewport performance, not RAM usage reduction. Therefore lots of proxies are, at best, the same as no proxies at all and at worst detrimental in very large scenes (since they all need to be loaded into RAM before rendering starts). If they are stored on a slow drive or on a network path, loading them will take time. Can you try removing temporarily at least the largest proxies or half the proxies to see whether that speeds up the pre-render preparation? Thanks!

Is blows my mind that for bigger scenes using corona proxies will actually make your scene less optimised and will take long to render... So it is better just to get a better graphics card set up and use as less or better yet no proxies at all. If its only for better viewport how does it differ from bounding box preview from within 3dsmax (apart from better overall visibility in terms of placement)? Will do some tests later for myself.

I am rendering another camera angle, started out with scene parsing 2 minutes, ater 3/4 of the animation done, the parsing increased to 5 minutes... Which is mind boggling, because its still the same aerial shot, the camera just rotated about 40 degrees, and no new geometry has been introduced to the scene, it is all there in that one shot from the beginning until the end. Its the same problem for all animation angles I render, the parsing time increases about 150% after half of the animation is done...




« Last Edit: 2019-08-18, 23:56:24 by Jak »

2019-08-18, 23:57:42
Reply #7

romullus

  • Global Moderator
  • Active Users
  • ****
  • Posts: 8779
  • Let's move this topic, shall we?
    • View Profile
    • My Models
Is blows my mind that for bigger scenes using corona proxies will actually make your scene less optimised and will take long to render... So it is better just to get a better graphics card set up and use as less or better yet no proxies at all. If its only for better viewport how does it differ from bounding box preview (apart from better overall visibility in terms of placement)? Will do some tests later for myself.

Proxies also makes max file much lighter, which really helps with saving times, especially autosave.
I'm not Corona Team member. Everything i say, is my personal opinion only.
My Models | My Videos | My Pictures

2019-08-19, 00:13:38
Reply #8

Jak

  • Active Users
  • **
  • Posts: 63
    • View Profile
Update: Set the scene to stop rendering after 5:30 minutes. It takes 14 minutes to complete one frame at this moment. So out of 14 minutes only 5:30 is actual render time. It got this bad after rendering 150 frames. Took about 7 or 8 minutes to render for the first 100 frames or so, including the 5:30 rendering time for one frame.

2019-08-19, 00:14:10
Reply #9

arqrenderz

  • Active Users
  • **
  • Posts: 990
  • https://www.behance.net/Arqrenderz1
    • View Profile
    • arqrenderz
Corona proxies DO NOT save RAM! Their only purpose is speeding up viewport performance, not RAM usage reduction. Therefore lots of proxies are, at best, the same as no proxies at all and at worst detrimental in very large scenes (since they all need to be loaded into RAM before rendering starts). If they are stored on a slow drive or on a network path, loading them will take time. Can you try removing temporarily at least the largest proxies or half the proxies to see whether that speeds up the pre-render preparation? Thanks!
Hi Papuca, we had such issue with a big scene, one thing that bothers us is the fact that Proxies take ram EVEN on a HIDDEN layer! WHY??? they are not being calculated for anything, so why they need to be loaded.... We were trying a script to change all the proxys geometry to a box but we ran in few bugs and had no more time to make it work, the script was working indeed, so if the corona team can do something about it, it would be really great for all.

2019-08-19, 00:17:10
Reply #10

Jak

  • Active Users
  • **
  • Posts: 63
    • View Profile
Is blows my mind that for bigger scenes using corona proxies will actually make your scene less optimised and will take long to render... So it is better just to get a better graphics card set up and use as less or better yet no proxies at all. If its only for better viewport how does it differ from bounding box preview (apart from better overall visibility in terms of placement)? Will do some tests later for myself.

Proxies also makes max file much lighter, which really helps with saving times, especially autosave.

I would take lower rendering times and higher autosave time without a doubt. I am working on SSD at this moment and I still am having long parsing times.

Edit:

I found other people have had similar problems:
https://forum.corona-renderer.com/index.php?topic=24802.15

However I have 82 GB of ram and still having problems...
« Last Edit: 2019-08-19, 01:03:10 by Jak »

2019-08-19, 01:24:31
Reply #11

Jak

  • Active Users
  • **
  • Posts: 63
    • View Profile
Update: Set the scene to stop rendering after 5:30 minutes. It takes 14 minutes to complete one frame at this moment. So out of 14 minutes only 5:30 is actual render time. It got this bad after rendering 150 frames. Took about 7 or 8 minutes to render for the first 100 frames or so, including the 5:30 rendering time for one frame.

Stopped rendering, restarted computer, loaded scene in and deleted all hidden layers
Quote
(...)one thing that bothers us is the fact that Proxies take ram EVEN on a HIDDEN layer! WHY???(...)
.
Started rendering animation, took 3 minutes to ready the frame (instead of ~9), and now its rendering... what the hell is going on...

Its as if Corona is somehow getting clogged up after a couple of hours of rendering, and its not my RAM either, because my ram usage is constanly at 50-60%. And its not 3dsmax either, as the person in the post I linked above is working under Cinema4D and still has these problems.
« Last Edit: 2019-08-19, 01:29:18 by Jak »

2019-08-19, 10:10:37
Reply #12

romullus

  • Global Moderator
  • Active Users
  • ****
  • Posts: 8779
  • Let's move this topic, shall we?
    • View Profile
    • My Models
Hi Papuca, we had such issue with a big scene, one thing that bothers us is the fact that Proxies take ram EVEN on a HIDDEN layer! WHY??? they are not being calculated for anything, so why they need to be loaded....

Do you have keep in memory enabled on your proxies? Try to disable it and see if that solves the issue.
I'm not Corona Team member. Everything i say, is my personal opinion only.
My Models | My Videos | My Pictures

2019-08-19, 16:17:25
Reply #13

Jak

  • Active Users
  • **
  • Posts: 63
    • View Profile
Another update:

Frame #1     : Total time to complete this frame ~7:30 minutes (5:30 actual render time, rest for parsing and preparing geometry)
Frame #88 : Total time to complete this frame ~10:37 minutes (5:30 actual render time, rest for parsing and preparing geometry)

Each frame is longer than the previous frame by seconds, and this piles up towards the later frames. And again, this does not make sense, as its the same geometry from an aerial view, only the camera is slighly rotated (yes slightly, as it is a slow shot).

Total estimated time was 26 hours for the completion of my animation. Now total estimated time is 32 hours (after 14 hours have passed). Still have 120 frames to render. Im really curious what the rendering time will be in another 80 frames.

How can anyone use Corona for production in animation at this point?

Also is there anyone from Corona support here?

@rowmanns, you mentioned scene optimisation, so I ask one more time. How can I optimise my scene to help solve this issue? If you have an idea of how to solve this can you please post info in this thread? Also, someone already had sent you a file with very similar problems (C4D file), did Corona team have any chance to look into the problem? Thanks.
« Last Edit: 2019-08-19, 16:30:15 by Jak »

2019-08-19, 17:29:49
Reply #14

maru

  • Corona Team
  • Active Users
  • ****
  • Posts: 12711
  • Marcin
    • View Profile
Papuca is from our support team, and Rowmanns from our QA teams. We are investigating this.
If you need immediate assistance, it is always best to use support@corona-renderer.com or https://coronarenderer.freshdesk.com/support/tickets/new
Marcin Miodek | chaos-corona.com
3D Support Team Lead - Corona | contact us