Issue with a huge scene - strange behaviour with GI

[Helldoor]

Hey there!

Here's the situation: I currently work on a very extensive exterior scene - a fisheye of a large building with a lively urban situation all around - a two-lane street with lots of cars, a lot of greenery (mostly FP) + kinda furnished shops on the first floor. Before adding the 3D people (around 200 pcs), the scene size was around 2,8GB. While rendering (in 4K) the scene would fill all of my 64GBs of RAM, but will still render out quite nice in around 40mins (at 4% noise limit).
After adding the 3D people, the scene grew up to 3,4GB and became a real nightmare: it needs around 15 mins to open and while doing so already fills out all of my RAM. As you can imagine, rendering it is impossible in this case.

What I did, trying to solve the issue:
. turned all the stuff in the shops into proxies
. turned all the 3d people into proxies

The result: the file size dropped back to 1,6GB. BUT all of the rest remains - the scene still needs around 15 mins to open +   it still fills out my 64GB RAM. What's more, when trying to make a small interactive render, the scene takes forever and then renders overexposed. When I switch off most of the content layers (IR is still running) and turn them back on one-by-one, the lighting/GI becomes normal again.

Corona Scene Check shows no issues. What the Corona Error Message Window shows I've attached.

So, my questions: is it the unsufficient RAM, causing all the trouble? Can I do something about it? For the final render I'll use a farm service, no question, but for some WIP stills I'd really be happy to be able to use my own workstation.

Thanks in advance!

SPECS: 3DS Max 2021 + Corona 6, Hotfix 1 (both licensed)
 CPU: AMD Ryzen Threadripper 2990WX 32-Core
 GPU: NVIDIA QUADRO P4000
 RAM: 64 GB

[Helldoor]

Here is what initially happens, when the Start IR button is pressed - clear overexposure (see attachment).
When I make most of the layers invisible for a moment and turn them back on the GI suddenly becomes normal again and the scene looks as it should (unfortunatelly cannot post it, because of NDA).

Interesting enough, this 'overexposure' also happened with the Conserve Memory option enabled and my RAM not being entirely used out for the IR (around 44GB were in use)...

[romullus]

Did you try to delete all the people models and see if the scene would render normally again? Also try to clean the scene from animation layers, track notes and other garbage that may have been accumulated in it. See if that helps.

[Helldoor]

Thanks a lot! Actually, the people are posed and not animated ones, but I'll take a look. Which script do you recommend for that? Or the SiniSoftware stuff?

[romullus]

I think any cleaner script will do. Personally i like Prune script, but you can use whatever you have, or will find. SiniSoftware tools should be good.

Edit: animation layers often has little to do with animated models, 3ds Max can add them on its own for whatever reasons.

[Helldoor]

Thank you again!
I installed Forensic and will upload the results after running a scan, but before deleting anything. It will be awesome, if you could take a quick look then!

[Helldoor]

Attacht's the result of the scan.

[romullus]

There is large amount of animation layers in your scene, which you could try to delete, but be aware that this operation may break up animation in your scene, so make sure you have a backup before doing this. Other than that i don't see other major issues.

[Helldoor]

Once again - thank you very much!

That's all very weird. I erased the animations and the animation layers. Scene size dropped down to 528Mb!!!
Opening it stll takes approx. 10 minutes and fills 65GB (?) out of my 64GB RAM. Once the scene's open, the RAM load drops to around 75% / 50GB. Interactive render still delivers the overexposed result first. After switching the layer off and back on again, it looks fine. The issue - if I started a final render, I won't have the control to switch the layers on and off, so it ends up renered as overexposed. Damn.

[romullus]

Sorry, i have no clue what might cause the overexposure. That sounds pretty odd. I think you should open a support ticket and ask the team to inspect your scene. That probably would be the fastest way to solve the issue.

https://coronarenderer.freshdesk.com/support/tickets/new

[Helldoor]

Thank you, will probably do.

After some further investigation, I was able to find out, that if I set the multiplier of the "Rest"-position in the Interactive Lightmix to 0.0 (instead of the default 1.0), the overexposure is gone. So, the overexposure doesn't originate from my GI (HDRI), but from something, that is among the "Rest" lights in the Interactive Lightmix. The problem is, that if I try to rebuild the Lightmix, I'll need at least 80GB RAM for the Instanced Lights option. For the others around 400GB...

Now I'm trying to copy to/paste from all the geometry in a new file, so that I can maybe get rid of some hidden light issue. Trial and error, I guess.
 

[wilbertvandenbroek]

I was working on quite a heave scene myself lately and from what I learned is that texture size makes the biggest difference in ram usage. Really go for the minimal texture resolution you need. I saved a lot of ram at render time without any loss in quality. To often we go and build scenes without the optimization in mind cause corona will most likely render it all :-)

[Helldoor]

I generally tend to use textures in 3K resolution. Usually this gives nice results, without the scene getting too heavy. Sure, using 1K textures would be easier on the RAM, but for closeups afterwards I'll have to rework all the textures... @wilbertvandenbroek what resolution do you go for?

As for my general issue: after figuring out, that the overexposure could be switched off, when giving the "Rest (unassaigned)" lights in the LightMix a value of 0,0, I know this must be some light. And given how strong it was (see screenshot above) I figured it must be a sun. I had a sun for a daylight study in the early phases of this scene, so I inspected furtherly and saw, that there was a second sun, that I had actually turned off. I guess that was too much for LightMix to handle, therefore it bugged out. I deleted all the suns and now the overexposure is gone.
The file still fills lot of the RAM, but I was able to render it in 4K/4% noise limit in less than an hour. Given how extensive the scene, that's pretty great.

Thanks again for the help!

[maru]

Glad to hear that the overexposure issue is solved! I was actually betting on the NVIDIA denoiser messing something up with exposure. :)

Just a note: even if RAM usage is not 100% in the Task Manager (50-75%), it can still cause some issues: https://forum.corona-renderer.com/index.php?topic=17892.0
When there is not enough RAM, all kinds of weird things may start happening (but overexposure would be really surprising).

Also, since you mentioned that you are rendering in high resolution and using LightMix, maybe you can lower the number of your LightMix and other render elements? This should help reduce RAM usage, especially if you are using denoising at the same time.

[Helldoor]

Thanks, maru - very helpful input!