Chaos Corona Forum

Chaos Corona for 3ds Max => [Max] I need help! => Topic started by: Helldoor on 2021-03-03, 18:29:23

Title: Issue with a huge scene - strange behaviour with GI
Post by: Helldoor on 2021-03-03, 18:29:23
Hey there!

Here's the situation: I currently work on a very extensive exterior scene - a fisheye of a large building with a lively urban situation all around - a two-lane street with lots of cars, a lot of greenery (mostly FP) + kinda furnished shops on the first floor. Before adding the 3D people (around 200 pcs), the scene size was around 2,8GB. While rendering (in 4K) the scene would fill all of my 64GBs of RAM, but will still render out quite nice in around 40mins (at 4% noise limit).
After adding the 3D people, the scene grew up to 3,4GB and became a real nightmare: it needs around 15 mins to open and while doing so already fills out all of my RAM. As you can imagine, rendering it is impossible in this case.

What I did, trying to solve the issue:
. turned all the stuff in the shops into proxies
. turned all the 3d people into proxies

The result: the file size dropped back to 1,6GB. BUT all of the rest remains - the scene still needs around 15 mins to open +   it still fills out my 64GB RAM. What's more, when trying to make a small interactive render, the scene takes forever and then renders overexposed. When I switch off most of the content layers (IR is still running) and turn them back on one-by-one, the lighting/GI becomes normal again.

Corona Scene Check shows no issues. What the Corona Error Message Window shows I've attached.

So, my questions: is it the unsufficient RAM, causing all the trouble? Can I do something about it? For the final render I'll use a farm service, no question, but for some WIP stills I'd really be happy to be able to use my own workstation.

Thanks in advance!

SPECS: 3DS Max 2021 + Corona 6, Hotfix 1 (both licensed)
 CPU: AMD Ryzen Threadripper 2990WX 32-Core
 GPU: NVIDIA QUADRO P4000
 RAM: 64 GB

Title: Re: Issue with a huge scene - strange behaviour with GI
Post by: Helldoor on 2021-03-03, 18:54:13
Here is what initially happens, when the Start IR button is pressed - clear overexposure (see attachment).
When I make most of the layers invisible for a moment and turn them back on the GI suddenly becomes normal again and the scene looks as it should (unfortunatelly cannot post it, because of NDA).

Interesting enough, this 'overexposure' also happened with the Conserve Memory option enabled and my RAM not being entirely used out for the IR (around 44GB were in use)...
Title: Re: Issue with a huge scene - strange behaviour with GI
Post by: romullus on 2021-03-03, 20:10:19
Did you try to delete all the people models and see if the scene would render normally again? Also try to clean the scene from animation layers, track notes and other garbage that may have been accumulated in it. See if that helps.
Title: Re: Issue with a huge scene - strange behaviour with GI
Post by: Helldoor on 2021-03-03, 20:19:56
Thanks a lot! Actually, the people are posed and not animated ones, but I'll take a look. Which script do you recommend for that? Or the SiniSoftware stuff?
Title: Re: Issue with a huge scene - strange behaviour with GI
Post by: romullus on 2021-03-03, 20:27:47
I think any cleaner script will do. Personally i like Prune script, but you can use whatever you have, or will find. SiniSoftware tools should be good.

Edit: animation layers often has little to do with animated models, 3ds Max can add them on its own for whatever reasons.
Title: Re: Issue with a huge scene - strange behaviour with GI
Post by: Helldoor on 2021-03-03, 20:56:27
Thank you again!
I installed Forensic and will upload the results after running a scan, but before deleting anything. It will be awesome, if you could take a quick look then!
Title: Re: Issue with a huge scene - strange behaviour with GI
Post by: Helldoor on 2021-03-03, 21:07:59
Attacht's the result of the scan.
Title: Re: Issue with a huge scene - strange behaviour with GI
Post by: romullus on 2021-03-03, 21:30:26
There is large amount of animation layers in your scene, which you could try to delete, but be aware that this operation may break up animation in your scene, so make sure you have a backup before doing this. Other than that i don't see other major issues.
Title: Re: Issue with a huge scene - strange behaviour with GI
Post by: Helldoor on 2021-03-03, 22:10:41
Once again - thank you very much!

That's all very weird. I erased the animations and the animation layers. Scene size dropped down to 528Mb!!!
Opening it stll takes approx. 10 minutes and fills 65GB (?) out of my 64GB RAM. Once the scene's open, the RAM load drops to around 75% / 50GB. Interactive render still delivers the overexposed result first. After switching the layer off and back on again, it looks fine. The issue - if I started a final render, I won't have the control to switch the layers on and off, so it ends up renered as overexposed. Damn.
Title: Re: Issue with a huge scene - strange behaviour with GI
Post by: romullus on 2021-03-03, 23:16:30
Sorry, i have no clue what might cause the overexposure. That sounds pretty odd. I think you should open a support ticket and ask the team to inspect your scene. That probably would be the fastest way to solve the issue.

https://coronarenderer.freshdesk.com/support/tickets/new
Title: Re: Issue with a huge scene - strange behaviour with GI
Post by: Helldoor on 2021-03-03, 23:24:37
Thank you, will probably do.

After some further investigation, I was able to find out, that if I set the multiplier of the "Rest"-position in the Interactive Lightmix to 0.0 (instead of the default 1.0), the overexposure is gone. So, the overexposure doesn't originate from my GI (HDRI), but from something, that is among the "Rest" lights in the Interactive Lightmix. The problem is, that if I try to rebuild the Lightmix, I'll need at least 80GB RAM for the Instanced Lights option. For the others around 400GB...

Now I'm trying to copy to/paste from all the geometry in a new file, so that I can maybe get rid of some hidden light issue. Trial and error, I guess.
 
Title: Re: Issue with a huge scene - strange behaviour with GI
Post by: wilbertvandenbroek on 2021-03-04, 15:00:38
I was working on quite a heave scene myself lately and from what I learned is that texture size makes the biggest difference in ram usage. Really go for the minimal texture resolution you need. I saved a lot of ram at render time without any loss in quality. To often we go and build scenes without the optimization in mind cause corona will most likely render it all :-)
Title: Re: Issue with a huge scene - strange behaviour with GI
Post by: Helldoor on 2021-03-05, 10:51:28
I generally tend to use textures in 3K resolution. Usually this gives nice results, without the scene getting too heavy. Sure, using 1K textures would be easier on the RAM, but for closeups afterwards I'll have to rework all the textures... @wilbertvandenbroek what resolution do you go for?

As for my general issue: after figuring out, that the overexposure could be switched off, when giving the "Rest (unassaigned)" lights in the LightMix a value of 0,0, I know this must be some light. And given how strong it was (see screenshot above) I figured it must be a sun. I had a sun for a daylight study in the early phases of this scene, so I inspected furtherly and saw, that there was a second sun, that I had actually turned off. I guess that was too much for LightMix to handle, therefore it bugged out. I deleted all the suns and now the overexposure is gone.
The file still fills lot of the RAM, but I was able to render it in 4K/4% noise limit in less than an hour. Given how extensive the scene, that's pretty great.

Thanks again for the help!

Title: Re: Issue with a huge scene - strange behaviour with GI
Post by: maru on 2021-03-05, 17:47:22
Glad to hear that the overexposure issue is solved! I was actually betting on the NVIDIA denoiser messing something up with exposure. :)

Just a note: even if RAM usage is not 100% in the Task Manager (50-75%), it can still cause some issues: https://forum.corona-renderer.com/index.php?topic=17892.0
When there is not enough RAM, all kinds of weird things may start happening (but overexposure would be really surprising).

Also, since you mentioned that you are rendering in high resolution and using LightMix, maybe you can lower the number of your LightMix and other render elements? This should help reduce RAM usage, especially if you are using denoising at the same time.

Title: Re: Issue with a huge scene - strange behaviour with GI
Post by: Helldoor on 2021-03-07, 21:13:27
Thanks, maru - very helpful input!
Title: Re: Issue with a huge scene - strange behaviour with GI
Post by: wilbertvandenbroek on 2021-03-08, 10:26:52
I generally tend to use textures in 3K resolution. Usually this gives nice results, without the scene getting too heavy. Sure, using 1K textures would be easier on the RAM, but for closeups afterwards I'll have to rework all the textures... @wilbertvandenbroek what resolution do you go for?

Everything that is far away will get 1k resolution, close to the camera max 4k. And really look at small stuff, like some screw or bolt or whatever doesn't need 1k but much smaller. Also I try to reduce the image size to a minimum by compressing to the max, Tinypng or just jpg compression. As long as it doesn't make a difference.
Title: Re: Issue with a huge scene - strange behaviour with GI
Post by: romullus on 2021-03-08, 11:07:59
Also I try to reduce the image size to a minimum by compressing to the max, Tinypng or just jpg compression. As long as it doesn't make a difference.

This won't help. Corona stores uncompressed textures in RAM. It doesn't matter if you'll save your texture on disk as 100 KB PNG, or 50MB BMP file, they will take equal amount of RAM in rendering if everything else is the same.

Here's what will save the RAM in terms of textures - lowering resolution, making sure that your textures has optimal bit depth, getting rid of excess channels.
Title: Re: Issue with a huge scene - strange behaviour with GI
Post by: wilbertvandenbroek on 2021-03-08, 12:51:55
I didn't know that. How does that work? When I work with lossy file formats like jpg there is no uncompressed version right?
Title: Re: Issue with a huge scene - strange behaviour with GI
Post by: romullus on 2021-03-08, 14:38:35
No matter in what format an image is saved on disk (compressed, or not, lossless, or lossy), Corona always allocates the same amount of bits per pixel times bit depth times number of channels. The math is easy - 2K texture will need 4 times less RAM than 4K, greyscale image will take 3 times less RAM than RGB and 4 times less than RGBA, 8 bit texture will occupy half amount of RAM compared to 16 bit. Format is only important if you want to save disk space.
Title: Re: Issue with a huge scene - strange behaviour with GI
Post by: wilbertvandenbroek on 2021-03-09, 12:39:03
Ok, another thing learned today. Thanks for clearing that up.