Chaos Corona Forum

Chaos Corona for 3ds Max => [Max] I need help! => Topic started by: flood on 2018-10-09, 15:44:03

Title: shadowcatcher and reflections problem
Post by: flood on 2018-10-09, 15:44:03
I want to use shadowcatcher to create a transparent background but want to keep the shadows so that I can composite it onto a background in photoshop.

When I render the scene everything look OK. But when I save the image as a PNG (with alpha) and open it in photoshop, the reflections on the ground are gone. It just doesn't look right.

When Saving the same image as a JPG, everything look OK but I have no transparency of course.

Top image is how it is rendered and saved as JPG
Bottom image is the same image but saved as PNG and then added a white background layer in photoshop. All reflections on the floor are gone.
Title: Re: shadowcatcher and reflections problem
Post by: pokoy on 2018-10-09, 16:20:22
First thing is, of course, how can reflections appear if your background is white? They would have to be brighter than RGB 255, which is not possible in 8 or 16 bits.
Second thing is, shadows are multiplied while reflections are additive. You simply can't have both without splitting the render output, that's what render elements are for.
Title: Re: shadowcatcher and reflections problem
Post by: sprayer on 2018-10-09, 16:21:14
Try to save EXR with layers or tga alpha and compose it in photoshop.
Title: Re: shadowcatcher and reflections problem
Post by: flood on 2018-10-09, 16:44:57
First thing is, of course, how can reflections appear if your background is white? They would have to be brighter than RGB 255, which is not possible in 8 or 16 bits.
Second thing is, shadows are multiplied while reflections are additive. You simply can't have both without splitting the render output, that's what render elements are for.

Thanks for your reply, I still don't seem to understand this.

First : It doesn't matter what color I use as a background, the reflections aren't there.

Second : If corona is able to output the reflections on the screen and save it as a JPG, why wouldn't that pixel information be available in PNG?
Title: Re: shadowcatcher and reflections problem
Post by: flood on 2018-10-09, 16:45:59
Try to save EXR with layers or tga alpha and compose it in photoshop.

I tried both, the problems seems to be in the alpha, parts where the reflections are, are being cut off by the alpha channel
Title: Re: shadowcatcher and reflections problem
Post by: sprayer on 2018-10-09, 17:31:27
please show you alpha too
Title: Re: shadowcatcher and reflections problem
Post by: pokoy on 2018-10-09, 17:32:43
Alpha probably considers only objects and shadows, which, if you think about it, isn't wrong at all.

Still, for what you want, you'd probably have to render all required elements and compose, as suggested above. There's simply no way to get reflections to appear correctly without using additive mode, which requires reflections to be separate.
Title: Re: shadowcatcher and reflections problem
Post by: flood on 2018-10-09, 17:40:52
Alpha probably considers only objects and shadows, which, if you think about it, isn't wrong at all.

Still, for what you want, you'd probably have to render all required elements and compose, as suggested above. There's simply no way to get reflections to appear correctly without using additive mode, which requires reflections to be separate.

But...but...but... I have seen it being done in the Corona shadowcaster tutorial (part 4). They have no problems with compositing with just rendering with shadowcaster only.

(fastforward to 2:03). I want/need just exactly the same effect.
Title: Re: shadowcatcher and reflections problem
Post by: flood on 2018-10-09, 17:48:19
please show you alpha too

This is the alpha channel. You can see clearly that the reflection in front of the box is just not there.
Title: Re: shadowcatcher and reflections problem
Post by: sprayer on 2018-10-09, 17:56:18
It's not  a reflection but light bounce. Beauty pass shows you all elements but alpha channel is different and you can a dd more shadows in material. In photoshop you can open your alpha in 32bit(what is way more than PNG alpha) and adjust levels to see more or less shadows
Title: Re: shadowcatcher and reflections problem
Post by: flood on 2018-10-09, 18:44:23

The "reflection" is indeed indirect lighting. So I rendered that out as a seperate element.
But unfortunatlyn when loading it in Photoshop, i get the same result.

In de rendering tab it looks good. But when saved to exr or png the indirect lighting on the floor is just gone.
Title: Re: shadowcatcher and reflections problem
Post by: sprayer on 2018-10-09, 19:55:18
Ok i think i am understand what you want, when you rendering in compositing mode and see in beauty pass this is not  real what is looks like,  turn off composing mode and you will  see real image, what you saw it's some kind indirect pass what will compose with alpha. Then you render without compose mode you will notice what you do not have light bounce, to add it setup more lights or some hdri environment then you will start to see bounce and then you  can change to compositing mode, now alpha will have this indirect bounce
Title: Re: shadowcatcher and reflections problem
Post by: PROH on 2018-10-10, 00:39:04
Just made a quick test with a teapot on a plane with shadow catcher material set to full reflection in compositing mode. Both the shadows and the reflections shows up in the alpha and is tranfered as expected when saved to tif-format with alpha activated.

Color-bleeding isn't transferred.
Title: Re: shadowcatcher and reflections problem
Post by: flood on 2018-10-10, 07:40:56
Ok i think i am understand what you want, when you rendering in compositing mode and see in beauty pass this is not  real what is looks like,  turn off composing mode and you will  see real image, what you saw it's some kind indirect pass what will compose with alpha. Then you render without compose mode you will notice what you do not have light bounce, to add it setup more lights or some hdri environment then you will start to see bounce and then you  can change to compositing mode, now alpha will have this indirect bounce

I tried rendering without compositing mode but the light bounce is still there.

EDIT : I set the backplate to black and now the light bounce is gone. It was white before.

Title: Re: shadowcatcher and reflections problem
Post by: sprayer on 2018-10-10, 08:54:48
That's strange. here how it works for me.

just white background in shadowcatcher and  direct visibility  with hdri in environment for light
http://imagizer.imageshack.com/img924/309/Fn5Xw2.jpg

this is how compositing looks, i've change direct visibility to black  for proper rendering alpha channel, it need in some case.
http://imagizer.imageshack.com/img922/7787/lNftla.jpg

Open exr in photoshop and apply alpha channel to mask layer
http://imagizer.imageshack.com/img921/4623/e5tzy4.jpg

with selected mask at layer cntl+l to adjust levels for mask to show less shadows
http://imagizer.imageshack.com/img922/4787/pKooIS.jpg

for  removing this bounce you need to use rayswitch material for shadowcatcher
Title: Re: shadowcatcher and reflections problem
Post by: flood on 2018-10-10, 09:43:32
That's strange. here how it works for me.

just white background in shadowcatcher and  direct visibility  with hdri in environment for light
http://imagizer.imageshack.com/img924/309/Fn5Xw2.jpg

this is how compositing looks, i've change direct visibility to black  for proper rendering alpha channel, it need in some case.
http://imagizer.imageshack.com/img922/7787/lNftla.jpg

Open exr in photoshop and apply alpha channel to mask layer
http://imagizer.imageshack.com/img921/4623/e5tzy4.jpg

with selected mask at layer cntl+l to adjust levels for mask to show less shadows
http://imagizer.imageshack.com/img922/4787/pKooIS.jpg

for  removing this bounce you need to use rayswitch material for shadowcatcher

Could you post your scene so that I can take a look at the settings? I probably am doing something wrong.

Title: Re: shadowcatcher and reflections problem
Post by: sprayer on 2018-10-10, 10:15:56
Here scene.
it's corona 2.0
Title: Re: shadowcatcher and reflections problem
Post by: flood on 2018-10-10, 11:29:22
Here scene.
it's corona 2.0

Thank you very much. This is helping me a lot. I have never used hdri mappings before. So i'm trying to replicate your scene without shadowcatcher and then build up from there...
I made the same scene from scratch and added your hdri mapping to the scene. I'm getting faceting results on the teapot and I don't know why.

I have included my scene in this post as attachment.
Title: Re: shadowcatcher and reflections problem
Post by: romullus on 2018-10-10, 11:40:14
Are you using dome mapping in CoronaBitmap for your HDRI?
Title: Re: shadowcatcher and reflections problem
Post by: sprayer on 2018-10-10, 11:58:25
The same for me. I have manage thats because of standard bitmap loader, if you load hdri via coronabitmap it will be fine as romullus mention,  so it's another bug i think.
Title: Re: shadowcatcher and reflections problem
Post by: romullus on 2018-10-10, 12:33:11
Yes, that's a known bug. You can "fix" it by changing dome mode to spherical, or you can retain dome and instead go to dev/debug settings and change enviro sampler from default to some other (fast or precise, seems to work fine).
Title: Re: shadowcatcher and reflections problem
Post by: flood on 2018-10-10, 13:05:06
Thank you all for helping me out. Faceting is gone now.
Title: Re: shadowcatcher and reflections problem
Post by: maru on 2018-10-15, 13:47:12
@flood - I am referring to the 1st post and the original issue.
Do you have some reflectivity enabled for the shadowcatcher material? If no, then those "reflections" are in fact GI.
GI should be picked up by the shadowcatcher anyway, and should be visible in the alpha channel.
Basically, the image should look identical if you render the shadowcatcher in normal mode (e.g. on a blue background) and if you enable the "for compositing" mode, and then composite the image in Photoshop (e.g. with a blue background).

I am not exactly sure what is going on here, and the thread has become a bit chaotic. If you would like us to take a closer look at this, please contact us using this form https://coronarenderer.freshdesk.com/support/tickets/new
Title: Re: shadowcatcher and reflections problem
Post by: flood on 2018-10-16, 08:00:53
@flood - I am referring to the 1st post and the original issue.
Do you have some reflectivity enabled for the shadowcatcher material? If no, then those "reflections" are in fact GI.
GI should be picked up by the shadowcatcher anyway, and should be visible in the alpha channel.
Basically, the image should look identical if you render the shadowcatcher in normal mode (e.g. on a blue background) and if you enable the "for compositing" mode, and then composite the image in Photoshop (e.g. with a blue background).

I am not exactly sure what is going on here, and the thread has become a bit chaotic. If you would like us to take a closer look at this, please contact us using this form https://coronarenderer.freshdesk.com/support/tickets/new

Hello Maru,

Reflectivity was not enabled for the shadowcatcher material. So yes, it's probably GI. I will create a ticket.