Chaos Corona Forum
Chaos Corona for 3ds Max => [Max] I need help! => Topic started by: Light&Bit on 2017-12-05, 11:05:05
-
Hi chaps,
as you can easly understand from the subject of my topic I am getting mad when I have to extract the RGB mask passes from the render elements in Corona, especially if I have trees (and this means opacity channel with a bitmap for the leaves) in my scene I have noticed the edges are not cleaned at all, there is always a border of 1-2 pxl (I have already tried to save them in different formats, tiff, tga, png but always the same problem) and this is quite frustrtating because there are useles if I have to do PP in photoshop.
any suggestion about how fix this issue?
thanks in advance
Dan
-
I think I know what your problem is. But to be sure I understood right: Are you using a Mask RE for your trees or have you set up an RGB-pass: A render with 'solid' red, green and blue shaders combining 3 masks within an rgba-render?
Screenshot would be great, too. thanks
-
hey mate,
I have assigned an objectd Id for each object and then added a CMasking_Mask in RGB mode, please see attachement. with Vray I have never had this issues.. :/
-
missing one attachment sorry
-
I would recommend to use Mask RE, not the RGB-solution, because you might get in trouble when objects overlap. But from the last screenshot I see that you're not rendering on a black BG?
BG always have to be black, otherwise you will get fringes in the edges.
-
so this means everytime I have to render out the masks i have to re-render the scene replacing the BG with just a black solid color? mh.....let me try, thanks for the suggestion.
Best,
Dan
-
You don't have to rerender. You render your beauty with a clean alpha(on black!) and then you add your sky/bg plate in post.
-
mh...ok I am not following you anymore.
I don't need to replace the sky, for example I need to tweak the curves/levels for few objects, like the trees.
So even after I tried to render out the masks with a black BG (I have to use the option "Render only masks" because otherwise Max runs out of Ram and crashes ) the problem is still the same the edges of my masks are not clean and sharp and they are not able to work properly in photoshop.
-
https://en.wikipedia.org/wiki/Alpha_compositing
or do a bit of googeling 'premultiplied alpha'. I know it is a bit of pain in the ass, but beneficial once you get your head around it.
Can you check the bitmaps you are using for the leaves? Maybe the problem with the alpha is there.
-
I know how fix it easly in AE or Nuke, but in Photoshop is a bit limited. but the point is it must be come out correct for Max and Corona somehow.
-
I know how fix it easly in AE or Nuke, but in Photoshop is a bit limited. but the point is it must be come out correct for Max and Corona somehow.
It's actually a limitation of PS. You can use the defringe command in PS but it's destructive. There used to be a site with another way to set it up but it's not online anymore and I don't remember the details. But this also involved deleting image portions.
Another idea is the Mask Edge command which can be really useful and may get you there.
-
It's actually a limitation of Corona :)
At least additionally. Because unfortunately you don't get a proper mask as soon as opacity is used.
This is not about aliasing or rendering on non-black/premultiplying. What dboldicotti wants to do is impossible right now with simple render elements because all of them: refraction, reflection and yes - even opacity (used here) is not handled correctly by masks at the moment.
Here is some kind of workaround - reminding of dark mr ages where you always had to have some additional "render elements scene":
https://forum.corona-renderer.com/index.php?topic=17085.msg106956#msg106956
Good Luck
-
Straight to the point, thanks Frood to confirm my doubt about the masks elements.
Best
Dan
-
I can't be sure how it would work in Corona, and I didn't tried it for the last few years, but in the past I used to export passes with tga and uncheck "Pre-multipled Alpha". In every viewer like XnView the picture would look strange, but in Photoshop and After Effects the edges would look just like it should. In AE I think you need to right click the image and select Interpret Alpha and select Unmultiplied or something.
(http://help.autodesk.com/cloudhelp/2017/ENU/3DSMax/images/GUID-2C86B9A6-4169-4CAC-9C09-645BAAFC2A10.png)
Another benefit - if you leave "Compress" checked - that's lossless (meaning identical to uncompressed) but very fast compression - so good for playback.
So Compress ON , Pre-multipled Alpha OFF
-
@Dan. Sorry man. I just saw your render against a nonblack BG and immediately thought of a premultiplying issue. Users complaining 'can't get rid of this fringe' do usually something wrong with the alpha.
I am not able to do some tests, because I have no active license at the moment. But as I understood Frood: At the moment you can't have a tree with 'sprite' mapped leaves and get a working RE mask for that? Is that right?
I can't be sure how it would work in Corona, and I didn't tried it for the last few years, but in the past I used to export passes with tga and uncheck "Pre-multipled Alpha". In every viewer like XnView the picture would look strange, but in Photoshop and After Effects the edges would look just like it should. In AE I think you need to right click the image and select Interpret Alpha and select Unmultiplied or something.
(http://help.autodesk.com/cloudhelp/2017/ENU/3DSMax/images/GUID-2C86B9A6-4169-4CAC-9C09-645BAAFC2A10.png)
Another benefit - if you leave "Compress" checked - that's lossless (meaning identical to uncompressed) but very fast compression - so good for playback.
So Compress ON , Pre-multipled Alpha OFF
It is about to know what you're target application expects: pre or unpremultplied. And you usually can tell your software how to treat the imported footage as well.
-
It's actually a limitation of Corona :)...
Sorry then, I assumed it's the typical white fringe problem in PS. Not working with opacity masked geometry too often so didn't come across this one... yet. Sorry for the confusion!
-
I just did some tests. I'm curious about opinions. It seems much easier to me, you don't need to process with special premult nodes in Nuke, or run a defringe, or other stuff. And the objects are always perfect with no dark or white edges.
I saved some renders on a black background with alpha channel and merged them on a background in After Effects. The images are zoomed in 200% for clarity
regular png render - notice the dark edges :
(https://forum.corona-renderer.com/index.php?action=dlattach;topic=18488.0;attach=75420;image)
..........
png and in AE I selected "premultipled with black. Notice the white edges
(https://forum.corona-renderer.com/index.php?action=dlattach;topic=18488.0;attach=75422;image)
..........
tga saved with premult checked. Again dark edges.
(https://forum.corona-renderer.com/index.php?action=dlattach;topic=18488.0;attach=75424;image)
.....
tga with premult unchecked - perfect edges
(https://forum.corona-renderer.com/index.php?action=dlattach;topic=18488.0;attach=75426;image)
-
I saved some renders on a black background with alpha channel
Well, that's the difference. Alpha works correctly. But this issue is about masks created by render elements like Cmasking_Mask. Try to do a standard rendering + mask for your foliage and adjust e.g. levels with that mask. It will be wrong because CMasking_Mask does not respect opacity values between 0 and 1 but only 0 (=no mask) and different from 0 (=mask).
Good Luck
-
And what about if material would have clip opacity checked on, would it still render wrong in masks?
-
Simple - if you will always render your beauty pass with a background in it. Your BG/Backplate blue/white/light pixels will bake in all of your corner pixels of your alpha channel. Windows corners etc and trees are the worst.
If you know that you will change you bg later on - render your image with black BG.
If you rendered it with a sky or some kind of color - sometimes in photoshop Layer>Matting (Defringe 1px) or remove white matte helps. But sometimes it does not help. Sometimes you can get artifacts etc - make it even worse.
Remember the smaller resolution you are rendering the worse it will get. I think 3-4k image minimum is good. For doing Layer>matting.
Good luck
-
And what about if material would have clip opacity checked on, would it still render wrong in masks?
In theory the mask would be ok because mask clips values > 0 and opacity clips values > 0.5 afaik. So the mask would be correct but only if those objects do not occlude each other. And you need a base rendering with clipping turned on of course. So all in all not feasable. Only solution is the ShadowCatcher workaround so far.
Good Luck
-
Well, that's the difference. Alpha works correctly. But this issue is about masks created by render elements like Cmasking_Mask.
But it seems like when using regular methods and using png or other formats you will get edges - if you are not using the method I posted above. You'll get some kind of edges and you'll need to remove those with certain methods in AE or Nuke.
-
@Frood, when rendering foliage, clipping opacity is the prefered way, but of course if occlusion is a problem, then that isn't solution either...
-
I saved some renders on a black background with alpha channel
Well, that's the difference. Alpha works correctly. But this issue is about masks created by render elements like Cmasking_Mask. Try to do a standard rendering + mask for your foliage and adjust e.g. levels with that mask. It will be wrong because CMasking_Mask does not respect opacity values between 0 and 1 but only 0 (=no mask) and different from 0 (=mask).
Good Luck
I'm not convinced alpha is working correctly (and by extension the masks aswell). I set up a scene with a tree that has a black background and a 1.0 white self-illumnated material on the tree(no opacity maps), with tonemapping disabled and the image filter set to none. In my screen caps from the VFB you can see the difference between the two... Perhaps I'm misunderstanding something, but in this scenario shouldn't they be identical? What's stranger is that the alpha shows it actually encroaches 1px into the beauty with a black background. If you do the opposite scenario, 0 albedo tree with white background, the alpha expands past the bounds of the beauty by 1px, producing the halo effect. Peculiar.
-
I'm not convinced alpha is working correctly (and by extension the masks aswell).
I think you have to apply gamma 2.2 to alpha or 1/2.2 to the beauty and then compare the result - it should match (don't use JPG doing so), maybe you can check this.
But once again @sebastian___ and RomasNoreika: The original issue is NOT about alpha or premultiply or rendering on non-black or whatsoever. It's the Corona mask which does not take opacity fully into account, appart from any post problems that may arise. The comparison I posted here (https://forum.corona-renderer.com/index.php?topic=18488.msg115947#msg115947) has a white alpha everywhere (BG and FG is rendered).
Render a gradient opacity object (you can use this (https://forum.corona-renderer.com/index.php?action=dlattach;topic=17085.0;attach=68790;image) map from that thread (https://forum.corona-renderer.com/index.php?topic=17085.0) for convenience) and try to get a mask from objects using it. This is what happens in small scale using opacity maps with values other that 0 or 1 on foliage and causes dboldicotti's problem.
Vote Refraction/Reflection working with masking render elements (CMasking_Mask,CTexmap, etc...) (https://forum.corona-renderer.com/index.php?topic=96.0) to hopefully someday get rid of it :)
Good Luck