Alpha Issues

[dj_buckley]

Hi guys

I still seem to be struggling with getting properly clean alphas for Sky Replacements.  See attached.

This render is rendered on black.  Saved as 32-bit exr, then opened in photoshop, converted to 16-bit, alpha applied, then a colour layer behind to show halos.

As you can see there's a slight halo (a dark one), if I use 'remove black matte' the halo turns white, if I use 'defringe' it looks much better but this screws up any transparent/refractive objects where the black background was visible beyond.

Can anyone help with the correct process to avoid this when saving to 32-bit exr's.  Is there a 'correct' order of operations when opening the .exr in photoshop and converting to 16-bit, replacing backgrounds to avoid this?  Am I missing something

The issue always magnifies if you do any heavy post work using additional masks etc.

It's worth noting I'm using ProExr for opening files, I've tried both 'Alpha Appears on Separate Layer' and 'Alpha Makes Layer Transperent' in the ProEXR dialogue.  Each option gives a slightly different result, 'Alpha Makes Layer Transperent' gives a slightly better result, but both give halos.  All appears fine while still in 32-bit, but when you convert to 16-bit the halos become more obvious.  Today I'll be testing Photoshop's native loader and EXR I/O.

Maybe a better question would be "What's the correct method for saving 32-bit exrs and ensuring a pixel.perfect alpha in Photoshop"

[Frood]

In short: Photoshop (PS) is the problem. When converting to 16bit, your alpha is handled like standard color information and gamma will be applied. You have to de-gamma the alpha by -for example- adding "exposure" to it and setting gamma correction to 0.45. For "pixel perfect alpha in Photoshop" you would have to go through all the (evil, evil!) "premultiplied" myths and facts as well, but you will never get happy using PS here.


Good Luck

[dj_buckley]

Thanks Frood, you've just confirmed my assumptions, that PS is the issue.

If I bring in the .exr using ProEXR with the option 'Alpha Makes Layer Transparent' it's negligible so I'll probably stick with that for now, unless you recommend an alternative software, I do have access to Fusion so maybe it's time to learn that.  I'm getting more and more annoyed with PS anyway these days so maybe it's time to switch.

Edit: apologies Frood (Droid), typing on my phone here and it autocorrect wins again

[LorenzoS]

n short: Photoshop (PS) is the problem

Hi Frood,
can you recommend an alternative to PS that works?

thank you

[BossCo]

Hello guys,
What I usually do is Refine the mask by shifting the edge in the minus direction...

To access this option in PS CC you have to: Click on your mask, then with a Shift key pressed you click on "Select>Select and Mask"...

I hope this will help.

PS I also tried with Affinity photo but results are the same like PS.

[dj_buckley]

Thanks guys

Glad it's not just me with the issue.

I'll keep testing other methods

[Frood]

I would not claim that you cannot use PS at all, you just have to solve some issues and have to know what happens. Personally I stick to de-gamma the alpha(s) while rendering on black (or a dedicated/pre-edited background) if the alpha areas should be tweaked in any way.
There are others solving EXR-PS issues by creating a linear profile to load files and then applying gamma to beauty passes for example, leaving the alpha intact as a data channel. I still have not found any good solution to unmultiply the background from the beauty, regardless of which EXR loader is used though. As a consequence, vast changes to the background are not always possible, instead you'd have to adjust the scene (background). Since the choice of a postpro software is a very individual one and depends on what you need, I cannot recommend any number one. Personally I would not trade layer based freedom and habits I got used to for a correctly unmultiplied beauty in a node based solution for stills. For animation there are other requirements of course.

Edit: apologies Frood (Droid), typing on my phone here and it autocorrect wins again

*chirp!* No problem, Sir! *bleep!* I'm going to switch myself off now to save power, Sir! *blip!*


Good Luck, Sir!
*click*

[maru]

Photoshop just doesn't have the necessary options to load the files with alpha channel correctly. Or at least I am not aware of such an option.
If you open the same file in "proper" compositing software like Fusion, you will see that the edges will be perfect regardless if you rendered against black or white background, sky, or RGB 100000 HDR.

[LorenzoS]

If you open the same file in "proper" compositing software like Fusion, you will see that the edges will be perfect regardless if you rendered against black or white background, sky, or RGB 100000 HDR.

Hi maru, from your experiences the worflow usig Fusion works also with 8 or 16bit images?

thanks

[maru]

If you open the same file in "proper" compositing software like Fusion, you will see that the edges will be perfect regardless if you rendered against black or white background, sky, or RGB 100000 HDR.

Hi maru, from your experiences the worflow usig Fusion works also with 8 or 16bit images?

thanks

I do not really have practical experience with it (like actually working with it).
It opens 8/16/32-bit images just fine.

[Frood]

Photoshop just doesn't have the necessary options to load the files with alpha channel correctly.

I think it's not that easy. You can even get an unpremultiplied RGB (what a monster of a word) using the native loader when choosing "as transparency", at least that's what it looks like. Multiply the result with alpha and you have everything you basically need (unpremultiply -> color correct -> premultiply in terms of a node based package). The main difference to e.g. Fusion is imho that you do not have the slightest insight/control over what's happening. In Fusion you have full control, even discrete gamut and gamma processing for alpha channels. And then of course there is the lack of 32bit processing features of PS generally, forcing us to convert to 16bit with all consequences.


Good Luck