Author Topic: Time to ditch sRGB/Linear as default (?)  (Read 102157 times)

2018-11-08, 00:25:01
Reply #150

Jpjapers

  • Active Users
  • **
  • Posts: 1644
    • View Profile
No, it can be many other scene-referred file formats, like various camera raw formats which are often 12bit at best. But for CGI there are fewer: 32bit full-float .exr/.hdr/.tiff and half-float 16bit .exr/.tiff . I do not believe 3dsMax supports 16bit half-float Tiff, this is very obscure format, I think only used internally for Adobe's DNG or something along that line.

16bit integer Tiff with embedded gamma curve (regardless of in which gamma you write those colors in) is not the same thing as linear 16 bit half-float .exr.


Then it matters how you extract the information from that format. If you open 16bit .exr in Photoshop, it will open it in linear 32bit environment and will be able to extract shadows/highlights /manipulate exposure or tonemap. The file itself (16 bit .exr) will have obviously less information than 32bit file but you rarely need this for post-production (while you do need it for image based lighting with above 16+ dynamic range stops, i.e. Sunlight for example).

If you change the environment to 16bit in Photoshop, it will instantly clamp (or tonemap, it gives you options) and you won't be able to extract any further dynamic range. 16bit will still give you the advantage of wider tonal gradient to avoid artifacts and posterization effect. 16bit Tiff/PNG/etc.. will open to this mode directly unlike .exr. But even if you open them into 32bit environment, they will still be clamped.


TL:DR :- )

16bit Tiff from 3dsMax if you already used highlight compression in Corona and only want to adjust local contrast, colors, etc.
16bit .exr if you want to composite or extract dynamic range operations (exposure, highlights, shadows, tonemaping, glare/bloom, etc..).

Super useful thank you!

2018-11-08, 01:04:05
Reply #151

Rhodesy

  • Active Users
  • **
  • Posts: 553
    • View Profile
Yes very interesting stuff Juraj, thanks.

When I say '32bit minefield' I mean potential issues with very large file sizes high res files and lots of passes but mainly the fact like you explained that you only have a few options and tools you can use in PS in 32bit mode so you have to tone and then downgrade to 16bit. The problem comes if you need to re-render for any reason and you cant remember your PS tone mapping settings and tweaks you made before saving down to 16bit so your base renders might not match your previously finished image.

2018-11-08, 01:25:40
Reply #152

Rhodesy

  • Active Users
  • **
  • Posts: 553
    • View Profile
I've done a very crude test in Corona C4D exporting as a layered 16bit PSD and opening in PS. The first image shows the base rendered image - note PS identifies the image as being in linear space. The second shows it with the reflection pass switched on (linear dodge (add) mode as default blending style) and still linear. The final one shows what happens when converted to sRGB (still 16bit) and the same as I get out of MAX TIFF format. Any subtlety has gone in the reflection pass and its completely blown out.

I dont use passes to rebuild an image from scratch but I do use them to embellish the image, especially with reflection and refraction passes. If I used a low opacity soft brush on that middle image to mask in some glancing angle reflections it will give an enhanced look but when using the sRGB version you lose the nice gradients and tones.

I will need to do some experimenting in MAX but do you know if this is possible for a 16bit output that PS will recognise as linear?

Thanks

2018-11-08, 09:06:31
Reply #153

Juraj

  • Active Users
  • **
  • Posts: 4743
    • View Profile
    • studio website
Yes, I wrote that above, half-float .exr is 16bit internally. But to work with them properly, you need to be in linear environment, and Photoshop only offers this properly in their 32bit mode. And yeah...Photoshop doesn't give you all the tools in their 32bit mode, because Adobe doesn't care.

But Affinity does, I suggest you give it a try. It's not perfect yet but they have much better 32bit support.

Also, do you really need linear information for this kind of post-production ? You could simply stay in normal environment and use 'Screen' mode instead of 'Linear(Add)' which the name implies what it does :- ).
Please follow my new Instagram for latest projects, tips&tricks, short video tutorials and free models
Behance  Probably best updated portfolio of my work
lysfaere.com Please check the new stuff!

2018-11-08, 10:08:02
Reply #154

Rhodesy

  • Active Users
  • **
  • Posts: 553
    • View Profile
Thanks Juraj Im going to give the half float.exr a go.

Doh I'll have to give screen a go, Im just so used to working with the linear PSD files from C4D I didnt think to try switching the blending modes I just thought its was a limitation of linear vs sRGB. Thanks for the prompt!

Yeah I have affinity and I can see its potential but I perhaps didnt give it long enough to get used to the different personas. I like how photoshop just has all the tools in one place. Also the brush editor had some great features but not all the PS ones last time I tried, so some of my presets didnt transfer over too well from PS. I was also reluctant to jump in to another 'non industry standard' software full time. 

2018-11-08, 10:35:41
Reply #155

Juraj

  • Active Users
  • **
  • Posts: 4743
    • View Profile
    • studio website
Same here ! I have hard time transitioning. But for certain 32bit tasks, like working with our 360 HDRis, Affinity is godsend.

Screen is sRGB (I prefer to call it 2.2 gamma, as it can be any color-space, not just particularly sRGB) version of LinearAdd. It does the same, albeit for non-linear files.
It is hack of course, sorta, so it won't create super-hot level of light but I rarely find I would need that. I use Screen mode to overlay all kinds of lens glare effects and it recreates the look very successfully.
Please follow my new Instagram for latest projects, tips&tricks, short video tutorials and free models
Behance  Probably best updated portfolio of my work
lysfaere.com Please check the new stuff!

2018-11-08, 10:42:41
Reply #156

Rhodesy

  • Active Users
  • **
  • Posts: 553
    • View Profile
Always top knowledge Juraj! Yes I use screen quite a lot for blending stuff along with soft light and overlay. I just didnt think to try it with the raw passes but your info makes absolute sense! Thanks again

2018-11-08, 20:21:17
Reply #157

Njen

  • Active Users
  • **
  • Posts: 557
    • View Profile
    • Cyan Eyed
Screen is sRGB (I prefer to call it 2.2 gamma, as it can be any color-space, not just particularly sRGB) version of LinearAdd. It does the same, albeit for non-linear files.

Just to get technical before any misinformation is spread, 'Screen' is actually closer to a multiply than an add. The math is: invert both layers, multiply them, then invert the result.

2018-11-08, 21:31:18
Reply #158

Rhodesy

  • Active Users
  • **
  • Posts: 553
    • View Profile
Yes just done another crude test this time out of max with 16bit half float exr and opened in PS with EXR-IO.

The ball on the left is the converted down 16bit version with reflection pass set to screen. On the right is the 32bit original set to add. The ball on the right gives better tonal range with less washing out. I also ran a test from C4D outputting 32bit and 16bit PSD files and visually they looked the same as the 32bit exr version setting the ref to add. i.e. more natural tonal range and less washed out than the sRGB variants.

I've downloaded the PSD manager plugin so will give that a spin.

2018-11-08, 21:58:04
Reply #159

Rhodesy

  • Active Users
  • **
  • Posts: 553
    • View Profile
OK so just installed PSD manager and output a 16bit PSD. Interesting result. It has the file as sRGB and it must make the gamma 1 as it saves out and then adds a 2.2 gamma LUT at the top of the PS stack. This gives the nice tonal range I was getting from native C4D PSD 16bit output which PS reads as linear in its colour profile. It also matches the other 32bit tests from just now. I realise that the LUT is just an exposure adjustment layer setting the gamma to 2.2 so I've done that and it works.

So as I think Romullus pointed out you can do the gamma 1 output in MAX for saving out the 16bit TIFF. So Ive tried this along with the exposure adjustment layer fix in PS and it gives the same result which is great, if a little disconcerting that it seems like a bit of an unorthodox step and going against the 'recommended' setting.

My only real concern is, am I degrading my image by adjusting the gamma so severely or is the gamma in this case just a straight visual conversion / reassignment? so its not actually boosting the pixels artificially like if I was to add an aggressive curves layer, leading to banding and image degredation? 

Very much a geek chat this one!

2018-11-08, 23:56:41
Reply #160

Njen

  • Active Users
  • **
  • Posts: 557
    • View Profile
    • Cyan Eyed
Photoshop has a horrible way of handling data, I wouldn't trust it one bit. The best thing you can do is use a program like Nuke to combine images to test.

Also, when someone says '16bit' not everyone is talking about the same thing. There is '16bit integer', and '16bit half float'. Generally speaking, 16bit integer files are usually defaulted to save in sRGB format (unless specifically set), and 16bit half floats are linear.

If you use 'adds' on sRGB images, you are incorrectly combining the data. It first must have an inverse gamma curve applied before you can use add to get a physically accurate representation.

16bit half float and 32bit full float formats will look more or less exactly the same to the eye when using any kind of merge.

2018-11-09, 09:54:49
Reply #161

Rhodesy

  • Active Users
  • **
  • Posts: 553
    • View Profile
More good info, thanks Njen. There are a lot of people with a lot more knowledge than me!

I'm going to give the 1.0 gamma output setting with the 2.2 gamma correction in PS a try on a couple of projects as I really want to recreate that type of subtlety in the passes. I dont rebuild images from scratch but I do use them to boost bits here and there.

Do you know if there will be any image degredation by adding the gamma offset as an adjustment layer from 1 to 2.2 in PS?

Thanks

2018-11-09, 10:07:31
Reply #162

Juraj

  • Active Users
  • **
  • Posts: 4743
    • View Profile
    • studio website
Fully agree with that, I wrote the distinction between Integer and Half-float Tiff above, like three times :- ).

And that is imho the issue with the Rhodesy's current workflow. If you write data into integer Tiff with gamma 1.0, it's baked down, and you loose the range in blacks.
Try this quick experiment to prove it: Change your gamma (in 8 or 16bit mode in PS) to 0.454 (to simulate writing it to 1.0), bake it down, and then bring it back to 2.2. The result will be slightly wrong, noticeably in blacks.
This will not happen in PS's 32bit mode with either 16 or 32bit .exr/.hdr files.

Njen is right that compositing softwares, heck, even Adobe AfterEffects offer you the option of linear compositing regardless of whether you're in 8/16/32bit depth mode. In AE, you even have check-box to blend colors linearly in sRGB environment, and vice-versa (and you can interpret each pass/layer as sRGB/linear individually).
But Photoshop is strict and doesn't offer this distinction. 8/16bit mode in PS is clamped, output-referred workflow and 32bit is linear, scene-referred workflow.

Looking at the posted examples, almost all results look wrong to me. I get some feeling you're abusing compositing workflow to get artistic effect. Basically working out-of-the-box and that is completely fine, people get fantastic results with all sort of strange workflow they invented :- ). But the point of LinearAdd in compositing is the exact math of recreating how the passes add up in renderer for physically correct result. It's not to provide any gradients or range for artistic purpose. All the images seem to lack any form of tonemapping and feature oversaturated overblown highlights.

Photography retouchers don't work in linear and they can get any effect they want.

Look at my quick (300perc. boosted in size) example. The left is original 2.2 16bit Tiff. The right is one which was converted to gamma 1.0 and taken back to 2.2. (this should be identical as writing it to 1.0 directly)
« Last Edit: 2018-11-09, 10:15:02 by Juraj Talcik »
Please follow my new Instagram for latest projects, tips&tricks, short video tutorials and free models
Behance  Probably best updated portfolio of my work
lysfaere.com Please check the new stuff!

2018-11-09, 10:41:51
Reply #163

pokoy

  • Active Users
  • **
  • Posts: 1850
    • View Profile
Just a short note to those using 32bpc in PS - Photoshop will shift tones (especially shadows) when converting from 32bpc to 16 and 8 bits - be aware that you can't get the same output as from Max/VFB/CIE - this means any EXR image converted from 32bits to 16/8 bits in PS will not match a 16/8 bits image saved from Max/CIE.
This may not be noticeable most of the time but if you work on darker images it'll drive you crazy. Only workaround is to stay away from PS for any 32bpc work, as Juraj mentioned AE works correctly (as do other comp apps).

2018-11-09, 10:43:13
Reply #164

Rhodesy

  • Active Users
  • **
  • Posts: 553
    • View Profile
I see what your saying Juraj, yes that must bake it down. This is what PSD manager does natively which is odd as it must be tampering with the image. I just wish there was PSD export in Max that behaves the same as C4D, which seems more advanced.

Below is a crop of an image I rendered last night on one of our final C4D projects. On the left is the base render and on the right is the base + reflection pass (add). This is at 100% so I would just brush in highlights where needed at lower opacities as it is overblown in parts. But I find it a really handy tool to enhance bits here and there artistically even if its not physically correct. I just need to find a way to get Max to write the tiffs with half float and for PS to recognise it as 16bit like C4D does.

Maybe we should all switch to Affinity! I might dust it off.

EDIT: crap just seen your example image Juraj. That is annoying! More experimentation required then! Thanks for all your help