Author Topic: Cryptomatte playground!  (Read 9681 times)

2022-02-03, 13:09:29

GeorgeK

  • Corona Team
  • Active Users
  • ****
  • Posts: 838
  • George
    • View Profile
With the release of the latest daily build of Corona version 8 2022-02-02, Cryptomatte render element has been added.

>>> For this daily, be sure to disable the allow propagation through reflection/refraction from the render elements > cryptomatte rollout, as it might create some issues <<<"

It can be found in the Render Setup (F10) > Render Elements > Add > CMasking_Cryptomatte. You can further customize it by Levels control, where you can determine the number of objects that can be identified by the cryptomatte in a single pixel, and by ID type you can determine the distinction between different objects. It supports motion blur, transparency and depth of field.

Note: Render outputs of this render element need to be saved as .CXR (do not save as OpenEXR) and either be renamed to .EXR when saving, or in the case of .CXR you can always select to open the file with "Open as..." and select which plugin you want to use. 



Please feel free to post your creations here, and offer any valuable feedback, using this new feature for Corona Renderer v8.

« Last Edit: 2022-02-04, 09:06:00 by GeorgeK »
George Karampelas | chaos-corona.com
Chaos Corona QA Specialist | contact us

2022-02-07, 16:03:31
Reply #1

romullus

  • Global Moderator
  • Active Users
  • ****
  • Posts: 8779
  • Let's move this topic, shall we?
    • View Profile
    • My Models
I'm not an expert, but it appears that cryptomate works just fine in Affinity Photo, no need to rename, or do anything special. On the other hand, it looks that cryptomate completely ignores volumetrics, will this be improved in the future?
I'm not Corona Team member. Everything i say, is my personal opinion only.
My Models | My Videos | My Pictures

2022-02-09, 12:03:58
Reply #2

romullus

  • Global Moderator
  • Active Users
  • ****
  • Posts: 8779
  • Let's move this topic, shall we?
    • View Profile
    • My Models
Does cryptomate element need to be rendered for multiple passes for antialiasing, or just one pass is enough. The preview shows zero antialiasing, but i suspect it may look very different under the hood.
I'm not Corona Team member. Everything i say, is my personal opinion only.
My Models | My Videos | My Pictures

2022-02-09, 14:04:48
Reply #3

TomG

  • Administrator
  • Active Users
  • *****
  • Posts: 5434
    • View Profile
It needs multiple passes - the colored layer visible in the VFB will always look "pixellated" and won't show any changes after the first pass or two, but the actual usable layers will improve with multiple passes. The best advice is to render the scene fully, until the Beauty is clean, then the cryptomatte passes will be clean too.

EDIT - especially true for motion blur and DOF, which is where cryptomatte is most useful. Those need to have cleaned up in the beauty for the matte to be able to be accurate and effective :)
Tom Grimes | chaos-corona.com
Product Manager | contact us

2022-02-09, 14:32:04
Reply #4

romullus

  • Global Moderator
  • Active Users
  • ****
  • Posts: 8779
  • Let's move this topic, shall we?
    • View Profile
    • My Models
Thanks for clarification Tom, it's a pity that there's no proper visual representation in the VFB though - sometimes you just need to render the masking elements alone, without any shading and it's impossible to tell wether cryptomate has enough antialiasing, or not. Not a problem when you're rendering everything in one go, of course.
I'm not Corona Team member. Everything i say, is my personal opinion only.
My Models | My Videos | My Pictures

2022-02-10, 14:51:37
Reply #5

marchik

  • Active Users
  • **
  • Posts: 256
    • View Profile
I'm not an expert, but it appears that cryptomate works just fine in Affinity Photo, no need to rename, or do anything special. On the other hand, it looks that cryptomate completely ignores volumetrics, will this be improved in the future?

Could you please explain in more detail how exactly it works in Affinity Photo?

For me the .cxr file opens without any passes, merged
and the renamed .exr has cryptomatte passes similar to corona vfb, but I can’t use them, as it is intended

2022-02-10, 15:13:43
Reply #6

romullus

  • Global Moderator
  • Active Users
  • ****
  • Posts: 8779
  • Let's move this topic, shall we?
    • View Profile
    • My Models
I didn't know that there should be separate passes. I opened CXR and sampled colors from the mask. As i said, i'm not an expert, if cryptomate supposed to work somehow differently, i would like to learn that :]
I'm not Corona Team member. Everything i say, is my personal opinion only.
My Models | My Videos | My Pictures

2022-02-10, 15:25:14
Reply #7

TomG

  • Administrator
  • Active Users
  • *****
  • Posts: 5434
    • View Profile
Oh, it is supposed to work differently - that colored pass is NOT cryptomatte. For one thing it does not resolve motion blur and depth of field, you will just get solid "on or off" pixels, while the cryptomatte pass for that object will in fact have correct fading of the mask. For example, the first image below is how it would look in that pass, and the second is how it will look in the layer.

When loaded by software that understands cryptomatte data, you will get one greyscale layer for each named object (or layer, or layer + name, etc. depending on the cryptomatte settings in the Render Element), see third image. You can then copy and paste the layer for a particular object into a mask for, say, an Adjustment Layer.
Tom Grimes | chaos-corona.com
Product Manager | contact us

2022-02-10, 15:30:34
Reply #8

romullus

  • Global Moderator
  • Active Users
  • ****
  • Posts: 8779
  • Let's move this topic, shall we?
    • View Profile
    • My Models
Oh crap :[ I wonder if it would be possible to add an option to export separate layers instead of single cxr file? For those that don't have cryptomate compatible software?
I'm not Corona Team member. Everything i say, is my personal opinion only.
My Models | My Videos | My Pictures

2022-02-10, 23:52:53
Reply #9

shortcirkuit

  • Active Users
  • **
  • Posts: 307
    • View Profile
im still confused by cryptomatte - with wire color i usually use the magic want tool  in photoshop to select the color i want - is this how we do this in photoshop for cryptomatte?  I usually save out my renders as 16bit tiffs so im not sure if am doing things right - can you please outline how we get these perfect masks as i cant work it out for the minute


Oh, it is supposed to work differently - that colored pass is NOT cryptomatte. For one thing it does not resolve motion blur and depth of field, you will just get solid "on or off" pixels, while the cryptomatte pass for that object will in fact have correct fading of the mask. For example, the first image below is how it would look in that pass, and the second is how it will look in the layer.

When loaded by software that understands cryptomatte data, you will get one greyscale layer for each named object (or layer, or layer + name, etc. depending on the cryptomatte settings in the Render Element), see third image. You can then copy and paste the layer for a particular object into a mask for, say, an Adjustment Layer.

2022-02-11, 04:05:05
Reply #10

TomG

  • Administrator
  • Active Users
  • *****
  • Posts: 5434
    • View Profile
Unlike other masks, which use a "puzzle matte" approach of a single layer with different colors in them to denote different objects, materials, etc., Cryptomatte places the mask for each item in its own separate layer in the file (the CXR, which you can rename to EXR). The layers will be named after each object, or layer (in the host application), or name and layer, etc. as determined by what options you checked in the cryptomatte render element, and will contain a greyscale mask for that object (or layer, or etc. depending on what you checked in the cryptomatte settings). The fact this color map type layer exists is just a side effect of the calculations, and is not the actual cryptomatte data in usable form, and can be ignored (and deleted, even, once in Photoshop).

To see these layers, the application you use to load it MUST understand cryptomatte data, otherwise you won't see them. For Photoshop, this would mean using Exr-IO, which in the latest version can read and translate cryptomatte data. There is other software out there that has a native understanding of cryptomatte, such as Nuke, but I don't have that so can't comment on the workflow :)

You can check tutorials like https://youtu.be/apk11RaFKo8?t=247 as an example (also see the 5 minute 40 second mark, where they copy and paste the cryptomatte mask for an object into the mask for an adjustment layer, which is how you use it to refine, say, exposure for a particular object, or color tint, or saturation, or whatever adjustment layer you are using).

Hope that helps! Naturally when time allows, we'll be making our own tutorials going through from start to finish too :)
Tom Grimes | chaos-corona.com
Product Manager | contact us

2022-02-11, 04:07:00
Reply #11

TomG

  • Administrator
  • Active Users
  • *****
  • Posts: 5434
    • View Profile
PS, as noted at the start of this thread, CXR is the ONLY file format that can save cryptomatte data from Corona at the moment, so you must save to that.
Tom Grimes | chaos-corona.com
Product Manager | contact us

2022-02-11, 10:34:19
Reply #12

romullus

  • Global Moderator
  • Active Users
  • ****
  • Posts: 8779
  • Let's move this topic, shall we?
    • View Profile
    • My Models
Unlike other masks, which use a "puzzle matte" approach of a single layer with different colors in them to denote different objects, materials, etc., Cryptomatte places the mask for each item in its own separate layer in the file (the CXR, which you can rename to EXR). The layers will be named after each object, or layer (in the host application), or name and layer, etc. as determined by what options you checked in the cryptomatte render element, and will contain a greyscale mask for that object (or layer, or etc. depending on what you checked in the cryptomatte settings). The fact this color map type layer exists is just a side effect of the calculations, and is not the actual cryptomatte data in usable form, and can be ignored (and deleted, even, once in Photoshop).

To see these layers, the application you use to load it MUST understand cryptomatte data, otherwise you won't see them. For Photoshop, this would mean using Exr-IO, which in the latest version can read and translate cryptomatte data. There is other software out there that has a native understanding of cryptomatte, such as Nuke, but I don't have that so can't comment on the workflow :)

You can check tutorials like https://youtu.be/apk11RaFKo8?t=247 as an example (also see the 5 minute 40 second mark, where they copy and paste the cryptomatte mask for an object into the mask for an adjustment layer, which is how you use it to refine, say, exposure for a particular object, or color tint, or saturation, or whatever adjustment layer you are using).

Hope that helps! Naturally when time allows, we'll be making our own tutorials going through from start to finish too :)

Just to clarify - does that mean, that information about cryptomatte usage on official Affinity channel is incorrect? https://affinity.serif.com/en-gb/tutorials/photo/desktop/video/365275093/
I'm not Corona Team member. Everything i say, is my personal opinion only.
My Models | My Videos | My Pictures

2022-02-11, 12:32:10
Reply #13

TomG

  • Administrator
  • Active Users
  • *****
  • Posts: 5434
    • View Profile
Pretty much, yep. You can use it like that, but it won't work with motion blur and DOF (as can be seen by how pixelated that is in my earlier example of the colored single layer), and really whatever they are doing there with the cryptomatte can be done with other ID matte passes like object, material, etc. The main benefit of cryptomatte is handling DOF and moblur because it is a proper greyscale and not just "on" or "off" (since both effects blend the object in question into what is seen in the background), not having to worry about fringing/aliasing as those should be accounted for afaik, plus having layers that are named and correspond to objects or groups of objects so you can find the mask that applies (which should allow you for example, in their case, to have all the windows on the building on the left grouped and appearing in one mask layer, not as 30 different colors).

So, does their use work? Sure, because nothing is moving or blurred or out of focus - but you can just do that, more correctly, with other types of such "puzzle matte" single layer solutions anyway. None of this is what cryptomatte is intended for, nor is it using the "real and full" cryptomatte data.
Tom Grimes | chaos-corona.com
Product Manager | contact us

2022-02-11, 14:49:29
Reply #14

davetwo

  • Active Users
  • **
  • Posts: 298
    • View Profile
You can see how Photpshop handles the Cryptomatte masks iwith the aid of the Exr-IO in this V-ray tutorial: https://vrayschool.com/vray-cryptomatte-exr-in-photoshop/

2022-02-11, 15:29:32
Reply #15

TomG

  • Administrator
  • Active Users
  • *****
  • Posts: 5434
    • View Profile
You can see how Photpshop handles the Cryptomatte masks iwith the aid of the Exr-IO in this V-ray tutorial: https://vrayschool.com/vray-cryptomatte-exr-in-photoshop/

Yep, good one! This is also what to expect from Corona from what I've seen, and how to work with it.
Tom Grimes | chaos-corona.com
Product Manager | contact us

2022-02-11, 15:56:36
Reply #16

romullus

  • Global Moderator
  • Active Users
  • ****
  • Posts: 8779
  • Let's move this topic, shall we?
    • View Profile
    • My Models
As nice and cool cryptomatte is, it would not make me get back to photoshop. Nope, thank you :]
I'm not Corona Team member. Everything i say, is my personal opinion only.
My Models | My Videos | My Pictures

2022-02-11, 16:03:04
Reply #17

TomG

  • Administrator
  • Active Users
  • *****
  • Posts: 5434
    • View Profile
You may want to send a feature request to Affinity ;)

Software that can handle cryptomatte is listed as Autodesk Flame, After Effects, Nuke, Blender (Compositing Nodes), Houdini (COP), V-Ray (which can handle them in the V-Ray frame buffer itself), and Photoshop with Exr-IO 2. If anyone knows of more, feel free to comment!
Tom Grimes | chaos-corona.com
Product Manager | contact us

2022-02-11, 16:56:29
Reply #18

romullus

  • Global Moderator
  • Active Users
  • ****
  • Posts: 8779
  • Let's move this topic, shall we?
    • View Profile
    • My Models
You may want to send a feature request to Affinity ;)

I will. I already saw some similar requests on their forum, but they are surprisingly very few - i guess not many 3D designers use Affinity products.
I'm not Corona Team member. Everything i say, is my personal opinion only.
My Models | My Videos | My Pictures

2023-09-17, 17:19:17
Reply #19

mlon

  • Active Users
  • **
  • Posts: 36
    • View Profile
Hey there, is it possible to have cryptomatte work without shading enabled?
If i disable shading in render overrides settings it gives no alpha masks in the crx.
ID for instance does work when shading is disabled. This saves allot of time.

mlon

2023-09-18, 16:46:36
Reply #20

maru

  • Corona Team
  • Active Users
  • ****
  • Posts: 12710
  • Marcin
    • View Profile
Cryptomatte relies on shading (things like reflections, refractions, etc). So I don't think this is possible. If you need "simpler" masking with no shading, then using the other masking elements sounds like a better approach: https://docs.chaos.com/display/CRMAX/Render+Elements#heading-CMasking
Marcin Miodek | chaos-corona.com
3D Support Team Lead - Corona | contact us

2023-09-18, 17:43:53
Reply #21

Ink Visual

  • Active Users
  • **
  • Posts: 159
    • View Profile
Is there any way to reduce the number of information that is stored in CXR, and limit it to only elements crucial for Cryptomatte to work?
The main cxr file output often results in extremely large file size. This makes it unusubale for VFX, (I mean try to manage hundreads of frames of 10GB size each).

For most projects we only save the main pass as .CXR, and all the other render elements are saved as JPEGs (with help of Pulze SM).
Cryptomatte element can also be saved as .CXR (keeping everything else as JPEGs), but as I understand, by curent design, this doesn't matter, becaouse in order to use Cryptomatte we still have to use main .cxr pass anyways. Can that be changed, so that if cryptomatte element is set with .CXR extension, it contains all the data (and only that data) that is needed for it to work?

I'm just not sure what contributes to .CXR file size, maybe there's some other way to reduce it for cryptomatte use?
« Last Edit: 2023-09-18, 17:53:07 by Ink Visual »

2023-09-19, 08:51:53
Reply #22

marchik

  • Active Users
  • **
  • Posts: 256
    • View Profile
Is there any way to reduce the number of information that is stored in CXR, and limit it to only elements crucial for Cryptomatte to work?
The main cxr file output often results in extremely large file size. This makes it unusubale for VFX, (I mean try to manage hundreads of frames of 10GB size each).

For most projects we only save the main pass as .CXR, and all the other render elements are saved as JPEGs (with help of Pulze SM).
Cryptomatte element can also be saved as .CXR (keeping everything else as JPEGs), but as I understand, by curent design, this doesn't matter, becaouse in order to use Cryptomatte we still have to use main .cxr pass anyways. Can that be changed, so that if cryptomatte element is set with .CXR extension, it contains all the data (and only that data) that is needed for it to work?

I'm just not sure what contributes to .CXR file size, maybe there's some other way to reduce it for cryptomatte use?

+1
 I was glad to see cryptomatte support, but due to the need to store huge CXRs for sequences, I just stopped using it

2023-09-19, 11:45:27
Reply #23

maru

  • Corona Team
  • Active Users
  • ****
  • Posts: 12710
  • Marcin
    • View Profile
CXR stores all information used by Corona for rendering, so it can sometimes get pretty huge, yes. The size comes from:
- All render elements including LightMix (if enabled)
- The hidden denoising elements (they are not visible in the VFB but they are used by denoisers - this way you can, for example, open a CXR in CIE and denoise it there)
- Internal elements used by Corona for the "resume from file" feature (importance, weights) - these should not be heavy though

Going back to the original issue, I think we can consider two separate things here:
- CXR files can get huge, maybe we could improve this. Maybe a new file format with only the cryptomatte content without the extra stuff? Maybe CXR but user can choose what exactly to store in it?*
- The way Cryptomatte is saved is currently not ideal. Maybe we could make it save as a regular render element? Optimize the process some other way?*

* Please do not treat these as any kind of promises. Just thinking out loud. :)

Anyway, we now have this logged in our internal systems:
CXR files are huge: (Internal ID=1204935321)
Cryptomatte saving improvements: (Internal ID=949161620)
 
Marcin Miodek | chaos-corona.com
3D Support Team Lead - Corona | contact us

2023-09-20, 03:52:34
Reply #24

marchik

  • Active Users
  • **
  • Posts: 256
    • View Profile

- The way Cryptomatte is saved is currently not ideal. Maybe we could make it save as a regular render element? Optimize the process some other way?*


Yes, in my case, I usually render more complex sequences (where post-processing using cryptomatte may be needed) on render farms, and even the banal transfer of 1000 .cxr files via the Internet to my computer can be painful.

 It would be cool if we could save a separate, not so heavy rendering element of the cryptomatte and then, say, in Fusion, do a channel merge and use it as usual.

2023-09-20, 09:02:54
Reply #25

Frood

  • Active Users
  • **
  • Posts: 1903
    • View Profile
    • Rakete GmbH
Still +1 (ever since Cryptomatte has been implemented). It's on top of my list of Corona's

"Missing/lost or unusable features (with the possible exception of a developers teapot scene)"


Good Luck


Never underestimate the power of a well placed level one spell.

2023-09-21, 09:43:45
Reply #26

Place

  • Active Users
  • **
  • Posts: 43
    • View Profile

2023-09-29, 10:26:34
Reply #27

busseynova

  • Active Users
  • **
  • Posts: 21
    • View Profile
Any plans to add cryptomatte by material to the options?

I used this all the time in VRay and I'm struggling without it!

2023-09-29, 12:18:33
Reply #28

dj_buckley

  • Active Users
  • **
  • Posts: 871
    • View Profile
Any plans to add cryptomatte by material to the options?

I used this all the time in VRay and I'm struggling without it!

+1 always do my masks 'By Material'