Chaos Corona Forum

Chaos Corona for 3ds Max => [Max] General Discussion => Topic started by: romullus on 2018-11-13, 20:31:18

Title: Image file format and RAM consumption
Post by: romullus on 2018-11-13, 20:31:18
When talking about RAM consumption during rendering, does it matter what file format is used for textures? Will 5 MB JPEG texture take less memory than the same texture saved in 100 MB TIFF? Or they both will take roughly the same amount of RAM?
Title: Re: Image file format and RAM consumption
Post by: Ryuu on 2018-11-14, 10:24:47
Corona stores the images in memory uncompressed, so this really depends on resolution and bit depth. An 8-bit 1024x1024 JPEG will take exactly the same amount of memory as an 8-bit 1024x1024 TIFF no matter what's their size on disk.
Title: Re: Image file format and RAM consumption
Post by: romullus on 2018-11-14, 11:03:43
Thank you for the answer! If i understand correctly, given the same image dimension and bit depth, the only thing that matters, is number of channels? So 1024x1024 8 bit texture will take 1 MB RAM if it's greyscale, 3 times as much if it's RGB and 4 times as much if it's RGBA?

Title: Re: Image file format and RAM consumption
Post by: romullus on 2018-11-14, 11:19:55
Also, what is the better option - to have 3 single channel textures, or 1 RGB texture, in terms of RAM consumption, scene parsing time?
Title: Re: Image file format and RAM consumption
Post by: Ryuu on 2018-11-14, 13:08:12
Thank you for the answer! If i understand correctly, given the same image dimension and bit depth, the only thing that matters, is number of channels? So 1024x1024 8 bit texture will take 1 MB RAM if it's greyscale, 3 times as much if it's RGB and 4 times as much if it's RGBA?

Yes, that's correct.

Also, what is the better option - to have 3 single channel textures, or 1 RGB texture, in terms of RAM consumption, scene parsing time?

Good question. I never tried to measure it, but as long as we're talking about 3 files vs 1 file, there should not be any noticeable difference in memory consumption or parsing time. When we get to 3000 vs 1000, the difference might be a bit more noticeable in terms of parsing time as there is some small overhead with opening each file, etc.

The overhead is much higher when the texture files are stored in a network location, as there are higher latencies involved in opening the individual files, etc.

So, I would definitely not worry about this unless you have problems with the parsing times and have a reason to suspect that the texture loading might be the cause of those problems.

One of the key points of Corona philosophy has always been that users should not worry about technical details. So if you ever get to the point where 3 files vs 1 makes any noticeable difference, I would prefer to solve this by optimizing the texture loading code rather than forcing the users to store their assets in a certain way.
Title: Re: Image file format and RAM consumption
Post by: Juraj on 2018-11-14, 15:33:41
Tangentially connected, but I had all sorts of issues with 3dsMax bitmap loader (not Corona connected!) with grey-scale image files. One is it can't load 32bit grey-scale files if they have attached color profile (found out by downloading 32bit float tiff files from TexturesXYZ and had to manually convert them all to .exrs without color profiles).
Regular integer 8&16bit files also gave me odd results here and there.

I need to test this again esp. since I use more and more 8k texture sets.
Title: Re: Image file format and RAM consumption
Post by: karnak on 2018-11-14, 15:55:06
Interesting findings Juraj, thanks for sharing.

During a project early this year, I also found out something.
I was testing my project for performance and found out the following:

Code: [Select]
    CoronaBitmap    8bit image   38 sec
    CoronaBitmap   16bit image   46 sec <<<

(Default) Bitmap    8bit image   41 sec
(Default) Bitmap   16bit image   41 sec

This measure was taken (average of multiple measurements) by looking at the UHD cache precom time, the loaded images were in TIF format but I can't find in my notes how many channels. Corona was version 1.7.4.
Title: Re: Image file format and RAM consumption
Post by: Juraj on 2018-11-14, 16:09:52
I am not even using Corona Bitmap, unless it's for 360 HDRi. I don't see any point of that map, it doesn't carry any improvement. I hoped they would introduce better resampling, something that would yield superior sharp results at glance angles. But since it has no performance (not care much) or visual (this is what I would love) benefits, I don't see why to use it at all.
Title: Re: Image file format and RAM consumption
Post by: romullus on 2018-11-14, 21:05:52
..

Thanks Ryuu, your answers helped me great deal! I'm working on a simple scene, but Corona was throwing low memory warnings even at lowest resolution renderings. I did few optimizations to some unnecessary heavy assets and now i can render at 4K without problems. That is quite an achievement for my 16 GB RAM! :]
Title: Re: Image file format and RAM consumption
Post by: pokoy on 2018-11-15, 07:23:24
I'm pretty sure a greyscale tif will convert to RGB for rendering, not sure if you can save any memory here.
Title: Re: Image file format and RAM consumption
Post by: romullus on 2018-11-15, 10:39:42
But did you do any tests to confirm your claims? I took 5 8K RGB textures, which should take ~1006 MB RAM in rendering, converted them to greyscale (~335 MB), assigned both sets to 5 boxes and rendered both scenes. Scene with RGB textures took 2092 MB RAM and scene with greyscale - 1455 MB. Theoretical difference should be 671 MB, in practice it was 637 MB. I think results speaks for themselves, isn't? :]
Title: Re: Image file format and RAM consumption
Post by: Juraj on 2018-11-15, 11:17:40
Just tested 32bit files again. 3dsMax has issue with reading grey-scale 32bit Tiffs, and in order to save them into .exr or .hdr, they need to be RGB. What is the solution to this ?

Instead of painfully re-saving something to grey-scale manually, is there something like in Unreal where I can read individual channels from bitmaps ?
If not, maybe Corona Bitmap could add such option.
Title: Re: Image file format and RAM consumption
Post by: pokoy on 2018-11-15, 14:31:18
But did you do any tests to confirm your claims? I took 5 8K RGB textures, which should take ~1006 MB RAM in rendering, converted them to greyscale (~335 MB), assigned both sets to 5 boxes and rendered both scenes. Scene with RGB textures took 2092 MB RAM and scene with greyscale - 1455 MB. Theoretical difference should be 671 MB, in practice it was 637 MB. I think results speaks for themselves, isn't? :]

No, I didn't but I remember asking the devs here and this was the answer, it also made sense since everything is done in RGB internally. Happy to learn it does make a difference.
Title: Re: Image file format and RAM consumption
Post by: romullus on 2018-11-15, 15:03:32
Instead of painfully re-saving something to grey-scale manually, is there something like in Unreal where I can read individual channels from bitmaps ?
If not, maybe Corona Bitmap could add such option.

Not sure if this is what you're after, but i'm using RGBTint map for this. Works fine.

No, I didn't but I remember asking the devs here and this was the answer, it also made sense since everything is done in RGB internally. Happy to learn it does make a difference.

Hm, maybe there was some miscommunication, or things indeed changed since then? Anyway, greyscale does make big difference and i couldn't be more happy about that. Every single megabyte counts :]
Title: Re: Image file format and RAM consumption
Post by: mraw on 2018-11-15, 21:50:57
I'm awfully sorry that I can't be more specific. But years(15+) ago I went the path of using greyscale-single-channel images to limit texture size in ram, but I ended up in all kind of weird, hard to track down errors. In Max and if I remember correctly in Maya as well. It's like there are kinds of scenarios developers don't expect the user to plug a single-channel-image in. I wouldn't use that technique in complex projects.
Title: Re: Image file format and RAM consumption
Post by: Ondra on 2018-11-15, 22:44:23
grayscale loading: I think it is simply the "road less travelled" - less people actually use those code branches, so errors are not discovered and fixed. When we switched to OpenImageIO, we found several bugs in various image implementations, but none were in 8bit jpegs - it was 16bit tgas alpha channel, monochromatic tiff gamma, etc.
Title: Re: Image file format and RAM consumption
Post by: Njen on 2018-11-17, 01:26:56
Just tested 32bit files again. 3dsMax has issue with reading grey-scale 32bit Tiffs, and in order to save them into .exr or .hdr, they need to be RGB. What is the solution to this ?
It is possible to save single channel exr's in Nuke.
Instead of painfully re-saving something to grey-scale manually, is there something like in Unreal where I can read individual channels from bitmaps ?
If not, maybe Corona Bitmap could add such option.
As romullus mentioned, RGBTint is a computationally lightweight method to split out R, G and B channels from an image. If you need the red channel, you make the red swatch white, and the green and blue swatches black.
Title: Re: Image file format and RAM consumption
Post by: nino anurogo on 2023-09-25, 03:47:05
..

Thanks Ryuu, your answers helped me great deal! I'm working on a simple scene, but Corona was throwing low memory warnings even at lowest resolution renderings. I did few optimizations to some unnecessary heavy assets and now i can render at 4K without problems. That is quite an achievement for my 16 GB RAM! :]

Hi Romullus, can you please describe steps to do the few optimizations you mention here ? Thank you
Title: Re: Image file format and RAM consumption
Post by: romullus on 2023-09-25, 10:41:02
Obviously i can't remember what exactly i did 5 years ago, but i think i went through my textures and made sure that all the textures that don't need extended bit depth, would be saved in 8 bits per channel and all the greyscale textures would be saved as single channel images. If that still doesn't solve low RAM issue, then the other most effective way to lower RAM consumption is to reduce texture resolution, just be aware that this potentially can come at the cost of lower final image quality. On the other hand, with modern Corona you may not need any of this, because it evolved since and introduced some memory saving features, like out of core textures.
Title: Re: Image file format and RAM consumption
Post by: nino anurogo on 2023-09-26, 07:26:03
Obviously i can't remember what exactly i did 5 years ago, but i think i went through my textures and made sure that all the textures that don't need extended bit depth, would be saved in 8 bits per channel and all the greyscale textures would be saved as single channel images. If that still doesn't solve low RAM issue, then the other most effective way to lower RAM consumption is to reduce texture resolution, just be aware that this potentially can come at the cost of lower final image quality. On the other hand, with modern Corona you may not need any of this, because it evolved since and introduced some memory saving features, like out of core textures.

Thank you for your reply, actually I'm still just studying Corona Renderer through various forums and haven't purchased it yet
Title: Re: Image file format and RAM consumption
Post by: romullus on 2023-09-26, 10:31:34
You can always download Corona and activate 30 days trial. That should make studying process much easier as well as help to make decision about purchasing :]