letsenhance.io - CSI-style image upscaling

[Eddoron]

Good find! it's really cool.

The missing features are being "hallucinated".
This thing should be trained for normal maps etc. and we don't have to bake large things anymore.
Not perfect but worked quite well in some areas of my test image.

[Noah45]

Often the SU models, I receive, have poor mapping images. However, it is the 'look' they want. Attached is a typical map- notice how well this magic process worked.

Daily Retail workflow.

[sebastian___]

It works especially well with high quality content. Or with down-resized photos.
Works less well with noisy, grainy, low detail pics.

Original = the original resolution image
then I took this and resized it down 400% (so I can have a well defined low grain image) and I up-scaled it back up 400% with the website and with photoshop.
The website algorithm seems to be able to "make up" or invent new details, including grain and surface detail so it closely match the original image. And it's better than the new "preserve detail 2.0" photoshop algorithm. And much better compared to some sharp bicubic or lanc algorithm.

Some parts of the up-scaled image look almost the same, or even better than the original, even though it's missing like 75% or something of original image content.

[maru]

Found another one if you're interested:
https://topazlabs.com/downloads#gigapixel
It offers 30-day trial with no image limit (?). From my initial tests it seems it is slightly worse than letsenhance, but offers more options (e.g. 600% upscaling).

[sebastian___]

It would be interesting if you could apply this to an image sequence (a movie).
I sent a message to the letsenhance people about it but I got no reply. It would be expensive anyway. But probably programs like these who invent new details for resizing would probably not have temporal coherence, I'm guessing it would flicker from frame to frame.

[TomG]

would probably not have temporal coherence, I'm guessing it would flicker from frame to frame.

But could still be trippy fun! ;)

[Juraj]

It would be interesting if you could apply this to an image sequence (a movie).
I sent a message to the letsenhance people about it but I got no reply. It would be expensive anyway. But probably programs like these who invent new details for resizing would probably not have temporal coherence, I'm guessing it would flicker from frame to frame.

This is possible and some other services even specialize it in. Most of them are in-house and the only public one is not available for even beta test yet, it's part of Artomatix suite. ( I am beta tester but currently the focus is for texture synthetization ).
But they did showcase upscaling to 4k using this tech, it was temporally well stabilized and you couldn't tell. But the detail enhanced wasn't that great compared to others I have seen, not sure why.

[sebastian___]

They are keeping this in-house or closed for the public or web based only, because they think it's something revolutionary and people will pay a lot of cash for each picture resized. Well it is something revolutionary, but I think soon the technology will leak to the public in forms of After Effects plugins and so on, like it always does eventually.

 "AI" resizing for everybody ! :)

Perhaps with only 200% resize, the artefacts will be almost invisible.
If anyone makes a video test up-size with this, or the topazlabs thing, please post here.

[maru]

By the way, does some kind of AI upscaling in the Corona VFB sound interesting to you?
The idea would be: render something, press a button, wait, you get double the output size.
I guess it would be interesting for prints and fast previews that turn out good enough.

It's not something the devs are working on, just a wild fantasy, but I am curious how many people would be interested in this. :)

[PROH]

I think this could be a great feature to have in the CIE! So + 1 for that :)

[Juraj]

By the way, does some kind of AI upscaling in the Corona VFB sound interesting to you?
The idea would be: render something, press a button, wait, you get double the output size.
I guess it would be interesting for prints and fast previews that turn out good enough.

It's not something the devs are working on, just a wild fantasy, but I am curious how many people would be interested in this. :)

In my opinion not in this way, but I could see it as integrated feature of interactive. Basically nVidia is doing this with their latest DLSS which is sort of up-scaling/AA technique.
It would be another trick to massively boost interactive as very small visual detriment, if at all.

The tech is super-fast, akin to Optix, so responsive enough for interactive.

[maru]

In my opinion not in this way, but I could see it as integrated feature of interactive. Basically nVidia is doing this with their latest DLSS which is sort of up-scaling/AA technique.
It would be another trick to massively boost interactive as very small visual detriment, if at all.

The tech is super-fast, akin to Optix, so responsive enough for interactive.

So basically you are IRing a lower-res image, which is then upscaled in almost real time?

[Juraj]

In my opinion not in this way, but I could see it as integrated feature of interactive. Basically nVidia is doing this with their latest DLSS which is sort of up-scaling/AA technique.
It would be another trick to massively boost interactive as very small visual detriment, if at all.

The tech is super-fast, akin to Optix, so responsive enough for interactive.

So basically you are IRing a lower-res image, which is then upscaled in almost real time?

Yeah :- ). In fact I could imagine this being the best order of operation:

1) Sample at half the resolution Resolution for best quality, or 1/4 for best speed.
2) Up-sample in real-time
3) Denoise the upsampled result.

Not that is to be seen if this gives better quality than sampling at true res and then denoising, but imho it could give a lot more information to the denoiser.
This is also how nVidia advises for it to be used.

[sebastian___]

If the upscale would be even 70% as good as letsenhance.... And then rendering and tweaking in IR with maybe a very small 480 x 360 pixels would be superfast, and then to have that upscaled on the fly with the GPU to maybe 960 x 720 or more with minimal loss ? That sounds awesome.

[maru]

So Photoshop now has a new AI upscaling option. I gave it a quick test on jpgs from https://corona-renderer.com/gallery

1) Original image by https://createdby.ma/
https://corona-renderer.com/comparer/uwGeCw

2) Original image by Nikhil Saini
https://corona-renderer.com/comparer/SP0E9w

3) Original image by Andrea Bertaccini
https://corona-renderer.com/comparer/gasfdF

The workflow was open the image in Camera RAW and apply the new "enhance" option VS open the image directly in Photoshop and enlarge using the "preserve details" option.
It doesn't seem to be a huge improvement on these examples. Some edges are clearly sharper, and both enlargement methods make jpg artifacts more noticeable. :)
I guess it works best on photos in RAW format.

You can read more about it here: https://petapixel.com/2021/03/13/adobe-photoshops-super-resolution-made-my-jaw-hit-the-floor/