Render to Texture issue (Normal Map Raytacing issue?)

[Cobra8472]

Hi Everyone,

Our texturing process is built upon baking in lighting information and data into a flat diffuse.
I'm running into an issue, where normal maps when doing a diffuse render to texture are providing incorrect surface information.

It may be easiest for me to illustrate via some simple imagery.

Here is a simple viewport render with a simple reflective material and HDRI background:
www.leatherneck-sim.com/RTT_Bug_01.jpg

Here is the exact same setup, rendered to texture. Note the seams on the front.
www.leatherneck-sim.com/RTT_Bug_02.jpg

If I unify the normals on the part in question, I get the same issue in the viewport render. See image below.
www.leatherneck-sim.com/Issue_01.jpg

Here is a more illustrative view of the issue in the RTT window:
www.leatherneck-sim.com/issue_02.jpg

This leads me to believe it may be something smoothing group/vertex normal/face normal related?
Perhaps there is a difference in how the surface normal is used in RTT? It happens on almost every surface, it's just most obvious on the front corners.

This happens with: Corona 1.4, Vray 3.2, MentalRay in 3Ds Max 2017.
Here is a image for reference of the complete asset: www.leatherneck-sim.com/HUD.jpg

Many thanks in advance for any help,

Nick

More info on the issue and scene to reproduce are in #24 post
romullus

[romullus]

Corona doesn't support projection mapping RTT, so you have to bake your normal maps elsewhere. You can read everything you need to know about normal maps here: http://wiki.polycount.com/wiki/Normal_map an here: http://wiki.polycount.com/wiki/Normal_Map_Modeling

[Cobra8472]

Hi Romullus,

Thanks for the reply-- but this is not about baking normal maps. The normal maps is done, baked by another tool and applied to the object.
The issue, in short: The normal map in render to texture is rendered incorrectly on the object. It works fine in the viewport.

[romullus]

Can you tell how normal maps on your model looks when doing regular rendering, not RTT? Maybe you have somerender time modifiers that affects smoothing on your object?

[lacilaci]

This issue looks like exactly what happens when you don't use cage for normal projection or average normals function(substance painter) the curvature of that rounded edge is being projected on the planes from 90 degree angle instead averaging it towards the edge.

Are you sure your normals aren't showing those seams elsewhere, only after RTT?

[maru]

Since the issue is visible near seams, could it be something with padding? Have you tried rendering with different padding settings? ( RTT > Objects to Bake > Selected Object Settings > Padding - I would check with 0 and values higher than the default 2)

This happens with: Corona 1.4, Vray 3.2, MentalRay in 3Ds Max 2017.

So it means that this is not something Corona-specific, right?

[Ondra]

padding should never influence anything inside the geometry, only fix the filtering seams, My guess would be some problem with normals

[Cobra8472]

Sorry to bump such an old topic; but this is still an issue-- I was just completely swamped to follow up on this topic.

It will happen to pretty much any object with baked down normalmaps (i.e. HP -> LP normalmaps)

I have tried almost every combination of seam setup in order to figure out the issue; but it simply seems to be a limitation in the way RTT handles surface normals?
Any advice would be greatly appreciated!

[romullus]

AFAIK RTT in 3ds max doesn't have any issues with baking normal maps. It may be not the most convenient baking tool to use, but it does its job well. Of course there are some rules how to properly bake normal maps and those rules should be respected in every baker. If you're not familiar with them, i strongly recomend to stop any baking attempts untill you'll learn proper techniques. I'd suggest you to start from here: http://polycount.com/discussion/107196/youre-making-me-hard-making-sense-of-hard-edges-uvs-normal-maps-and-vertex-counts and continue your research further, either on polycount or anywhere else on the web.

[Cobra8472]

Thanks for the reply! but again;  this is not about normal map baking techniques. My normal maps are perfectly correct, and render correct in custom engines, UE4, Marmoset, Unity, et al- I'm extremely well versed in normal map baking, and I've done bakes in everything from zbrush to substance and the new marmoset 3.

This seems to be an issue with something like the camera projection matrix or surface normal directions when an object is flattened by the renderer to RTT.
It's the best way I can explain it. Oddly enough; the issue is non-Corona specific (meaning several renderers have the same issue), alas I hope that there is a workaround.

Take a normal mapped (from HP, smooth edges) Low Poly cube: put a reflective material on it, do a RTT of the object, and the resulting object will have seams (whereas rendering the object in perspective or camera view, it appears perfectly normal)

[Cobra8472]

Ah, crap, I just realized I had broken the image links.

I will make a new (much simpler) example ASAP.

[maru]

Hi, just reviving this.
Cobra8472, can you provide some super simple example images demonstrating this, with a step-by-step guide?
I know that there is already a description here, but images always help greatly.

OTOH if this is happening in other engines too, my first bet would be that it's a Max issue...

[Cobra8472]

Hi Maru,

Sorry for the delay-- development crunch ramped up and I worked around this by using the Max scanline renderer.
I am now running into this issue again.

To be very clear, here is a sketchfab of the mesh, illustrating that the normals appear perfectly fine in a realtime renderer. They also appear perfectly fine in a normal render in Corona.
https://skfb.ly/6sNET

Seams however appear on mesh seams when doing a Render-To-Texture, even when it otherwise appears perfectly in the Corona renderer.
See this image: http://i.imgur.com/4a7A2Y9.png

I've uploaded a package with this model, normalmap and Max 2017 scene here: media.heatblur.se/RTT_Issue.rar


Appreciate any help!

All the best,
Nick

[romullus]

I still don't get it. If you already have normally working normal map (judging by sketchfab viewer), then why you're trying to rebake it with Corona RTT? I can't look at your scene, since i'm on max2016.

[Cobra8472]

I'm not trying to bake a normal map.

I'm baking a diffuse (Beauty) map. The Normal map does not render correctly  when doing so, even when it is fine in a normal camera projection.