Chaos Corona Forum
General Category => General CG Discussion => Topic started by: n.merazzi on 2023-09-25, 11:40:21
-
I'm sure you guys have seen this, but please take a few minutes and type it on Google for anyone who hasn't seen it.
- for Archviz Artists.
for all 3D Artists.
Ondra and Corona team, do you guys think this kind of training can be applied to a 3D rendering engine?
with a single video, you can transform everything in 3D - with incredibly high quality and high-speed frame.
Well... the discussion is open. =)
-
I think it's shocking that this has no replies. I have no interest in turning my 3d scenes into 3DGS, but I have a huge interest in going out and capturing a location and turning it into a 3DGS. This is something fantastic for arch viz, which is Coronas main audience, right?
It's been oddly quiet on the autodesk, v-ray and corona front about this tech. The UE, Blender and Unity scene is all over it. Alteast from my biased internet bubble.
This is what's going to push me into doing an actual project in UE soon.
If any developer sees this, I would be really interested to hear if the ability to read and render 3DGS would be best implemented on the autodesk/Max side of things, or in the actual renderer. So I know who to nag ;)
-
Yeah, was wondering this too, especially since setting this up from actual 3d data might be more accurate, I guess it could help to some degree with slight changes in animations without the need to re-render everything.
FWIW, here's a thread over at Chaos forums asking basically the same question, but don't expect too much:
https://forums.chaos.com/forum/v-ray-for-3ds-max-forums/v-ray-for-3ds-max-wishlist/1193659-3d-gaussian-splatting-in-v-ray (https://forums.chaos.com/forum/v-ray-for-3ds-max-forums/v-ray-for-3ds-max-wishlist/1193659-3d-gaussian-splatting-in-v-ray)
I wouldn't wait for Autodesk, their 'evangelists' are probably trying to come up with a way on how to get rid of expensive 3d dev and somehow utilize AI and be done. Their approach seems to be adding 1-2 modifiers per release cycle, adding stuff no one asked for and fixing bugs introduced while doing so... or not.
-
Absolutely amazing tech. I want to use this for some construction help in family garden as quick sort of visualizing scanned pointcloud of environment :- ) As I've seen Unreal will be able to read/use this
-
Absolutely amazing tech. I want to use this for some construction help in family garden as quick sort of visualizing scanned pointcloud of environment :- ) As I've seen Unreal will be able to read/use this
Here's some inspiration in case you need to put smoke monsters in your family garden:
-
In a browser:
https://www.reddit.com/r/Spline3D/comments/17v8ii1/update_we_are_adding_gaussian_splatting_to_spline/
Live demo: https://my.spline.design/girlstudio-8b6211e0b6ab456c8764297c6ff3ed45/
If a developer, or any one else who's knowledgeable enough, sees this: is combining splats with a pure ray/pathtracer a difficult scenario?
-
When I saw that, the first thing that came in mind was to record the context buildings, so We don't need to spend time on it and we will have the most realistic results, but at some point I start to see we have a lot of gaussian splats over the scene that makes it unuseful. Also, It is working on UE, and perhaps Vantage can import the file as Polycam allow you to export the GS as .ply.
That's one of the reasons that I start to study even more UE. If they could make the materials system more friendly would be amazing. hehe.
Juraj, send your results please =)
-
Hah that will be some time until I get to it :- )
-
Some initial tests here at Chaos were somewhat disappointing, though those are but early tests so we can't speak to what may or may not happen long term. For interest, can you share how you see this being used when it comes to integration with path tracing? Lighting, reflections, etc. would already be baked in if adding them into the scene - so maybe as an environment? Or are you picturing creating Gaussian splats from the 3D scene as a new kind of output vs. a still or animation?
-
For me for example, I don't expect any modification much on the Splatted object, but I would like the regular geometry to take it into account (reflect the Splatted object, occluded light,...)
So yes, basically environment.
-
TomG:
In the near term, my interest is on the input side of things. To get locations, and possibly objects, that are difficult to scan or model with traditional methods. It would replace photogrammetry. And I do a lot of photo matched architectural work, so I'm thinking this would allow me to not only nudge the camera after the fact, but also crop and cull unwanted things in the volume. A 3D photograph, that's what I want! :D
Output seems like not a problem already? Just time consuming to render a bazzilions views and process it, just as you would with photographs from the real world. Maybe its' possible to think of some more effective method. But I think input is more interesting.
Juraj: You sell high quality hdris of locations right? Imagine 3DGS with hdri support *drool*
-
On "replace photogrammetry", thing is photogrammetry aims to remove the effect of lighting so that the scene can be relit in 3D and used in any situation, but splatting is more about capturing the lighting, reflections and baking those in, meaning you'd have to make the scene around it (and all such "splatted objects" would also need the same lighting and reflections, which would seem very unlikely). Is there a way you visualized splatting working differently than that, so that it would replace photogrammetry? Locations maybe, as then it would surround anything you add into it (ie "environments" as mentioned earlier).
-
I am sure some smart people are already thinking how to de-light these to create "PBR" Point-Clouds/Nerfs/Splats :- ).
But even photogrammetry is useful in many way with baked light into it, just not as flexible.
And lighting already lit object while fully incorrect from accuracy point, can still be useful and is often used trick in VFX.
So I wouldn't discount something because it doesn't adhere to common workflow.
-
I am sure some smart people are already thinking how to de-light these to create "PBR" Point-Clouds/Nerfs/Splats :- ).
Exactly, "how many papers down the line?" ;D
TomG:
I understand the difference between 3DGS and traditional photogrammetry that generates a mesh and textures (that one probably de-lights in some way).
With 3DGS I would have some of the same problems that I do with my regular workflow of matching a rendered building into a photograph of a location. I would have to match the lighting on the model to fit the environment in both cases. But there would be more interaction. The model could reflect and possibly be shadowed by the 3DGS surrounding. Preferably also cast shadows on it, even if that might cause non-physical effects of something being shadowed twice by the sun.
Why 3DGS? It seems easier and more lightweight than photogrammetry. Also 3DGS capures view dependant effects in a way that traditional photogrammetry does not. And not to mention handles thin objects such as vegatation and eyelashes (look at the spline demo!). It all seems very nice.
-
Great info ty! We were wanting to see how people would put this to use, I'll pass it along for any future testing we do with this technology.
-
Hey TomG,
Glad to see you here.
My idea is to use Gaussian as an environment for animations and stills. If we could use (not sure if possible) the geometry + textures from 3DGP, that would be great. So, basically, all we need is to take videos on overcast days (not hard in London) and then use them as our environment. This way, we will have the real world inside our software as the background and can add our project in the middle of it. Of course, all 3DGP elements need to receive Corona sun + sky light, and even simple Corona lights, receive and cast shadows, etc. Essentially, replace the context of the project.
I would be more than happy to help.
-
Hi everyone!
Apparently things are moving ahead with Vray, it's a very important subject, because this capture technology is incredible, Meta has just made some great announcements, Varjo too, and even if it will never replace traditional polygonal techniques, it's fantastic for environment management for example! Today, we're obliged to integrate it into After Effect, with its 3D mode, but with Corona's 2D flat rendering .. and lot of animated masks, whereas it would be much simpler just to put it into Corona! We might as well be mixing 3Ds, rather than using 2Ds in the middle! Here's an example of a simplified drone scan for web display, and an example of an integrated model with Corona rendering. I hope this will be compatible with Corona soon, we can already do it with Blender, Houdini, UE5, Unity, .. and soon Vray!!!!
VRAY INTEGRATION
DRONE GAUSSIAN SPLATTING
https://adnfamily.com/studio/rd/ia/gs/dunkerque.html
CORONA & AFTER EFFECT INTEGRATION
https://adnfamily.com/studio/sovi/caldera/maquette.html
I already made a post in [Max] Feature Requests, but I can't answer it.
Already BRAVO for your work on Corona,
Thank you & nice continuation!
Best regards.
_
-
With the work completed on integrated it into V-Ray, we will be considering this for next year in Corona, as in theory all the hard lessons have been learned by the V-Ray team ;) Naturally not possible to do it this year, as our release this year has been mapped out for some time.
-
Wow, this is VERY good news, Tom! Speaking of what's been mapped out, the trello board is a bit empty, and I've been wondering if it's a sign of people being too busy working on stuff, or that there's nothing planned?
Nice examples of your work SA3D46. And thanks for sharing that video, it slipped by me.
-
Trello just got updated on Friday, with the things we have some level of confidence in for the next release :) Others to be added if we hit a level of confidence in them being ready. The update is purely on the next release, longer term plans are not updated (and won't be until after the 12 Update 1 release in November, as that is when we'll have a clearer picture for next year).
-
This is fantastic!!! Great to be able to integrate this into our workflows, game changer for capturing environments!
Lots of new advances every day, here are some useful links for those who want to discover a little more about this new approach!
https://radiancefields.com/
https://linktr.ee/gaussian_splatting
https://gaussiantracer.github.io/
https://www.jawset.com/
https://playcanvas.com/supersplat/editor
https://github.com/mkkellogg/GaussianSplats3D
https://docs.nerf.studio/
https://volinga.ai/
Nice continuation to all!
Thank you!
_
-
Can't wait !! :D
-
Hi!
Any news on the integration of Splats in Corona?
That would be really handy! Today we have to transform the Corona renderings into Splats, mix it with the drone's Splats, and do the render / animation in After Effect! Not so easy!
Here are some examples optimized for the web,
CORONA
https://adnfamily.com/studio/rd/ia/gs/nhood/experience/index.html (https://adnfamily.com/studio/rd/ia/gs/nhood/experience/index.html)
(https://adnfamily.com/studio/rd/ia/gs/nhood/experience/vignette.jpg)
CORONA + DRONE
https://adnfamily.com/studio/nhood/counord/experience/index.html (https://adnfamily.com/studio/nhood/counord/experience/index.html)
(https://adnfamily.com/studio/nhood/counord/experience/vignette.png)
Thanks a lot!
_
-
No news yet as the planning is yet to take place, as several of us were on quite long holiday season time off (e.g. me, as I hadn't taken much time off during the year so caught up at the year end!). We'll update Trello once we have some plans (though as always that reminder - development being what it is, Trello is a plan and not a promise, once we do get around to updating it :) ).
-
Just spotted this yesterday: