GAUSSIAN SPLATTING

[Stéphane Agullo]

Hello,

First of all a big thank you to the Chaos team, Corona is just fantastic, great evolutions, stability, and impressive rendering quality!

However, we can't wait to be able to integrate Gaussians Splatting in 3D scenes, it's a great technology for capturing a drone environment for example, here's a capture with 80HD photos!



It's a real shame to have to integrate in AfterEffect in 2D render! The most classic format is PLY, and even with unlit luminosity management, and without shading at first, it would be great!

I hope to be able to use them in the near future!

Thanks & nice continuation!
Best regards.
_

[James Vella]

In the mean time you could try cloudcompare and export to stl or fbx etc. I havent tried it myself but seems fairly straight forward

[Juraj]

Absolutely! I am all for this too :- )

[Stéphane Agullo]

Hello James Vella,

You might as well use direct photogrammetry, but then again, it's not as good in terms of weight/quality!
How many billions of polygons would it take to render a neighborhood like this? Only 1M gaussians!



Can't wait! Thank you!
Best regards.
_

[James Vella]

Yes I use photogrammetry. I'm also interested in gaussian splatting but havent had the time, only so many things I can fit into a day :)

There is always decimation methods you can implement in the process to reduce poly count if necessary, once the visual data is stored you can just project it back over the mesh. But if it runs smooth and is just used for scenery then I dont see why the process needs to be over complicated.

[Stéphane Agullo]

Hello James Vella,

I've been using photogrammetry for a long time, for the Palace of Versailles, INRAP, CMN and other major French organizations, for heritage digitization. It's a functional technology, but it requires a lot of resources, first for capture, then for digitization, then for optimization. And some elements can't necessarily be optimized; they have to be reworked. Vegetation, for example. Don't hesitate to try out gaussian splatting, you'll be surprised by the results. It's quick to set up, with only 80 photos in my example, it's very light, 1M vertex for the whole neighborhood, it's quick to generate, 1 hour of calculation on a 3080ti, and the rendering of vegetation and details is very interesting.  It's very complicated to get this quality so quickly in photogrammetry, which is why the organizations I work for are also very interested in this technology, but it can also be applied to archviz, when Corona will be compatible!

Here's a quick experiment with Corona rendering,
https://adnfamily.com/studio/barety.html

Thank you & have a nice day!
Best regards.
_

[James Vella]

Not sure if theres lots of compression on this? All the floorboards look pixilated/blurry, is that the same on your end? Same when it comes to everything outside of this 2 meter circle, or is there just not enough photos in this test case?

[aaouviz]

Hello James Vella,

I've been using photogrammetry for a long time, for the Palace of Versailles, INRAP, CMN and other major French organizations, for heritage digitization. It's a functional technology, but it requires a lot of resources, first for capture, then for digitization, then for optimization. And some elements can't necessarily be optimized; they have to be reworked. Vegetation, for example. Don't hesitate to try out gaussian splatting, you'll be surprised by the results. It's quick to set up, with only 80 photos in my example, it's very light, 1M vertex for the whole neighborhood, it's quick to generate, 1 hour of calculation on a 3080ti, and the rendering of vegetation and details is very interesting.  It's very complicated to get this quality so quickly in photogrammetry, which is why the organizations I work for are also very interested in this technology, but it can also be applied to archviz, when Corona will be compatible!

Here's a quick experiment with Corona rendering,
https://adnfamily.com/studio/barety.html


Thank you & have a nice day!
Best regards.
_

This looks pretty incredible. This was with 80 renders? Just various random views and angles? What res were the renders?

I also LOVE that opening where it shows the point cloud and morphs into the render - is this bespoke or you found it somewhere? I'd love any links to resources you've used.

I've been struggling with some large-scale webgl based projects and splattering might save my day...

[TomG]

There are investigations and research going on into Gaussian Splatting at Chaos of course. Until that is complete though there would be no work done on integrating rendering of Gaussian Splats into Corona; we'll wait and see how the investigations work out, and then after that we'll consider if and when we'd add this to Corona. Which all adds up to this being a 2025 thing at the earliest, assuming it gets chosen for implementation.

[romullus]

Here's a quick experiment with Corona rendering,
https://adnfamily.com/studio/barety.html

I personally couldn't care less about gaussian splatting support in Corona, but its usage for virtual tours, like in your example, looks super cool, especially if it's also possible to have walkthroughs.

[TomG]

Virtual tours would be generating splats from renders, which should be possible with any splat generator out there I would imagine :) Can be done today via that approach I would guess, but have never tried it. The Corona side would only come in for rendering splats inside Corona (use case would almost exclusively be for surrounding environments, from what I can see).

[Stéphane Agullo]

Hello everyone,

Thanks for your feedback! Yes, two aspects, integrate splatts in Corona, it could be useful for hairy objects for example, but also vegetation, even if the big advantage is on the management of very large environments, for full 3D backgrounds. The neighborhood is 1.2 GB in size, for example, but that's light compared with the size and quality of the model, and it's generated from only 80 HD drone photos. My web experiment is deliberately very light, less than 15 MB, but that's to show the power of this display technology, but also its limits. Even so, it's more than enough for quick Internet viewing, and of course, you can capture an entire space, with 3D renderings or photos, but you'll need to develop the navigation system accordingly. For capturing, it's very close to photogrammetry, with the advantages & disadvantages of an AI.

Knowing that my priority is to be able to integrate splatts into Corona, I'm delighted to hear that Chaos is studying the possibility, which could really simplify our workflow when it comes to integrating realistic & lightweight very large environments!

Another quick experiment with Corona rendering,
https://adnfamily.com/studio/tanins.html

Thanks & nice continuation,
Best regards.
_

[maru]

What a time to be alive!

https://poly.cam/explore?feed=trending

https://lumalabs.ai/interactive-scenes

[Stéphane Agullo]

Best to process in HD on your big GPU!
Waiting Corona compatibility!
Great technologies!
🙂

[zaar]

I've posted about Gaussians splats before, and I was surprised by how little response there was. Maybe it's just that the forum is not as active as I remebered. Or maybe there's only for a few of us that this would revolutionize our workflow. Nice to see that SA3D46 sees the same potential that I do. For now it would be too much of a work around to combine renders with splats in AE. I also want them in reflections and refractions! Possibly even occluding lights.

Where's the entusiasm!? Like holy shit this is so cool! I would like to see a little more shooting from them hip from Corona/Chaos. Don't be afraid to add something quickly and mark it as a experimental feature. Of course you need research to figure out how to do this. Maybe it's just the wording Tom, that chaos would first be taking time for research, then consider adding or not, and then start working on adding it? That's the order of things, but it's like how I talk to my kids when I don't really want to do something. "Yes, I'll play house, but only after I've made dinner and washed the dishes"