from Hacker News

Gsplat: CUDA accelerated rasterization of gaussians with Python bindings

by tylerneylon on 10/14/23, 4:43 AM with 33 comments

  • by jimmySixDOF on 10/14/23, 10:41 AM

    There is an interesting problem being worked on with Gaussian Splats for use in VR stereo rendering [1]

    >Gaussian splats are 3d, but currently rendered as 2d cards ordered on centre, not per pixel like they should be.

    >So as centres reorder, they "shuffle". This is a showstopper for general adoption in VR

    >The solution? Ordering per pixel will solve this, as would stochastic methods which write depth correctly (though those have other problems for VR).

    >For it to be viable you need to cull much more aggressively but also can't have any warp divergence when processing samples.

    >Unforunately those things are in direct conflict. Paper's renderer is at extreme end of "weak cull, perfect warp convergence".

    Apparently need some new Order-Independent Transparency algorithms that handle depth layers.

    [1] https://x.com/charshenton/status/1710207169407447396?s=20

  • by speps on 10/14/23, 9:04 AM

    Aras has also done some very interesting blog posts regarding this technique:

    * https://aras-p.info/blog/2023/09/05/Gaussian-Splatting-is-pr...

    * https://aras-p.info/blog/2023/09/13/Making-Gaussian-Splats-s...

    * https://aras-p.info/blog/2023/09/27/Making-Gaussian-Splats-m...

    Gsplat gets a mention:

        Aside: the excellent gsplat.tech by Jakub Červený (@jakub_c5y) seems to also be using some sort of VQ/Clustering for the data. Seriously, check it out, it’s probably be nicest gaussian splatting thing right now w.r.t. usability – very intuitive camera controls, nicely presented file sizes, and works on WebGL2. Craftsmanship!

  • by sprash on 10/14/23, 7:56 AM

    If you want to see some impressive gaussian splat demos take a look here: https://gsplat.tech

  • by polskibus on 10/14/23, 7:56 AM

    What is the use case for this library, in layman terms? I pass it a blurry pic and it „enhances it” ?

  • by qwertox on 10/14/23, 2:10 PM

    This week I saw a YouTube video which left me impressed regarding Gaussian Splatting.

    https://www.youtube.com/watch?v=kShNYOuDnlI&t=224s

    I got it recommended because I watched the following video:

    https://www.youtube.com/watch?v=HVv_IQKlafQ&t=64s

    Both are worth watching if one didn't know it exists.

  • by Yenrabbit on 10/14/23, 2:23 PM

    One reason this implementation is exciting is that the code that accompanies the Gaussian Splatting paper requires companies wanting to use the tech commercially to pay for a commercial license. There are existing viewer implementations, but this seems like the first alternative that would let me do everything without depending on the Inria code. The code being nice and clean is a bonus too :)

  • by gravypod on 10/14/23, 3:49 PM

    Is there any method for animating scenes captured using this method? It would be really interesting to build a hyper realistic world in a game with moving foliage or being able to "use" an item and watch it turn. Would be great for Myst like games.