Gaussian Splatting: online and local

Gaussian splatting offers an interesting alternative to photogrammetry for specific use cases, particularly where real-time rendering, photorealistic results, and the ability to capture reflective and transparent surfaces are necessary. There are lots of online options for Gaussian splatting, both paid and unpaid. Kiri Engine seems to be a very complete suite and now has the option to make meshes out of splats, for use in for instance Blender in the paid version.

For all online platforms (paid or unpaid), please be aware of your data and privacy! 

Running Gaussian Splatting locally

If you don't want to be reliant on external systems, you can make Gaussian splats (and photogrammetry) locally with a somewhat beefy computer. There are multiple tools available for this, but here we chose Postshot (Gaussian splatting) and RealityCapture (photogrammetry) for a quick comparison. 

Workflow in Postshot:

• Install Postshot from https://www.jawset.com/
  
• Make a video of the object or space. You can import multiple videos in the software, taking all videos with the same camera will have better results.
• Drag the videos into Postshot
• Render. Postshot mainly runs on GPU. The render below (50 sec video) took about 20 minutes.
• After rendering you can crop the image to exclude all the fuzzy blobs. For this look under Paramters - Edit in the menu on the right
• After rendering you can export to .ply
• To import to different software you will need a plugin

There are plugins for

• After Effects (not tested here)
• Unreal (paid plugin, not tested here)
• Blender (lower resolution, slightly more abstract results). Blender does not produce a Mesh!
• Unity (not tested here)

Below: the same chair model in Postshot and Blender

Compared to local photogrammetry (RealityCapture)

We've used the same source video for a render in RealityCapture. This render took about three minutes and shows one of the problems with photogrammetry: shiny objects become invisible. You can fix this by using a polarizing filter over your lens.

Thoughts on using Gaussian Splatting

As long as Gaussian splatting does not easily convert to meshes, it's use in live 3d engines might be limited. Models can be quite heavy (the chair above is 130MB as a splat, vs 20MB as a .obj) and can't be reduced easily. 

It might be more applicable to pre-rendered applications, where you have can re-edit the camera from the original recording. Change angles, change camera movement, etc.

Various tutorials on Gaussian Splatting

importing .ply gaussian splat in Blender