Gaussian Splatting is gaining significant attention for its efficiency and versatility among emerging 3D scanning technologies. But what is it exactly? In a nutshell, this revolutionary approach uses mathematical formulas to represent points in a three-dimensional space.
While both name and description may sound complex, the concept is straightforward when broken down. In this article we’ll explore what Gaussian Splatting is and how it works, as well as its applications and limitations. If you’re in the 3D scanning world, this knowledge is essential in staying ahead of the curve.
Gaussian Splatting is a relatively new approach to creating 3D representations of objects. Traditional 3D scanning techniques typically rely on collecting data points (i.e. point clouds) and processing them into polygon meshes with detailed geometric information. These meshes then get textured to create a photo-realistic 3D model. Gaussian Splatting, on the other hand, represents objects not as meshes, but as overlapping Gaussian functions. A Gaussian function is a mathematical expression that describes a bell-shaped curve, which represents color and texture data in a smooth, continuous manner.
Imagine a pointillism painting, which consists of a multitude of colored dots that, when sufficiently dense, create the illusion of an object. The crucial difference to a pointillist painting is that the individual color dots are not arranged on a two-dimensional surface, but in a three-dimensional space.
To understand Gaussian Splatting, we have to break it down into simple steps:
Like other 3D scanning methods, Gaussian Splatting starts with capturing data about an object. As with photogrammetry, multiple photos are taken from multiple perspectives. These images are analyzed to determine the geometry, color and texture of the object.
Instead of storing the object as points or triangles, the visual data is converted into Gaussian “splats”. Think of splats like a soft, blurry “blob” of color and texture, defined by a Gaussian function. (Remember the comparison to pointillism.) These blobs are layered on top of each other and organized based on depth information from the data. Unlike other 3D scanning techniques, no mesh is created from the data. This skips an entire step in the model reconstruction process.
The overlapping blobs fill in the details of the object and blend to create a final natural, cohesive appearance without sharp edges or gaps sometimes found in traditional scanning methods. Due to the simplicity of the rendering process, scenes and objects can be rendered in real-time, creating almost instant results.
Gaussian Splatting offers some unique advantages in the scanning world:
One of the key advantages of Gaussian splats is their adaptability. As each splat is defined by a mathematical function, they can dynamically change when the perspective of the 3D model or scene is changed. For example, if you zoom in closely, the splats can adjust to reveal finer details, maintaining clarity and sharpness. Conversely, if you zoom out, the splats simplify, reducing unnecessary processing. Another huge advantage of Gaussian Splatting is the ability to scan and represent small details and structures such as hair, a feat previously impossible with any 3D scanning techniques.
To prove this, here is hair scanned with our 3D Full-Body Scanner NEO. The first image is a model processed through standard photogrammetry software. The second is a model processed using Gaussian Splatting, through PostShot software:
Since this method uses mathematical “blobs” to create the scanned object or scene, a geometric mesh doesn’t have to be processed and created. An entire step in the usual 3D scanning and model creation process is then skipped, making it faster and more efficient, especially for large or complex objects.
While still a developing technology, Gaussian Splatting has exciting potential in various fields, namely computer graphics and virtual and augmented reality (VR/AR). Its ability to render complex dynamic models in real-time makes it indispensable for creating realistic, natural-looking scenes for virtual environments and characters, especially when rendering speed is crucial. The technology can also be integrated into already-existing photogrammetry setups, as done with our NEO, to speed up processing time and offer greater opportunities for innovation.
Like any technology, Gaussian Splatting is not without its challenges and limitations. The underlying mathematical functions, although efficient, still require high memory usage and specialized hard- and software. As a newer technique, it is still being integrated into standard 3D workflows, leading to limited tools and resources for widespread use. Additionally, in some cases, traditional scanning methods capture sharp edges and extremely fine details better than Gaussian splats. This technology cannot be used for industrial applications, for example quality control and reverse engineering, or any application that requires a mesh, texture map or precise geometric data of scan objects.
The biggest hurdle for commercial use is that Gaussian Splatting currently only works within special rendering software. These software don’t allow you to export a file in another format to edit the 3D model or use it for any other purposes. This limits its use in areas such as high-end product renderings for marketing or e-commerce purposes. It also means you can’t create models from which you could subsequently produce products (e.g. in 3D printing). You have a beautiful, detailed image, but unfortunately nothing tangible, as the illusion is as concrete as a mirage in the desert.
Gaussian Splatting is a fascinating innovation in 3D scanning that shifts the focus from rigid meshes and structures to smooth and dynamic representations. By using overlapping Gaussian functions, it offers a fresh approach to creating digital models, with benefits like efficiency and adaptability. However, its effectiveness depends on the specific use case and requirements. Where it shines in gaming and VR applications, it falls short in industries requiring precise geometric data and texture maps. As it evolves, Gaussian Splatting may open new doors in different industries, proving to be a valuable addition to the toolkit of 3D scanning technologies. Here at botspot, we’re thrilled to explore Gaussian Splatting and see all the possibilities it brings with it.