Context and Clarity: Radiance Fields
Evident Forensics was named for the belief that modern technologies can quickly and affordably preserve compelling evidence in a clear and easy to understand way. High resolution photos and 3D models, made of point clouds and/or meshes, remain the most widely used methods of preserving crash scene evidence but a new technology has combined the visual clarity of high resolution photos with spatial context of a 3D model.
Introducing Radiance Fields
While most professionals investigating, reconstructing, or litigating motor vehicle crashes are at least somewhat familiar with laser scanners and may have heard the term “point cloud” before, very few are likely to have heard of radiance fields at this point in time.
In August of 2020, while most of the world was preoccupied with the spreading pandemic, a group of researchers from UC Berkley, Google Research, and UC San Diego published a paper titled: “NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis.” While the paper introduced the technology and method, it was still a long way from practical application. It wasn’t until 2022 that a group of researchers from NVIDIA made the technology available to the public in a far more usable way (winning a “Best Invention of 2022” award from Time Magazine). Even then, early users would have to download the computer code and install the software through command prompts. As of the writing of this post, the technology continues to be rapidly developed and bleeding-edge software still requires advanced computer skills to install.
Installing software this way is not for the faint of heart.
What are radiance fields for?
Point Clouds First:
When you call a crash reconstructionist to inspect a vehicle, regardless of who you call, they’re almost guaranteed to start the inspection by laser scanning the vehicle. My favorite answer to “what’s that thing?” is that the scanner is the most expensive tape measure you’ll ever see. It’s a dramatic oversimplification but it neatly describes why we use scanners, because they capture every measurement you could want by modeling the full exterior of the vehicle. You may recall that a laser scanner’s main job is to produce a "point cloud” that accurately represents the geometry of the vehicle (or whatever you scanned) in the form of millions of floating points. While point clouds are excellent for measurements they have a major shortcoming - they virtually disappear when you zoom in to get a better look at a mark on the car.
See for yourself:
This point cloud appears to disappear as you zoom. Good luck examining fine details.
Maybe Meshes?:
We’ve discussed meshes before so I won’t go into detail about them here. In short, a mesh works much like draping a virtual blanket over the point cloud to make the surface solid rather than a cloud of points. Color and texture details from photographs are projected onto the solid surface of the mesh to make a solid model that doesn’t disappear upon zooming.
Meshes can produce remarkable results when the subject is carefully photographed under good lightning conditions but they struggle with big shiny flat surfaces - like cars. With enough time and effort, a beautiful mesh model of a car is doable but not convenient or realistic for most cases.
Textured Mesh Model
The mesh model shown above was made by scanning the vehicle, generating a point cloud, cleaning stray points from the cloud, solving for the positions of the cameras when the photos were taken, creating a solid mesh over the cleaned point cloud, and finally re-projecting the images onto the mesh to add color and texture. The result is a decent looking solid model but, even after all that effort, the fine details appear fuzzy. If you want a quick and easy way to make a mesh of the same car, the outcome would look more like the one below:
This mesh was fast and easy to make but looks horrible. Notice how the shiny body panels look the worst while the less reflective tires and buildings look far better.
Back to Radiance Fields:
Radiance fields are nothing like meshes but they effectively serve the same purpose for today’s discussion. They make a “solid” model from a point cloud that doesn’t disappear when you zoom in. The big difference is that radiance fields are far better at generating photo-realistic three-dimensional models than point clouds or meshes.
Remember how point clouds allow us to capture all the measurements you could want in one model? Radiance fields do something like that too, but for photographs instead of measurements. These models allow us to view a vehicle with the realism of a high-definition photograph from perspectives where no photos were taken.
First, take note of the image below showing the position of the camera for each of the 209 photographs used to document the car. Next, watch the video below and notice how the video shows continuous movement around the car in incredible detail, even for perspectives not captured by the photos.
The pyramids surrounding the car represent the position of the camera when each photo was taken.
This animation shows the final radiance field. Checkout the realism of the reflection in the dashboard clock face.
To the point
The three-dimensional modeling industry has made a lot of promises to allow users to easily “capture reality” and preserve three-dimensional spaces with revolutionary realism. The truth is, that nine times out of ten, I’d rather examine a decent photo of some collision damage than a point cloud. Point clouds are excellent for taking measurements and aligning damages between two vehicles but they’re pretty awful at providing visual clarity for small details - like examining paint transfer on a car’s fender.
Radiance fields are still a new and developing technology but they have already greatly surpassed any other 3D modeling method in photo-realism. The 1946 Ford that stared in today’s post is an immaculate show car that was freshly cleaned and waxed before being scanned and photographed in full daylight. The highly reflective Ford presented some of the most difficult conditions for capturing a three-dimensional model, yet, the radiance fields look incredible. Another key feature of this technology is that the radiance field can be directly applied onto a laser scanned point cloud - ensuring that the radiance field remains verifiably true to the geometry of the car.
We’ll be eagerly following along as this technology continues to develop. In the meantime, keep an eye out for a follow up post showing how radiance fields work on intersections and roadways.