Exploring Virtual Reality as a Forensic Tool
Written by Eduardo Neeter   

VIRTUAL REALITY (VR) OFFERS unparalleled capabilities to support and facilitate forensic activities. While VR and other related technologies—like augmented reality (AR) and mixed reality (MR)—have been around for more than half a century, it is only in the last few years that the technology has shown the potential to go mainstream. VR is especially well suited for supporting use-cases where spatial information is critical, such as accidents and crime scene reconstruction. When used appropriately, this emerging technology will be able to easily leapfrog over current alternatives.

There are unique elements of VR that are like nothing technology has been able to deliver before. VR is a unique technology developed completely around humans and human senses. It is designed to trick those senses and immerse the user in a computer-generated world. Advanced implementations of VR make use of specific components that direct customized information to each of our senses (such as sight, hearing, and touch) in a way that the user then perceives and interacts with. This creates the sensation of actually being there, inside the virtual environment. This is referred to as the “sense of presence.” The sense of presence is well defined as “the subjective experience of being in one place or environment even when physically situated in another” (Witmer, 1998). In addition, the VR environment can be shared over the internet and users can interact remotely, therefore the term “presence” in the VR context is also referred to as “telepresence”.

A VR environment can produce the sense of presence by leveraging immersion capabilities. Immersion is related to presence, social presence, and self-presence in a simulated environment. Immersion occurs when the virtual environment perceptually envelopes an individual such that the individual perceives him or herself to be interacting within that virtual environment rather than within his or her physical surroundings (Blascovich et al, 2002).

A well-created VR environment will reproduce images and sounds of a scene from the point of view of the user, and deliver in real-time these images and sounds adjusted to each user’s eyes and ears, based on the user’s position and orientation. It will also render the digital images from the user’s perspective with enough precision and sufficient frequency that it causes the user’s brain to reconstruct a 3D model of the scene and place the user at the center of such a model.

Long story short, the VR environment makes the user “feel” that he or she is at the scene.

Current Needs in Forensics

The administration of justice and the overall judicial process requires significant human intervention and preparatory work. In general, this is a time-consuming and expensive process. Processing a crime scene involves purposeful recording and documentation of the conditions at the scene, and the collection of all physical evidence that could provide any clue and help determine what happened. The most logical explanations that investigators can recreate, in many cases, can still be very confusing for most people who didn’t have the opportunity to be present at the scene.

When considering the needs for the forensic domain, a good source of insight is the National Institute of Standards and Technology (NIST)—in particular, the 2017 NIST/OSAC Research Needs Assessment titled “Use of Technology for Crime Scene Documentation”. The report identifies the need for “modernizing current practices” and “improving workflow.” An overarching goal is to “reduce human error and expedite data transfer.” The report also highlights the need for improving “general interfacing with computer systems to simplify back-end work and data distribution.”

The current “pain points" relate to the costs and difficulties associated with replicating a scene as an investigative and demonstrative working environment. Another area where improvement offers significant value is the collaboration and communication between investigators and prosecutors.

Why Should We Use VR as a Forensic Tool?

To improve justice and support a fair judicial process, society needs the most powerful and cost-effective tools that investigators can use across multiple types of scenes.

There is value in reducing the associated analysis and processing costs, while improving the quality and access to findings and results. VR capabilities are uniquely suited to help in the replication, visualization, annotation, collaboration, and communication process. The premise is that the immersion capability of VR can be leveraged to provide a new working environment for processing crime and accident scenes.

A quote from a seasoned detective summarizes it very well: “The person in charge of the case sometimes doesn’t go to the crime scene for days or weeks, and in some cases doesn’t go at all. With this technology, they could walk into the scene right away, whenever they want.”

Why Now?

We are finally starting to see commercially affordable VR with a performance level suitable for mainstream adoption. The time is right to leverage these capabilities and provide the best tool possible for investigators and others involved in criminal justice—including prosecutors, defenders, judges, and, eventually, jurors. It is time to allow all of them to “teleport” to a virtual crime or accident scene as needed.

Today, many of these agencies already capture the scenes with highly sophisticated 3D laser scanners, and they not only have this 3D raw data readily available, but they are required to translate, reproduce, and communicate such abundant and relevant 3D/spatial information into a 2D metaphor (such as diagrams, blueprints, pictures, and videos). In today’s world, users are still confined to 2D interfaces and are forced to communicate and collaborate within these constraints. 2D user interfaces are suboptimal platforms for visualizing, interacting, navigating, and manipulating this kind of 3D/spatial data.

In addition, significant amounts of time and productivity are lost by people involved in the investigative and prosecutorial activities due to the limitations of available tools and current working environments when they need to process and share information.

Moreover, unlike a crime scene, which is often dismantled shortly after the crime has taken place and sometimes before defense counsel can engage in its own investigation, VR systems could recreate the most realistic and comprehensive record, providing the best means to permanently preserve the scene for all parties to a case.

Early conversations and demonstrations with recognized experts in the use of forensic 3D metrology in the private and public sector—including detectives, crime scene technicians, and forensic professionals—have provided very positive insight. One expert user said: “There is a distinct need for an intuitive method for the visualization of these rich, 3D evidence data sets. VR provides this method and does it with little additional effort.”

The Benefits that VR Can Offer

Virtual reality can improve investigation capabilities by integrating data from multiple sensors used to capture and document a crime or accident scene (such as 3D laser scanners) and can enhance investigative and communication capabilities by accurately replicating the scene in a shared immersive virtual collaborative environment. Users would be able to collaborate, communicate, and share knowledge within and about the scene. These capabilities can be obtained by leveraging commercially available hardware.

VR can virtually reproduce multiple instances of a given scene and allow users to perform the activities that are required for the proper processing—for example, navigating the scene, taking measurements, annotating the scene, or presenting evidence—all with a level of user immersion and realism that no other technology can provided.

Investigators with different specializations, regardless of their geographical location, would be able to collaborate synchronously (or asynchronously) within the VR environment. They could invite the prosecutors to join in for a walkthrough, and they could discuss and test potential alternatives while they are all immersed in a virtual crime scene. The technology would be extended to public and private defenders, and eventually to judges and jurors.

This technology could offer investigators, prosecutors, defenders, and other stakeholders multiple key benefits:

• Revisit the crime scene: Users would be able to quickly and cost-effectively "teleport" to the crime or accident scene, walk through the scene as many times as needed, assess as many points of view as needed, and perceive the scene as close as possible to the original incident.

• Collaboration: Users would be presented with the same virtual scene and share the same collaborative space. Participants from multiple specialties and from multiple locations could be given access to a virtual scene as soon as it is digitalized. Users would be able to share annotations and knowledge contributed by individuals.

• Knowledge acquisition: Prosecutors would be given increased capabilities to learn the facts around the scene being investigated, and to become more proficient about investigative techniques. This would increase their capacity to communicate with other stakeholders in a more compelling and cost-efficient manner, reducing the amount of back-and-forth and rework.

• Compliance: VR environments could enable agencies to better observe, control, and report on users as they perform their investigative tasks. The immersive nature of the VR environment facilitates the verification of required procedures. Successful completion of the required investigative tasks could be tracked and reported.

The use of VR as a forensic tool would be a great application of this emerging technology as a productivity tool, as it would allow people do things they couldn’t do before, and at the same time, be able to do it more efficiently and with less friction than ever.

Potential Risks of Using VR for Forensics

The adoption of any new technology always comes with associated risks. There are many aspects that would need to be tested and adjusted before making it practical to be released into the field. Here are some considerations that should be considered:

• Digital Divide: One significant consideration, especially for technology with such potential societal impact, is the topic of the Digital Divide. This relates to the advantage that rich people might have over poor people due to the cost of accessing the underlying technology. An argument can be made that the adoption of VR in the criminal justice domain will actually benefit underprivileged populations. Utilizing VR to improve the investigative process and the way that evidence is presented to a judge or jury will only help exonerate the innocent or aid those who have been victimized by crime. In addition, everyone accused of a crime is guaranteed their right to an attorney through the 6th and 14th Amendments of the U.S. Constitution. All evidence to be presented at trial is “discoverable”. This means all evidence (including VR) must be turned over to the defense for copy and inspection prior to trial. It applies to everyone regardless of status.

• Access to equipment: During the investigative process, the biggest hurdle is access to equipment by opposing counsel who do not have the advanced technology. The courts ensure that both parties have a copy of all relevant evidence or access to any technology that will be presented to a jury before going to trial. Also, parties without the use of a VR platform will always have access to previous methods of collection and presentation for review of evidence, such as video, audio, or photographs.

• Personal Bias: Another potential issue to consider is the risk of personal biases being introduced. One advantage of immersive VR environments is that every user action is captured and digitally processed, and such actions can be recorded for future analysis. The VR system could record all user’s additions, deletions, changes, viewing angles, and the travel path through the scene.

• Transparency: The information contained in VR environments should be preserved in a way that enables thorough auditing and oversight. Any reported comparison error between original data and data presented and processed in the VR environment should be highlighted, and it should enable investigators and others to judge whether or not undue uncertainty exists in a VR scene.

• Human Factors: A critical aspect to consider is how being in a VR crime scene would affect the users. In addition to regular VR ergonomic considerations, there are considerations on the possible effects—both physical and emotional—that may result from immersion in a virtual crime scene. These effects would have to be studied and understood. Protocols would need to be established that create an evidence-based decision matrix to determine those who could safely enter the VR environment and those who should not. Experts in the appropriate fields would need to be consulted on the proper way to avoid inducing additional trauma in users. Sample data sets and other multimedia content could be issued in order to gradually introduce users to the immersive digital environment and quickly exclude the few who experience ill-effects prior to actual immersion.

Expected Changes in Near Future

We can anticipate the continued adoption of 3D laser scanners by law enforcement, due to reduction in acquisition cost, simplification of use and operation of the equipment, as well as its demonstrated efficiency and efficacy to the forensic teams.

As a consequence, there will be a demand for innovative ways for processing and sharing scene data in VR. Companies in the forensic software and hardware industry will move further into VR and provide more advanced features. As examples, Faro has recently launched a VR capability to let users visualize scanned data. Veesus Arena4D also provides VR viewing capabilities.

The most influential change we anticipate in the next few years is increased demand by users. The adoption of consumer VR products will have the similar effect that smartphones and tablets had in organizational environments (such as enterprise, academy, and government). We could expect that users will be demanding the use of VR, given that it will become ubiquitous and pervasive.

As normal citizens get accustomed to VR in their day-to-day lives, entertainment, and education, this mainstream acceptance will accelerate the case for adoption in the criminal justice and crime scene investigation domains. We are looking forward to this future!


About the Author

Eduardo Neeter is the founder and CEO of FactualVR, a startup developing a virtual reality platform to help law enforcement and prosecutors to accurately replicate and communicate the facts around a crime scene. Neeter first began working in the field of virtual reality in a Japan-based research lab 22 years ago. Upon his return to the United States, he worked in software development in several startup companies in Silicon Valley, including a spinoff from SRI (Stanford Research Institute). He worked in VR as graduate research assistant while at Georgia Tech. Eduardo is also the co-chair of the VRARA (VR/AR Association) Criminal Justice Committee.

References

Blascovich, J., J. Loomis, A. Beall, K. Swinth, C. Hoyt, & J.N. Bailenson. “Immersive virtual environment technology as a methodological tool for social psychology,” Psychological Inquiry, 13(2), 103-124 (2002). Retrieved from: https://vhil.stanford.edu/pubs/2002/immersive-virtual-environment-technology-as

National Institute of Standards and Technology (NIST, 2017) - OSAC Research. Retrieved from: https://www.nist.gov/sites/default/files/documents/2017/02/09/csi_sub-_use_of_technology_for_crime_scene_documentation.pdf

Witmer, B. G. & M.J. Singer. “Measuring presence in virtual environments: A presence questionnaire,” Presence, 7(3), 225-240 (1998).


This article appared in the Winter 2018 issue of Evidence Technology Magazine.
Click here to read the full issue.

 
Next >






Item of Interest

The language barrier between English-speaking investigators and Spanish-speaking witnesses is a growing problem. (Updated 28 February 2011)

Read more...