October 2019

Through an informal yet highly engaging presentation, Dr David Murphy explained a variety of concepts around VR in an insightful and stimulating way. Giving a colloquium on the knowledge he has gained, being involved in VR since 1992. Beginning with presenting a reality continuum to explain the variety of reality manipulation and where his area of expertise is. We then explored the software and hardware requirements for VR and the various judgement calls that must be made in VR projects. Going on to understand some of the trends and realistic future possibilities for VR/AR.

Starting by presenting a reality continuum, to show the space that he’s working in. On one end of the continuum there is real reality RR and on the other extreme there is virtual reality VR, in between there is a spectrum where merged reality and augmented reality lay. Essentially these are various combinations of reality with computer generated data being overlaid on top of reality or else reality being brought into a virtual synthetic environment. As you begin to develop VR, there are judgement calls which need to be made for your project. The trade offs between various elements.

Beginning with the early developments of VR software and hardware, Murphy explained how initially the software was written specifically for particular graphics cards, systems or headsets. The style of programming is classed as imperative programming building from the ground up, this idea has informed a lot of modern graphics languages. The manner in which one programmes used to be written meant that they were essentially locked to the devices they were intended for and failed to be transferable or useful beyond their initial purpose.

Alternatively there is a declarative approach that describes the intended outcome which lends itself better to VR developments because when creating VR environments, there’s little concern about the implementation details on how it’s done and achieved as this is offloaded to another piece of software or part of ones system. As a programmer one wants the ability to design their system in terms of features, content and how these interact or behave. Outsourcing the other elements gives you the space to focus on what your personal contribution. This opens the development to non programmers as well as this made it intuitive and simple to interact with, with the uptake of declarative languages it gave artists the ability to develop VR systems.

Declarative approaches have evolved over time, starting with Iris systems where they introduced the concept of a scene graph which has since become a crucial part of VR development. A scene graph is essentially a hierarchical description of the content, entities and objects in your scene, and how they relate to each other. There has been different variations since with Inventor going on to be OpenInventor and open standard which later then became an API known as Coin 3D, which is a high-level, retained-mode toolkit for effective graphics development. Alongside this you have object oriented languages like Java 3D which took the concept of scene graphs and declarative styles of description which is said to be a heavier approach which allows you to create a distributable executable.

These languages all typically come together as there are strengths and weaknesses to all of them so the optimal efficiency is met through the combination of the various elements. Combined approaches using procedural and declarative languages in VR has become the mainstay of modern development. The web is more or less the main medium of delivering VR content. Developers must consider how your intended audience will consume your software or content. Hosting on a server and using the web infrastructure to deliver your content allowing them to participate in the experience through a web browser. All one needs to be able to engage in the material is the necessary hardware devices and then a web browser for viewing various content.

Alternatively there are game engines which offer more complete solutions to VR development. They are typically professional grade software development environments, the same tools which are used to develop AAA title games. They offer sophisticated, polished and high standard graphics which you’re then able to leverage however you have to be mindful that these are predominantly proprietary systems, so you must adopt a single game engine and you build your platform out on that. The systems can be difficult to learn and typically very complex, usually only function by having large teams who specify in a particular area with very defined roles.

From personal experience, Murphy strongly advised against using proprietary systems if you’re considering taking on a substantial project as you’re powerless against the future of the system. Instead, using standards based solutions which are ISO standards meaning they will be supported going forward. Whereas game engines, although they’re sophisticated, they are privately owned so the future may be uncertain. Open source game engines exist but are far behind the competitors in terms of quality and refinement.

The hardware environment for VR is developing rapidly, there’s a wide variety of equipment available from the low end inexpensive pieces to those which are considerably expensive at the high end. Commercial headsets which use mobile devices to render VR are relatively inexpensive and quite widely available. It’s a complex and moving space, trends are arising which are pushing towards wireless systems.

There is a more complicated configuration to a true VR experience, more than just a headset VR involves an entire workstation; controllers, displays, spatial sound, haptic interfaces, invisibles and wearable technology. It’s almost impossible to think of VR development in isolation because the technology is situated in quite a dynamic and advanced environment.

In terms of architecture VR isn’t simply one thing, there’s web, mobile, fishtank, desktop, room scale, wide scale and environmental VR. Which essentially describes the application of the VR experience. Taking into account considerations about how the user will be interacting with the system as it’s essential to have this in mind as you’re developing a VR experience. There isn’t a sufficing way to scale VR from one type of mode to another, room scale VR doesn’t condensate well to a seated desktop experience. VR can be designed to be individualised for a singular person or it can be collaborative which provides a social component and normally highly distributable. These are all considerations that need to be addressed at the outset, not after the fact.

Presence is the measure of how much a user is immersed in the experience, ideally VR will engulf the person entirely into the synthetic environment. How present does the participant feel in the VR space or conversely how detatched is the person from reality. There’s passive VR where users simply consume the content, they tend not to be very engaging and it’s difficult to direct the participants attention from the outset and throughout. Navigational or exploratory VR which gives the freedom and ability for the user to move around, they are given an element of agency and they’re actively engaging with their environment.

Immersive VR is when the participant is completely surrounded by the synthetic space on multi modal levels, visual, auditory, haptics, environmental cues and they then become oblivious to the real world. Their senses are so saturated with information that they don’t have the ability to be present in the external environment, the perceptual system is being dominated by the artificial stimulation inside the VR world. Finally there’s interactive VR which essentially builds on immersive VR where the user is a full agent participating in the VR itself but they’re also constructing or manufacturing elements of VR as they’re in the space.

To fully engage someone in VR it’s essential to stimulate the participant as much as possible, based on Aristotle’s classic declaration of the five senses as smell, sight, touch, taste and hearing. VR covers over half of these but there is still major room for further development. Common research trends are expanding their knowledge exponentially, specifically looking to introduce further stimulation of all of our senses. Research labs are actively pursuing artificial smell, touch and taste. If tactile senses, taste and smell can be triggered using VR it will add a deeper dimension to the level of immersion a participant is able to experience.

In terms of proprioception which is known to be a sixth sense, an awareness of the self in an environment, it’s informed by external signals and inputs such as atmospheric pressure, balance and upright gravity. It’s an important consideration for VR as the ideal objective is to have a total immersion into the virtual world meaning total disconnect from real reality.

By reducing and minimising the size and needs of virtual reality systems you naturally broaden the possibilities expand the prospects for usability. Murphy explained how VR has been around since the 1960’s and his involvement began in early 1990. In terms of the hype around VR he explains how roughly every decade the technology becomes popular and so this creates a serge in the awareness of VR. He explains how since the 00s though the attention which VR has received is unlike anything previous, it’s fundamentally different as it’s not driven by science fiction or Hollywood but in fact driven by the industry and it’s the first time it has ever been that way. There has been incredibly significant technological developments that were driven by the market, convergence of technology, miniaturization, improvements in power consumption and most importantly, consumers are willing to accept new technology. There is a readiness to embrace and take on new technology, it has become pervasive and it’s not going to be going anywhere any time soon.

Murphy introduced the Gartner Hype Cycle 2018 model which provides a graphic representation of the maturity and adoption of technologies and applications, and how they are potentially relevant to solving real business problems and exploiting new opportunities. At the moment, augmented reality is in the trough of disillusionment as people are disappointed with the developments in AR. There’s still high expectations in relation to VR and Murphy proposes that the issue isn’t with the technology but simply the richness of the available content. There’s a huge need for people to actually develop the content to be able to bring VR to the next level. However, integrity as Murphy explained is crucial, false claims as to the possibilities for AR/VR can be incredibly dangerous, especially in terms of its involvement in medical, safety or dangerous situations.

Ericsson identified three fundamental paradoxes of VR which Murphy outlined to us briefly. Firstly mobility, VR allows you to go anywhere however due to the large amounts of computational workstation needs you cannot take VR anywhere. Secondly isolation, it allows you to socialise with anyone in the world however physically you are on your own, perhaps it will allow for an even further level of disconnection from society. Lastly is integration, those who are socially challenged in the real world often are the ones who are more socially active in VR, this can be argued as a positive thing as they have a place and purpose however it doesn’t address or solve the real problem.

To round up Murphy left very interesting parting thoughts on responsible VR, there are social concerns with this new false world. In VR you have the availability to join online communities people who have similar traits married with a whole new set of social norms and social rules which are not necessarily in line with those of the real world. The normailities of an online society are determined by each community and the members which make up that community. There could be a space created which not only tolerate violence, bullying and abuse but could actively encourage such behaviours. As the boundaries between the virtual world and the real world are blurring there is huge concern and substantial social issues we need to acknowledge as the advancements in this area are developing so rapidly.

There’s also psychological concerns as there has been reports of participants showing signs of PTSD after VR experiences. The premise is very important to understand, the purpose of VR is to have the user fully immersed in the synthetic environment and by manipulating and stimulating their senses for them to truly believe that what they are experiencing is real. You are going to get the same visceral response in VR as if you were present in the real world. Therefore there is major concerns in relation to psychological well being of VR users going forward.

References

Murphy, David. “Virtual Reality/Mixed Reality”. 2019.
Ericsson.com. “An Ericsson ConsumerLab insight report – VR”. 2019.