The review should be one page, double spaced. The review should be a ? page summary of the article, and a ? page application of the material to the real world. Please, see

Ilrnovative Food Science and Emerging Tedmologies 56 (2019) 102178

Contents lis ts available at ScienceDirect

Innovative Food Science and Emerging Technologies

journal homepage: www.elsevier.com/locate/ifset

Potential applications for virtual and augmented reality technologies in

sensory science

E.C. Croftona ,,,, C. Botinestean3 , M. Fenelon E. Gallagher" • Teagruc Food Research Cl'nlre, Ashtown, Dublin, Ireland • Teaga.,c Food Research Ce/lJre. Moa-eparlc, lrd.and

ARTICLE INFO ABSTRACT

Keywords: Sensory science has advanced significantly in the pa�1: decade and is quickly evolving to become a key tool for Virtual reality predicting food product success in the marketplace. Increasingly, sensory data tedmiques are moving towards Augmented reality more dyn amic aspec.1:S of sensory perception, taking account of the various ,1:ages of user-product intenK.1ions. Emerging technologies Recent technological advancements in virtual reality and augmented reality have unlocked the potential for new Sensory science

immersive and intemt1ive systems which could be applied as powerful tools for capturing and deciphering the complexities of human sensory perception. This paper reviews recent advancements in virtual and augmented reality technologies and identifies and explores their potential application within the field of sensory science. The paper also considers the pOS1.ible benefits for the food industry as well as key challenges posed for wide­ spread adoption. The findings indicate that these technologies have the potential to alter the research landscape in sensory science by facilitating promising innovations in five principal areas: consumption context, biometrics, food structure and texture, sensory marketing and augmenting seruory perception. Jlthougb the advent of augmented and virtual reality in sensory science offers new exciting developments, the exploitation of these technologies is in its infancy and future research will understand how they can be fully integrated with food and human responses. In�trial relevance: The need for sensory evaluation within the food industry is becoming i ncreasingly complex as companies continuously compete for con,"ll[l]er produt1: acceptance in today's highly innovative a nd global food environment. Recent technological developments in virtual and augmented reality offer the food industry new opportunities for generating more reliable insights into coru"Ulller sensory perceptions of food and bev­ erages, contributing t o the design and development of new products with optimised consumer benefits. These technologies also hold sigrrificant potential for improving the predictive validity of newly launched produt1:S within the marketplace.

1. Introduction acceptability and compe titiveness of food products within the market­

place (Tuorila & Monteleone, 2009). However, in the current highly

Sensory evaluation is a scientific discipline that is used to unde r­ competitive and global food environment, the need for sens ory in­ s tand h ow hu mans perceive and respond to the various stimuli in food formation is becoming increasingl y complex, as industry face constant

using the five senses of sight, smell, touch, taste and hearing. From a pressure to develop high quality food products a t reduced time-to­

fundamental perspec tive, i t attempts to understand the intricacies of market (Delarue, 2 015). Over the pas t decade, advancements in digital

sensory pen:eption and consumer behaviour, w hile at an applied level i t technol ogies, such as smartphones and social media applications, have

can b e us e d as a tool across the produc t developmen t process to char­ stimulated a new era of consumer connectivity, providing researchers acterise and understand h ow the s e nsory properties of food drive con­ with opportunities to collect new types of sensory information (Jaeger

sumer decision making and hedonic resp onse (Kemp, Ng, Hollowood, & et al., 2017; Jaeger & Por cherot, 20 17). As a result, the range and so­

Hort, 2018). For many years, sensory data was based on ave raging phistication of techniques available for capturing and deciphering

sensory responses from consumers e valuating food s under controlled consumer's sensory per ceptions toward s food is evolving substantially

conditions in a sensory laboratory (Hathaway & Simons, 2017), and to meet industry demand s. Sensory evaluation is b e ing increasingly was simply viewed by industry as a means for comparing the applied by comopanies in both developed and emerging markets, as a

• Corresponding author. E-mail address: [email protected] (F.C. Crofton).

https://doi.org/10.1016/j.ifset.2 019.102178 Received 24 November 2018; Received in revised furm 29 January 2019; Accepted 13 J u ne 2019 Available online 19 J 1111e 2019 1466-8564/ © 2019 Published by Flsevier Ltd.

E.G. Crofton, el' al

powerful tool for predicting product success across a range of industrial

fuoctions including research and development, quality control and

marketing (Delarue, 2015; Kemp et al., 2018).

As the digital world continues to evolve at a rapid pace, new virtual

reality (VR) and augmented reality (AR) technologies are emerging

with the potential to transform the landscape for collecting and pro­

cessing sensory and consumer information. Although both virtual and

augmented reality have existed in various forms for decades (Ong &

Nee, 2004), it is only recently these technologies have advanced to a

point of radically changing how people connect and interact with the

world (Porcherot et al., 2018; Velasco, Obrist, Petit, & Spence, 2018).

Research on the industrial applications for VR and AR is a strong and

rapidly growing area, with the market for these technologies projected

to reach $162 billion by 2020 (Business Insider, 2016). Recent tech­

nological developments in VR and AR are already showing a demon­

strable impact across a number of industries including healthcare

(Silva, Southworth, Raptis, & Silva, 2018), manufacturing (Bottani &

Vignali, 2018), engineering (Singh & Erdogdu, 2004), entertainment

(Aukstakalnis, 2017), education (Merchant, Goetz, Cifuentes, Keeney­

Kennicutt, & Davis, 2014), automotive (Lawson, Salanitri, & Waterfield,

2016) and travel (Gibson & O'Rawe, 2018). For example, within in­

dustrial manufacturing these technologies have been applied

throughout the production process from initial product design and as­

sembly operations, through to enabling real-time discussions between

multidisciplinary teams located at across the globe, resulting in fewer

design flaws, enhanced workflow efficiency and increased savings in

terms of costs and man-hours (Aukstakalnis, 2017). Beyond manu­

facturing, AR technology is being used by the automotive industry to

support accident and emergency services. For example, Mercedes Benz

are placing quick response (QR) codes on the B-pillars and fuel doors of

all new cars, enabling first responders to quickly view colour-coded

images of wiring and fuel systems using an AR mobile application

(Etherington, 2016). As these technologies open up a world of possi­

bilities for transforming the real world and how people in teract with it,

the food industry is now endeavouring to understand how to capitalise

on these digital tools for competitive gain. The focus of this paper is to

review recent advancements in virtual and augmented reality technol­

ogies and to explore their potential applications within the field of

sensory science, highlighting the potential benefits for the food industry

and outlin.ing the challenges that currently exist for widespread adop­

tion of these technologies.

2. Recent advancements in virtual and augmented reality

technologies

2.1. Virtual realily

Although virtual reality and augmented reality are both evolving

interface systems for displaying digital information, they are distinct

technologies with fuodamental differences in the type of computing

systems required to experience them. Due to continuous advances in

core enabling technologies and the conflicting meaning of the words

virtual and reality, the term virtual reality has been exceptionally dif­

ficult to define and no single definition exists in current literature as a

result (Aukstakalnis, 2017). Generally, VR is described as an immersive

human-computer interaction in which an individual can explore and

interact with a three-dimensional computer-generated environment. A

VR experience is typically accomplished through the use of a stereo­

scopic head mounted display (HMO) which completely replaces the

user's view of the real physical world with an interactive synthetic

environment (Siegrist et al., 2018; Silva et al., 2018 ). Virtual reality as a

concept is not entirely new, a nd dates back to the late 1950s when

Morton Heilig, an American cinematographer, developed (and later

patented) the Sensorama, an arcade-style cabinet which stimulated the

senses through the use of stereoscopic 3D images, stereo speakers, fans

and a vibrating chair. The Sensorama is considered one of the earliest

Innovative Food Science and !'merging fechnologies � (2019) 102118

examples of immersive, multisensory technologies. In the late 1960s,

computer graphics pioneer Ivan Sutherland, alongside his student Bob

Sproull, engineered what is widely considered to be the first HMO

system known as the Sword of Damocles (Sutherland & La Russa, 2017).

The weight of the system required it to be suspended from the ceiling,

but the technology was capable of tracking the movement of the user's

head, and could display simple 3D wireframe images in the user's

viewing direction. Throughout the 1960s and 1970s, the United States

government, particularly the National Science Foundation, Department

of Defence, and the National Aeronautics and Space Administration

(NASA) were also involved in their own research efforts, which yielded

a large pool of skilled researchers in areas such as computer graphics,

network infrastructure and simulation modelling (Lowood, 2018;

Sutherland & La Russa, 2017). However, the term virtual reality was

not popularised until the late 1980s, when Jason Lanier, founder ofVPL

Research, developed the first commercially available VR products in­

cluding the Dataglove and the EyePhone head mounted display (Lauria

& Ford-Morie, 2015).

Between these early systems and today, major advances in comp u­

tational power and visualisation and tracking technologies have given

rise to a new era of affordable, fully immersive stereoscopic HMDs that

are widely available to consumers. Nowadays, there is a multitude of

VR HMO devices on the market ranging from high-end PC- based or

"tethered" display systems to lower end devices driven by smartphones.

While the majority ofVRasystems developed to date focus on controlling

the user's visual and auditory experiences, technologies such as haptic

gloves and full-body haptic suits are being increasingly used for adding

tactile and kinaesthetic content across a range of VR applications

(Aukstakalnis, 2017; Gallace & Spence, 2014). Haptic gloves, such as

CyberGrasp"' , Dexmo"' and HaptX, are capable of stimulating the sense

of touch by transmitting tactile inputs (e.g. vibrations and pressure) to

the user's skin using force feedback tec hnology. However, effectively

reproducing tactile sensations within a VR system poses considerable

technological challenges due to the complexity of the human nervous

system (Aukstakalnis, 2017; Perret & Vander-Poorten, 2018).

lo terms of PC-based VR, the Oculus Rift and HTC Vive were the first

modern, commercially available pla tforms to retrigger the public's in­

terest in virtual reality technology. Following the pre-release of two

developer models and acquisition of Oculus by Facebook for $2 billion

dollars, the consumer version of Oculus Rift was released in early 2016.

VR competitor to the Oculus Rift, HTC Vive, was released one month

later following collaboration between Valve Cm:poration and smart­

phone manufacturer, HTC (Sutherland & La Russa, 2017). Both HMDs

are equipped with a high resolution display of 1080 x 1200 pixels per

eye with a field of view extending 110-degrees, and have built-in sen­

sors for tracking the position and orientation of the user using six de­

grees-of-freedom, enabling the user to move around the virtual space.

Additionally, motion-tracked controllers allow the person to use their

hands to intuitively manipulate virtual objects and interact with the

computer simulation, further enhancing the feeling of immersion

(Siegrist et al., 2018). While PC driven VR undoubtedly offers the most

powerful immersive experience of modern VR technology, the headsets

are expensive and require the user to be physically tethered to a com­

puter with advanced processing power. As a result, mainstream adop­

tion of consumer PC VR has been primarily in the gaming industry to

date (Aukstakalnis , 2017). Nonetheless, new headsets are increasingly

being launched into the market delivering more sophisticated levels of

technology. For example, the HTC Vive Pro, launched in April 2018,

provides wireless capability and features a 78% resolution increase to

1400 x 1600 per eye, promising the consumer a more immersive and

comfortable VR experience (Warren, 2018).

Advancements in smartphone technology (e.g. increased processing

power, higher pixel counts, and high-performance sensors) have played

an integral role in connecting consumers to virtual worlds. Smartphone­

based VR requires a headset with custom lenses in which a user can

simply insert a compatible phone. Smartphone VR devices currently

2