[ update: the research idea described in this post has supported a successful outline proposal to EPSRC High-risk speculative engineering and ICT research: New Horizons ]

In the past few years, we have had a series of projects on capturing and modelling human attention in VR applications. Our research shows that eye gaze and body movements share a pivotal role in capturing human perception, intent, and experience. We truly believe that VR is not just another computerised environment with fancy graphics. With the help of biometric sensors and machine learning, VR can become the best persuasive technology known to HCI designers. In a recent project, we demonstrated how machine learning can be automated to study visitor behaviours in a VR art exhibition without any prior knowledge of the artwork. The resultant model then drives autonomous avatars (see below) to guide other visitors based on their eye gaze and mobility patterns. With the “AI avatars”, we observed a significant increase in visitors’ interactions with the VR artwork and very positive feedback on the overall user experience.

The COVID-19 pandemic and its prolonged impact on health services made us rethink our research priorities. While we are still enthusiastic about digital arts, we wanted to make good use of our VR and data science know-how for healthcare innovations. Using VR and AI in healthcare is not a new idea. There are already tons of existing research on VR-based therapies, especially for the treatment of phobia and dementia. AI has been used to develop chatbots, to detect COVID-19 symptoms, etc. The research we’ve seen so far are very promising from an academic perspective but most of them aim at augmenting traditional practices for improved outcomes. This means that any developed application will still need to be operated by a technician in a controlled setting. Recognising the healthcare innovations in the research communities, we are interested in a new form of design that can deliver automated or even autonomous assessment and treatment of diseases in a remote location, e.g., patients’ own home or an easily accessible community centre. This will ultimately help reduce the amount of health care appointments and patients’ trips to hospitals.

The pandemic has added long-lasting impacts on public mental health due to social isolation, loss of coping mechanisms, reduced access to health services, etc. We believe VR and AI research should see a major shift from exploratory proof-of-concept to product-focused development with wider public engagement. Just like how every Tesla car and every Google search improves their underlying ML models, mental health innovation must aim at large scale user trials to achieve any major transformation. To this end, we now pair with the R&D department of a leading mental health institution to engineer new VR applications for new adventures. We hope that customised VR stimuli and NLP dialogue engines will lead to more effective treatment that was not possible in the past due to constraints in the physical world. We are also quite excited about the opportunities to automate the assessment of mental disorders through biometric sensors and machine learning.