• 

• 

I had the great pleasure of joining a Westminster Higher Education Forum event today as a speaker. My session was chaired by the Labour MP Mr Alex Sobel and the main theme of the session is around the opportunities and challenges in adopting new technologies in colleges and universities in the UK. The venue was packed with 100+ delegates from over 60 institutions and businesses across England. I spoke about our research findings on the use of VR in education and shared my views on how technologies can empower human educators in Education 4.0. The following are my notes. The official transcripts from all speakers will be available on Westminster website.

---

Virtual reality in its early days was mainly used for industrial simulation and military training in a controlled environment using specialised equipment. As technologies become more accessible, we started to see more use of VR in gaming and education. In education, VR is mostly used as a stimulus to enhance students engagement and learning experience. It helps visual learners, breaks barriers, and can visualise things that are hard to imagine. So we are mostly encapsulating on the indirect benefit.

My research group is interested in whether such stimuli can truly improve learning outcome and how, so we know how to improve the technology or use it more appropriately. We conducted an experiment with two groups of university students to compare how well they learn hard sciences using VR materials and Powerpoint slides. Their performance was measured using a short exam paired with interviews. The results suggest that the majority of students prefer learning in VR but there is no significant difference between the two on average scores. A recent research by Cornell University shows a similar finding. However, When we look at the breakdown of scores on individual questions we discovered that students who studied via VR can do very well with questions related to visual information recognition but they struggled to recall numerical and textual details such as the year and location of an event. We think its due to how information is presented and the extra cognitive load in VR. So VR made something better but others worse, it’s a double-edged sword.

This does not mean what VR is a waste of money. We need more work to learn how to better use the tool. This means two things: One, we need VR to be more accessible. Not only its cost but more importantly easy-to-use design tools and open libraries that help average lectures to embrace the technology. We also need appropriate metrics and measurement tools to access the actual impact of new technologies, and share that experience with the community.

Furthermore, we need to keep eyes on what roles VR should take in education. One thing we can learn from the past is Powerpoint in education. (Powerpoint was invented in 1980s, acquired by Microsoft, and went on to become one of the most commonly used tools in business and education. Powerpoint has drastically changed how teaching is done in classrooms. ) Powerpoint was meant to augment a human presenter but it has become a main delivery vehicle in classroom while lecturers are the operators or narrators of slides. People conclude that Powerpoint has not empowered academia. (Some institutes have banned teachers using Powerpoint. According to NYTimes, similar decisions were also made in the US Armed Forces because they regard it as a poor tool for decision-making.) Many institutes including University of Northampton are moving away from pure slideshow to active and blended learning and use data sciences and smart campus to support hands-on, experimental and interactive learning. So we can certainly learn from the past when we approach VR and other new technologies.

Another important aspect is the human factor. At the end of the day, only human educators are accountable to the teaching process. We listen to what learners say, observe their emotions, sympathise with their personal issues and I reason with them for every decision I made while trying to be as fair as possible. My team is work on many computer science research topics related to human factors such as interpretable machine learning, understanding human intent. However new technologies such as VR and AI should be designed and integrated to empower human educators rather than replacing us.

---

Like many academics, I regularly engage with the reviewing process of renowned conferences and journals. I have not ventured into any substantial editorial role yet but I try to help out as much as possible. Any of my journal review activities are registered on Publons to keep a record (mainly for myself). It is to my surprise that I received the Publon’s Peer Review Awards 2018 as the “Top 1% of reviewers in Computer Science”. I know many people will see this as a “gimmick”, but hey, our research communities rely heavily on quality peer reviews from volunteers. Also, an award is an award! 🙂

While Part 1 “rants” about how function-based machine learning overwhelms data research, this part aims at future opportunities.

Many of my childhood friends and I counted ourselves lucky to own a copy of the A Hundred Thousand Whys (Chinese: 十万个为什么), a children’s science book series. It was the only child-friendly encyclopedia you could find in the 70s/80s in China. The book series originated from a book with the same name authored by a Ukrainian writer Mikhail ll’in (Russian: М. Ильи́н). The fact that our parents invested half a month salary on these books in those difficult years showed how much they struggled to answer our never-ending questions of whys.

Reasoning is a very important part of the learning process. We humans and other animals learn very quickly from direct experiences for survival. At the same time, abstract knowledge is the core of our human intelligence. We discover rules and develop theories from our observations and experiments so to generalise our findings and use them for situations other than our initial observations. It is believed that “the power of generalization is a property that comes with neural networks” and “abstraction is often attributed to humans and the development of the language”. A conventional function-based machine learning models tend to give us a very good answer under normal circumstances but there will be no reasoning on how the conclusions or predictions were made. It is therefore often referred to as a black box. We don’t know what’s in the black box. Even if we do by illustrating the ML neural networks, chances are we won’t be able to explain it rationally. What made things more challenging (I am ranting again…) is the opacity of the input data to the black box. Data science researchers need to be very careful when handling and documenting the training data because they determine the sensitivity and operational range of your model. However, that’s just me training my own model for research. Do we know exactly what data Google uses to show stuff to us? So we have no clue how the machine works nor how the machine gets its knowledge from.

Some may argue that they don’t really care about how things were done as long as we have a good result. This kind of attitude has helped the function-based ML to thrive in the past few years. We are now getting used to the convenience enabled by ML-based software and services without bothering about the whys.

So is reasoning still relevant?

Even if we don’t expect a machine to ever be as smart as humans, we still need results interpretable to ensure fairness, transparency, and accountability. The fairness concerns with whether resources are shared “equally” between parties or whether a rule is applied consistently. Many researchers including myself argue that fairness is subjective and therefore needs to be measured at the user level. We spent a lot of time looking into human factors (sensual, perceptual, societal, etc.) and studying how the machine can take into account individual preferences and experience. However, I often feel what we do is an exercise of incorporating biases. The algorithm would allocate more resources to picky users and hungry user devices so that there is minimum QoE (quality of user experience) discrepancy across a group of users. You might disagree with my understanding of fairness but at least we are able to discuss what’s right and wrong based on my reasoning. With a high degree of complexity, an ML model can produce excellent outcomes with the wrong reason. The image below from Dr Kasia Kulma’s talk is a perfect showcase. The model which achieves high precision in classifying wolf and husky apparently recognise wolves by detecting snow in the picture. It certainly capitalised on the high probability of snow and wolf co-appearance in an image: a smart move to deliver results but not necessarily what we consider as intelligent. For fairness, the reasoning often outweighs the outcome.

The transparency is a measure for the general interpretability or opacity. There are different opinions on the measurement target. Some believe it’s about the model itself (i.e., the interpretability of how the model works). However, this might be too much to expect for intricate deep learning models. Furthermore, forcing higher transparency affects how models are built and potentially limit the use of some useful features. There is generally a trade-off between transparency and model performance. As a compromise, the interpretability can be set on the results coming out of a model rather than the model itself. This means that we attempt to interpret how the input features impact the results (on a much smaller scale). It would also enable some hypothetical reasoning to support humans to optimise a solution.

When it comes to using ML in critical missions such as performance review, credit rating, and autonomous vehicles, the main concern is people can get hurt when things go wrong. And that may be due to the wrong usage of training data or the algorithm itself. Ultimately, humans are accountable for what machines do. Even if its something that the machines taught themselves to do, its human who instructed the learning process. Prof Ben Shneiderman‘s talk covers this topic quite extensively from an HCI perspective. However, not many people consider the human-computer relationship as part of ML/AI research. Zero-control automation has a pivotal role in future network and system designs: it’s impractical to get human to make every single decision and we want to reduce human errors when humans are involved in decision making. Therefore, algorithm accountability is a technical challenge but perhaps more of a regulatory challenge.

[to be continued]