Mixed Reality for Education (MRE) Implementation and Results in Online Classes for Engineering

Technology has always been present in the education sector; profound changes have occurred in the devices used to teach classes. However, face-to-face classes remain the preferred option for students and teachers. When the pandemic came, it changed all sectors, and education was no exception. In 2018, before the pandemic, only 35% of students who studied a degree reported having taken at least one class online; that is, 65% of students completed their studies in person¹. As of April 2020, by government order (Mexican), all public and private schools were prohibited from teaching face-to-face classes; for this reason, 100% of the students had to take distance classes. Universities were the first to act, using tools for video calling, preparing classes, homework management, etc. This makes sense, since people of university age (between 18 and 25 years old) are people who have been in contact with technology since birth.

Some classes can be fully adapted virtually; however, laboratory practices are complex to perform remotely, and students do not have the necessary material, which is often expensive. The impact that online classes have on the quality of knowledge is unclear, and some studies show that online courses generally yield worse student performance than in-person coursework². But one thing is certain, not carrying out laboratory practices that bring students closer to what they will experience in the industry will negatively affect their professional performance. Therefore, the importance of real-scale experiences becomes necessary in the current teaching of engineering^3,4,5. For these reasons, new technologies are being used to mitigate these problems. Among them are virtual reality (VR), augmented reality (AR), and mixed reality (MR). It is important to mention that VR is a technology that allows the creation of a totally immersive digital environment, whileAR overlays virtual objects in the real-world environment. On the other hand, MR does not just use virtual objects, but also anchors these objects to the real world, making it possible to interact with them. Thus, MR is a combination of VR and AR⁶. On the other hand, some organizations have also made efforts to develop remote laboratories, where real equipment exists but can be controlled remotely⁷.

The term MR dates to 1994; however, in the last 5 years, it has taken on special importance, thanks to large companies that have focused their efforts on developing environments, such as the Metaverse⁶. MR can be applied in different areas; two of the most common are training and education. Training has been one of the great drivers of MR; it is very expensive for a company to stop a production line to train new employees, or in dangerous environments, and it isn't easy to carry out training in the field. Education is not far behind; although face-to-face classes have changed very little, there are great efforts to incorporate MR into classes^8,9. For education, there are professional careers where it is necessary to carry out laboratory practices to have complete training. Many existing studies and research are in medicine, with VR, AR, and MR playing a key role. Multiple papers show how MR surpasses traditional teaching methods in surgical and medical subjects, where the practice is a clear advantage for developing students^{10,11,12,13,14}.

However, there is not the same amount of research on engineering issues. Normally in engineering careers, a student has theory classes complemented by practices. In this way, there are studies on MR and VR showing the benefits in engineering pedagogy¹². However, some of these studies focus on analyzing the complexity of the environment and the tools used^8,15. Tang et al. devised a study where students from different areas and with different knowledge used MR to improve their understanding of geometric analysis and creativity¹⁶. In a subsequent test, people who took their classes using MR finished faster, making it clear that MR positively affects learning¹⁶. Moreover, Halabi showed the use of VR tools in engineering education. Although it is not MR, it shows tools that can be used for teaching. It makes a real case study to show that it is possible to introduce VR in engineering classes¹⁷.

On the other hand, remote laboratories (RLs) are technological tools composed of software and hardware that allow students to remotely carry out their practices as if they were in a traditional laboratory. RLs are generally accessed through the internet, and are normally used when students are required to autonomously put into practice what they have learned as many times as they require¹⁸. However, with the arrival of COVID-19, its use has been to replace traditional laboratories and to be able to carry out practices during online classes¹⁸. As mentioned above, an RL needs a physical space (traditional laboratory) and elements that allow it to be controlled remotely. With the arrival of VR, laboratories have been modeled virtually, and through physical mechanisms, the elements of the laboratory can be controlled¹⁹. However, having an RL is very expensive, impeding many schools especially in developing countries. Some studies mention that costs can vary between $50,000 and $100,000^20,21.

Moreover, since the pandemic began, changes have had to be made quickly; in the case of RLs, attempts were made to send kits to the homes of each student to replace the traditional laboratories. However, there was a cost problem, as studies showed that each kit cost around $700^18,22. Nevertheless, the studies used expensive and difficult-to-obtain components. The pandemic affected education worldwide, and not many people could spend thousands of dollars to automate a lab or buy a kit. Most studies consider face-to-face classes and complement them with MR. However, in recent years, classes have been online due to COVID-19, and only some works show the improvement of virtual classes using MR and affordable devices^23,24.

The research that exists so far is mainly focused on medicine, with little information on engineering. However, without a doubt, we believe that the greatest contribution and difference is that our experiment was carried out for 6 months and was compared with subjects with the same characteristics who did not use virtual models, whereas most previous works carried out short experiments to compare single technologies or procedures; they did not apply them over several months. Therefore, this paper shows the difference in learning that can be made using MR in a university subject.

For this reason, this work shows the development and results of an MR system to help carry out laboratory practices in universities focused on electronic engineering. It is important to mention that special emphasis is placed on keeping the cost of the device low, making it accessible to the general population. Three groups use different teaching methods, and an exam is conducted on the class topics. In this way, it is possible to obtain results on understanding the topics in distance education using MR.

The project explained in this work is called mixed reality for education (MRE) and is proposed as a platform where students use VR glasses with a smartphone (i.e., no special VR glasses are used). A workspace is created where students can interact with virtual environments and real objects simply by using their own hands, due to the use of virtual and real objects, a mixed reality system. This workspace consists of a base with an image where all the virtual objects are displayed and interacted with. The environment created focuses on conducting laboratory practices to show electronic components and physics for engineering careers. It is important to highlight the need to provide feedback to students. For this reason, MRE incorporates a feedback system where an administrator (normally the teacher) can see what is being done to rate the activity. In this way, feedback can be given on the work done by the student. Finally, the scope of this work is to check if there are advantages in using MR in online classes.

To achieve this, the experiment was carried out with three groups of students. Each group consisted of 10 students (30 students in total). The first group did not use MRE, only taking theory (online classes) on the momentum conservation principle and electronic components. The second group used MRE without feedback, and the third group used MRE with feedback from a teacher. It is important to mention that all students have the same school level; they are university students in the same semester and with the same career, studying mechatronics engineering. The experiment was applied in a single course called Introduction to Physics and Electronics, in the second semester of the degree; that is, the students had been in university for less than 1 year. Therefore, the topics covered in the class can be considered basic from an engineering point of view. The experiment was carried out on 30 students, as this was the number of students who enrolled in the class where the experiment was authorized. The selected class (Introduction to Physics and Electronics) had theory and laboratory practices, but due to the pandemic, only theory classes were being taught. The students were separated into three groups to see the impact that the practices have on general learning and if MR classes could be a substitute for face-to-face practices.