This study describes a WebVR-based online virtual reality (VR) laboratory system that provides users with immersive and interactive experimentation capabilities supported by VR devices. The proposed system not only helps to enhance the realism of user participation in online experiments but is also applicable to a wide range of online laboratory frameworks.
Online laboratories play an important role in engineering education. This work discusses a WebVR-based virtual laboratory system. The user enters the simulated laboratory environment through a virtual reality (VR) device and interacts with the experimental equipment, similar to hands-on experiments in a physical laboratory. In addition, the proposed system allows users to design their own control algorithms and observe the effects of different control parameters to enhance their understanding of the experiment. To illustrate the features of the proposed virtual laboratory, an example is provided in this paper, which is an experiment on a double inverted pendulum system. The experimental results show that the proposed system allows users to conduct experiments in an immersive and interactive manner and provides users with a complete experimental process from principal design to experimental operation. A solution is also provided to change any virtual laboratory into a WebVR-based virtual laboratory for education and training.
With the advancement of the Internet and the popularity of mobile devices, the demand for online education is increasing1. In particular, during periods of widespread epidemics, traditional educational institutions often face challenges in conducting in-person instruction effectively, which highlights the importance of online education as an important pedagogical approach2. Theoretical courses are relatively easy to transfer to online platforms. They can be conducted with the help of tools such as remote video conferencing software and massive open online courses (MOOCs)3. However, practical courses face greater challenges as they require users to perform hands-on experiments in traditional laboratories.
Researchers have made significant contributions to addressing the challenge of making experimental equipment available online. Over the past two decades, extensive studies have been conducted on the concepts and technologies of online laboratories4,5. Online laboratories typically encompass remote laboratories6, virtual laboratories7, and hybrid laboratories8. These online laboratory approaches have found widespread application in various engineering disciplines, including control engineering9, mechanical engineering10, and software engineering11.
While significant progress has been made in terms of the convenience of experimental operations in online laboratories12, users still perceive a lack of realism and similar hands-on practical operations compared to traditional laboratory environments, which affects their overall experience13. This discrepancy in user experience motivates further research and development efforts to enhance realism and engagement in online laboratory environments.
To solve the above problems, virtual reality (VR) technology has been applied in virtual laboratories14 to improve the immersiveness and interactivity of virtual laboratories15. VR-based virtual laboratories provide users with a close-to-realistic experimental experience. Users can complete group assignments in the architectural education process through avatars16, performing the architectural surveying process together immersively, just as they would in a traditional classroom environment. Furthermore, the VR-based virtual laboratories allow users to enter the immersive environment of virtual laboratories and interact with virtual experimental equipment by wearing VR headsets and handles17, improving users’ hands-on abilities18. For different educational purposes, we can design different virtual environments. For example, VR can be combined with gamification theory to enhance engineering education for the general public and to improve the efficiency of disseminating difficult-to-understand knowledge such as sustainable development19.
Similar to online laboratories, particularly virtual laboratories, WebVR-based virtual laboratories have many advantages. Firstly, they break through the time and space limitations of traditional laboratories, and users can conduct experiments anytime and anywhere20. Secondly, online laboratories can provide a safer experimental environment to avoid possible dangers and accidents in experimental operations21. Thirdly, virtual laboratories can also provide more experimental resources and simulation situations to extend users’ experimental scope and experience22. Most importantly, WebVR-based virtual laboratories can stimulate users’ learning interest and initiative and enhance their experimental experience and participation23.
Compared with other VR-based virtual laboratories, WebVR-based virtual laboratory seamlessly combines the merits of VR-based virtual laboratories with web-based online laboratories. Virtual Instrument Systems in Reality (VISIR)24 builds a basic analog electronic remote laboratory by constructing real circuit boards. Users can perform simulated experiments on the web interface to complete real circuit board experiments. Weblab-Deusto8 builds the water tank Field Programmable Gate Array (FPGA) laboratory where users can interact with the three-dimensional (3D) model of the water tank in the web platform without relying on other plug-ins. The system proposed in this paper introduces the capability to seamlessly integrate WebVR as a modular component into the existing virtual laboratory infrastructure. This integration can be achieved without destroying the original architectural framework of the laboratory, thus preserving the basic structure and function of the laboratory. This integration is also applicable to the framework of an online laboratory with separate front end and back end.
The system proposed in this paper is implemented based on Networked Control System Laboratory (NCSLab)25, which inherits the flexibility, interactivity, modularity, and cross-platform features of the NCSLab system. Users can conduct experiments according to different modules and can also customize algorithms and configuration interfaces, providing users with enough space for self-realization. Online experiments are driven in real-time according to the algorithms run by the user. Users can interact with the virtual model to change the inputs of the experimental algorithm when conducting VR experiments and can even change the parameters of the control algorithm through the components so that users can experience the principle of the control algorithm more realistically.
WebVR-based virtual laboratories bring great potential for online education. It can provide an immersive experimental experience, overcome the limitations of traditional laboratories, and promote hands-on practical skills and innovative thinking among users.
This study met the guidelines of the Human Research Ethics Committee at Wuhan University, and informed consent was obtained for all experimental data. In this paper, the experimental steps for the double-inverted pendulum system are discussed, and all the steps are performed in the WebVR-based NCSLab.
1. Access WebVR-based NCSLab system
2. Selecting the access method
3. Experimental procedure
The VR experiment system presented provides users with the capability to engage in immersive experiments using VR devices, thereby enhancing the interaction between users and the experimental equipment. Furthermore, the system is web-based, eliminating the need for users to configure local environments. This design allows for the system's scalability, making it suitable for large-scale applications and training and educational purposes.
In traditional laboratory environments, users are required to personally configure and install software and hardware devices, which can consume a significant amount of time and resources26. However, virtual laboratories leverage cloud computing and virtualization technologies to move the laboratory environments to the cloud. Users can simply access the corresponding website through a web browser to utilize the functionalities and resources offered by the laboratories.
Figure 3 demonstrates that users can engage in WebVR experiments using different approaches. Users who do not have readily available VR devices can quickly conduct experiments through browser extensions. Users who have access to VR devices can immerse themselves in the experiments and interact directly with the experimental equipment, enhancing realism when exploring the experimental process. These two different ways of conducting WebVR experiments provide users with more options and enable a wider range of users to utilize the proposed system.
The double inverted pendulum examples demonstrate that the proposed WebVR-based virtual laboratory can run directly in a web browser without the need for additional software installations or configurations. This approach not only reduces user inconvenience but also greatly enhances the system's scalability. Additionally, users have the option to use VR devices for immersive interaction with the experimental equipment. By using handle controllers to adjust system parameters, users not only enhance their hands-on experience but also improve their theoretical knowledge and practical skills.
A total of 21 students participated in the experiment, where a questionnaire was conducted to further validate the applicability and effectiveness of the proposed system. We included students with backgrounds in automation and control engineering, and all of these students had previously participated in virtual experiments in NCSLab and had some basic knowledge of virtual experiments but had not participated in VR experiments in WebVR-based NCSLab. By adopting anonymous statistical data, we guarantee the privacy and security of the participants when filling out the questionnaire, thus ensuring the reliability of the questionnaire data.
The results of the questionnaire are shown in Figure 5, and the data indicate that the system proposed in this paper performs well in terms of realism and interaction with the device and achieves significant improvement compared to the traditional mouse-keyboard virtual experiment. In addition, participant feedback showed that the system not only increased students' interest and experimental skills in learning but also helped them better understand the experimental content, thus enhancing learning outcomes.
It is worth noting that most of the students believed that this type of experimentation is not only applicable to the current course and experiment but also has the potential to be applied in other courses and experiments.
The system proposed in this paper uses 3DS Max software for modeling the experimental equipment, which renders the experimental scenes using Unity engine software27 and allows users to interact with the equipment using VR devices. Finally, the experimental scenes are packaged into Web Graphics Library (WebGL) format and seamlessly integrated into the online laboratory system in the form of modularized components to construct a WebVR-based virtual laboratory system.
Figure 1: Control algorithm design for the double inverted pendulum system. Users can select different modules from the module library on the left to build the control algorithm for the double-inverted pendulum system. The selection and connection of modules are similar to those in MATLAB/Simulink. In the realm of double-inverted pendulum systems, a plethora of control methods abound. For the present system, the chosen strategy is the Linear Quadratic Regulator (LQR) control approach, and the figure illustrates the feedback matrix fashioned in accordance with the LQR controller. Please click here to view a larger version of this figure.
Figure 2: Configuration design for monitoring the double-inverted pendulum system. Users can select components from the component library above to design the monitoring configuration. If a VR experiment is desired, the 3D Model component must be selected. Users have the flexibility to opt for the Chart component to visually track alterations in the angular orientation and position of the double inverted pendulum or the input component for making adjustments to controller parameters. The double-click on the component permits users to establish associations between system variables for parameter configuration. Within the double-inverted pendulum system, the parameters of the chart are configured to encompass both the set and actual positions of the cart, along with the angles of the first order and double pendulums. After the completion of the monitoring configuration design, users should first activate the experiment by clicking the Start Experiment button. Following this, they can initiate the VR experiment by clicking the VR button located in the lower right corner of the 3D Model component. Please click here to view a larger version of this figure.
Figure 3: Conducting the double inverted pendulum system experiment using VR headset and the WebVR emulator extension. Users can conduct WebVR experiments through VR devices or the WebVR emulator extension. The cube is controlled to set the setpoint for the double inverted pendulum using a handle. Once the position of the cube is determined, the double inverted pendulum will steadily move towards the setpoint direction until it eventually stabilizes at the set position. On the right side of the 3D model is a chart that records the cart position and the angles of the first order and double pendulums. The chart also allows for the observation of the trend of changes in key system parameters. Please click here to view a larger version of this figure.
Figure 4: Structure of the double inverted pendulum system. There is a cube above the base, and the position of the cube is the setpoint of the cart. Users can pick up the cube and adjust the position by the handle. Once the alternating current (AC) servo motor propels the belt into rotation, the cart will proceed along the guide rail under the impetus of the belt. In concert with this motion, the first-order pendulum and the double pendulum will also undergo corresponding displacement and rotation. Please click here to view a larger version of this figure.
Figure 5: Data results of the survey questionnaire. The questionnaire comprised six questions, each meticulously detailed here. Each question had five options, roughly meaning strongly disagree, disagree, neutral, agree, and strongly agree, on a scale of 1 to 5. A total of 21 valid responses were collected. The mean values and standard deviation were calculated from these scores and graphically presented in the figure for clarity and interpretation. Please click here to view a larger version of this figure.
The presented protocol describes a virtual laboratory system that enables users to conduct VR experiments online but also uses a low-cost PC controller28, which is conducive to large-scale application promotion. Users can gain knowledge about the entire experimental process, from principles and algorithms to practical experimental operations. This system allows users to immerse themselves in the experiments, eliminating the reliance on traditional mouse and keyboard input. This system provides an immersive experience for observing the experimental process and hands-on manipulation of experimental devices.
This system goes beyond traditional interfaces and provides users with a more intuitive and engaging way to interact with experimental equipment. Similar to hands-on experiments in a physical laboratory, this virtual laboratory strives to recreate the experimental operations as faithfully as possible. This online access approach provides virtual laboratories with the following advantages.
Flexibility and convenience: Users can access virtual laboratories anytime and anywhere through a web browser without being limited to specific physical laboratory locations and schedules. This approach greatly enhances the convenience of remote learning2.
Scalability and cost-effectiveness: Virtual laboratories can easily scale and provide additional computing resources and experimental equipment to meet the demands of large-scale applications. Users do not need to purchase and maintain expensive hardware devices themselves but can conduct experiments using web-based resources, reducing their cost investments3.
Security: Virtual laboratories can offer enhanced security measures. Users do not need to worry about accidents resulting from mishandling during experiments, which helps ensure their safety to a certain extent29.
By leveraging VR technology, users can enter a simulated laboratory environment where they can interact with objects and conduct experiments using the handle, similar to a physical laboratory. As shown in Figure 3, users can use the handle to pick up and move the cube to set the setpoint for the cart in the double inverted pendulum system. This form of interaction not only adds a new level of realism and interactivity to the virtual laboratory experience but also enhances users’ understanding of the experiment.
Additionally, this system provides users with opportunities to explore experiments. They can design their own control algorithms and observe the effects of different control parameters, which helps them gain a deeper understanding of the experimental principles30. It cultivates a sense of participation and active learning among users.
Currently, VR laboratories are primarily designed and utilized for specific scenarios, lacking a framework for large-scale applications. Users are often limited to conducting experiments according to predefined steps, with limited opportunities for implementing their own ideas. In contrast, a WebVR-based virtual laboratory seamlessly integrates experimental content as component modules into the virtual laboratory. This approach is not only versatile, fitting into a wide range of application frameworks, but also empowers users to interact with experimental equipment and carry out customized experiments according to their preferences and needs.
Nonetheless, certain issues deserve attention and resolution. These include the need for a more extensive repository of virtual resources, as well as the requirement for enhanced precision in simulating the dynamic behavior of virtual devices in comparison to their physical counterparts. We plan to expand our Virtual Resource Repository by working with subject matter experts in different fields, which will ensure that we cover a wide range of experimental content from natural sciences to engineering to meet the needs of different users. In addition, we encourage users to actively participate in the construction of our system. In addition to providing suggestions for the repository, we also plan to conduct user surveys and interviews in the future to gain a deeper understanding of the types and areas of resources that users expect. To address the precision challenge of simulating dynamic behaviors, advanced modeling techniques, such as machine learning-based methods or more complex mathematical models, are used to improve the precision of virtual device representations. Additionally, real-world systems are often characterized by uncertainty, which needs to be incorporated into the approach to virtual device simulation while maintaining accuracy, allowing for a more realistic representation of the real world.
In summary, the proposed virtual laboratory system enables users to participate in VR experiments in an immersive and interactive manner. By providing an experimental experience that is as realistic as possible, it enhances users’ understanding of the experimental process, from principles and design to experimental operations. The online accessibility of the system also offers a flexible, convenient, and safe experimental environment, making it a promising solution for scientific research and educational training on a large scale.
The authors have nothing to disclose.
This work was supported in part by the National Natural Science Foundation of China under Grant 62103308 and Grant 62073247, in part by the Fundamental Research Funds for the Central Universities under Grant 2042023kf0095, in part by the China Postdoctoral Science Foundation under Grant 2022T150496, and in part by Wuhan University Experiment Technology Project Funding under Grant WHU-2022-SYJS-10.
3DS Max | Autodesk | 3ds Max professional 3D modeling, rendering, and animation software enables you to create expansive worlds and premium designs. https://www.autodesk.com/ca-en/products/3ds-max/overview |
|
Meta Quest 2 | Meta Platforms | 10036728220341 | meta quest 2 is a standalone virtual reality headset that allows users to experience WebVR content. https://www.meta.com/it/quest/products/quest-2/ |
Unity | Unity Technologies | Unity is the platform for real-time 3D interactive content creation and operation. All creators, including game developers, artists, architects, automotive designers, film and television, use Unity to bring their ideas to life. The Unity platform offers a complete suite of software solutions for creating, operating, and realizing any real-time interactive 2D and 3D content on cell phones, tablets, PCs, game consoles, augmented reality, and virtual reality devices. https://unity.com/cn |