1. INTRODUCTION

The onslaught of the COVID-19 pandemic has impacted the world in many ways. Online technologies have experienced increased demand and adaptation. The risk of the virus prompted a quick restructuring of existing teaching methods, leaving learners and instructors to adjust to an unfamiliar environment. Yet the global need for qualified engineers continues to grow and the infusion of new technologies, such as mixed reality (MR) will meet the ever-changing needs of diverse learners and the discipline of engineering. This need presents a rich opportunity for researchers who are interested in the application and effectiveness of MR tools.

Engineering colleges are progressively becoming more invested in motivating and retaining diverse engineering students. At the same time, old issues of learner motivation, failure rates, dropouts, and career transitions away from engineering continue to plague the academic and workforce communities. In 2018, the American Society for Engineering Education (ASEE) analyzed data from 431 institutions in the U.S. offering engineering degrees. Overall undergraduate engineering degrees increased to 622,502 full-time students who continued an upward trend of the previous ten years (Roy, 2019). MR has the potential to meet the needs of learners with disabilities, unique accessibility requirements, language challenges, emergent bilingual possibilities, as well as serving learners who may be otherwise marginalized or underserved.

2. MIXED REALITY

Over five decades after Ivan Sutherland's invention of computer-based virtual reality (VR), the field has rapidly advanced in terms of complexity and application. With its roots in gaming and entertainment, the technology has spawned new mediums in augmented reality (AR) and MR with the application for use spreading to industry and education. Dascalu et al. (2014) defined the Reality-Virtuality Continuum, which provides a comprehensive understanding of the technology. Four components exist: first, reality FOREIGN, is what is physically known to exist by sight, smell, or sound. Next, AR is represented by virtual objects combined with the real world or objects. Third, VR manifests as a complete digital representation of an environment. Lastly, MR is created with a combination of the physical and digital worlds.

The field of engineering has seen explosive growth in the number of tools that can be leveraged to save time and energy innovatively. Examples include rapid prototyping, process automation, and SMART simulations (LaCoche et al., 2019; Ramirez et al., 2019). The widespread use in industry has necessitated the adoption of preparatory engineering programs. Engineering students should learn the use of MR tools as part of the discipline and future careers. Still, the tools also serve an essential pedagogical purpose, particularly in an online learning environment. MR tools enable students to simulate scenarios and manipulate digital content in an alternative world containing computer-generated items that interact with human sensory and motor systems. This concept is further defined as the user interaction and manipulation of real and virtual objects and spaces through learner immersion or presence within the educational context (LaCoche et al., 2019; Milgram & Colquhoun, 1999; Müller & Ferreira, 2003).

MR tools may help bridge the content dissonance learners encounter because of transactional distance in an online course; see transactional distance theory by Moore (1973) and Moore & Kearsley (2012). Moore & Kearsley (2012) assert the physical separation in distance education leads to a psychological space of potential misunderstandings and a communication gap (i.e., transactional distance) between the instructor and the learner. This condition is likely compounded in online courses with complex content or abstract concepts. Transactional distance theory is relevant conceptually because it explains why the use of tools, such as virtual, augmented, or MR tools, can help bridge the perceived distance in an online learning environment (LaCoche et al., 2019; Moore & Kearsley, 2012).

Despite the potential benefits of incorporating MR tools into engineering courses, the barriers to adopting the technology are many. Faculty identified obstacles to technology integration, including MR, such as concerns of decreasing the quality of learner interaction, lack of preparation time for the courses, increased workload with little or no instructional design support, the difficulty of addressing technological savvy and complexity of the online environment, lack of motivation, and lack of funding (Lin et al., 2014; Oh & Park, 2009; Porter et al., 2016). The resistance to incorporating technology-based learning experiences such as MR is part of a more significant learning equity issue. The U.S. Department of Education (2017) identified faculty's discomfort or unwillingness toward the use of technologies, such as MR, as an exacerbation of the digital use divide separating the learners into two groups. One group actively utilizes technology transforming the learning, and the second group implements technology in a passive way, such as completing mundane tasks like multiple-choice tests and assignments in an electronic format. Therefore, lowering barriers and faculty resistance to incorporating MR learning experiences into online engineering courses serves an important pedagogical and learning equity purpose. Quality instructional design support is critical for helping faculty overcome perceived barriers for incorporating MR learning experiences into their online and hybrid courses.

3. MR IN EDUCATIONAL CONTEXTS

MR is growing in popularity for face-to-face, online, and hybrid course development because of its ability to combine the real world and virtual world for the learner in powerful ways. The 2020 COVID-19 pandemic protocols requiring social distancing and remote learning alternatives have again emphasized technologies as beneficial to the learning environment. MR empowers students to view, touch, and manipulate both real and virtual objects in immersive environments. It allows them to interact with abstract concepts and complex scenarios in a low-stake fashion to “learn by doing,” sometimes in a collaborative experience with their classmates. These types of virtual environments offer great flexibility, agility, and customization to meet diverse learner needs. The approach enables the learner to safely encounter scenarios reflective of trial and error, failure, and success with a new degree of security and exploration. Learning is enhanced through the inclusion of the real world, as well as sounds and other natural user interfaces (Juraschek et al., 2018). The relationships and potential applications are highlighted in Figure 1.

FIG. 1: Potential application fields of MR

Kumaravel et al. (2019) noted learners and faculty are frequently limited by communication bandwidth ranges and interactivity. Rigid configurations could be lessened through a bidirectional approach utilizing video, audio, and spatial capture. Engagement could be further enhanced by permitting the learner to discover and annotate the remote and local environment, while reflecting and providing feedback about their experience and the work of peers (Lin et al., 2014; Kumaravel et al., 2019). Moreover, a differentiated and personalized teaching approach that considers learners holistically could be enhanced through the inclusion of technology (Tumkor, 2018). Educators need additional capacity to invite learners to engage in new and transformational learning experiences utilizing technology (U.S. DOE, 2017).

4. HOW MR TRANSFORMS LEARNING

MR creates an experience for learners where real and digital worlds merge and interact, creating new environments and visualizations in real time. MR is considered an enhancement of VR and AR environments. This relationship is referred to as interdisciplinary technology (Descalu et al., 2014). The MR environment transforms learning in several ways through multisensory interaction and object visualization, including scaffolding of the learner's transitional understanding from abstract to concrete concepts, low-stakes scenario-based/situational learning, and real-time collaboration opportunities.

4.1 Multi-Sensory Interaction and Object Visualization in MR

A meta-analysis of 16 studies in virtual mixed reality (VMR) found an improvement of task mastery requiring spatial and visual memory, observation, and emotional responses in stressful environments and situations. Students have reported a higher level of engagement and improved learning outcomes through the utilization of VMR. Potential misuse of VMR is derived from the overuse of stimuli, such as colors, shapes, and data inputs, which increase the cognitive load of the learner. Some users experienced motion sickness and dizziness, which distracted them from learning. The most notable benefit of VMR may be the inclusion of all learners, particularly those with special needs (Morimoto & Ponton, 2019).

In chemical engineering, using an AR cubic-marking tool and a tracking software known as augmented chemical interactions help learners visualize concepts such as chemical properties, spatial relations, and direct manipulation of three-dimensional (3D) interaction. These tools increase learners' understanding of the abstracts occurring within a chemical reaction. This revelation provides a much deeper understanding of chemical reactions than the two-dimensional (2D) representations seen in textbooks and molecular formulas. It also elucidates the learner's perspective of prototyping, developing, and understanding new chemical molecules (Cibulka & Giannoumis, 2017). Object visualization ability is greatly enhanced using MR, as discussed in the abovementioned example, and is an essential skill in various engineering settings.

4.2 Scaffolding of the Learner's Transition of the Abstract to Concrete

MR objects and environments support scaffolding and the transition from learning abstract to concrete (Quarles et al., 2009). Object visualization, coupled with immediate assistance with directions, instructions, and critical knowledge, can be employed in creating a personalized tool to assist object visualization. This attribute is possibly due to the various points of immersion in MR (Tumkor, 2018). Examples indicate that learner motivation and engagement increase if the learner can change the distance of an object, adjust the lens, and receive real-time data. Studying the covalent bonding of molecules in chemistry, DNA structure in biology, electron movements during current flow in electronics, open and closed-loop analysis for control systems, transistor action for electronic devices, and visualization of memory while programming in computer languages are only a few examples.

The development of automobiles is one concept example that requires scaffolds, both abstract and concrete representations of topics. Engineers designing automobiles use abstract representations to understand and analyze the internal dynamics of an engine. The automotive production process is then connected to concrete or tangible representations, such as driving the vehicle to test for user safety (Quarles et al., 2009). Through the use of visual analysis and tangible testing, engineers are able to thoroughly understand topics using MR in the labs, field, and classroom.

4.3 Low-Stakes Scenario-Based/Situated Learning

Juraschek et al. (2018) asserted learners are empowered to take control of learning, pacing, retrieval, and sequencing through MR learner-centered approaches. When gaming elements, such as levels, are created with a mastery threshold for advancement, learners can advance at their own pace in a low-stakes environment (Pelland et al., 2020; Juraschek et al., 2018). Review and retrieval of critical knowledge elements are available to the learner at their specified frequency. Adding the gamification elements, learners can be motivated through competition. Recognizing the value of cooperative learning, early adopters of this approach created the “software hut game,” which asked learners to develop software and analyze the submittal of other teams for improvement suggestions (Horning & Wortman, 1977).

Additionally, in a study of eight chemical engineering students, a lab activity was designed to help undergraduate students better understand reactor engineering and kinetics (Barrett et al., 2018). Each of the eight students had previously passed a junior-level chemical reactor design course, and their instructor identified them as competent in the course content. The quick feedback of MR provides learners with low-stakes opportunities to engage and explore. Because of the low-stakes time and energy level required, learners using MR initiated more cycles of inquiry than learners not using MR. This condition resulted in more exploration and interpretation. MR provided a “safety net” where failures could be explored and presented pathways to the discovery of multiple solutions. This interpretation and exploration helps learners improve understanding and build intuition, which aids in the tackling of more complex and rigorous engineering problems (Barrett et al., 2018).

4.4 Collaboration Opportunities

An environment employing an augmented learning design tool, such as gaming, assists teachers in conceptualizing and creating collaborative learning experiences (Descalu et al., 2014; Pozzi et al., 2017). Dascalu et al. (2014) noted that mobile MR applications offer a new facet of the learning experience supporting collaboration, which enhances situated learning, high interactivity, and increased engagement. The transaction of knowledge has become embedded in the rote and sometimes mundane daily activities of the learners frequently engaged within the workforce. Juraschek et al. (2018) concluded MR offers an uncommon immediacy since collaboration tools for interactive visualization engage learners in real time.

MR allows for collaboration between remote sites to solve tasks. In a case study about collaborative task solving between remote sites, a shared virtual and remote laboratory for electro-pneumatics based on Hyper-Bond technology was used (Müller et al., 2007). In this study, MR allowed a remote laboratory workbench to be paired with a local virtual workbench and vice versa. The virtual workbench software allows access to multiple users simultaneously. Learners at different locations can solve tasks together at their own virtual workbench. Each individual can then send the results to the physical lab workbench to test their common solution (Müller et al., 2007). This example demonstrates that MR brings new collaboration opportunities previously not possible with the potential to improve communication skills. As an added benefit, learners are not required to be geographically and collectively tethered in one location to conduct team labs.

5. HOW MR BENEFITS LEARNING

Furthermore, beyond investigating the MR transformation of learning experiences, several researchers have studied the benefits that technology can have on learning itself. The studies explored improved spatial skills, enhanced content and concept knowledge, improved focus, interest, and motivation, enhanced opportunities for learners with disabilities, increased application accuracy and repeatability, scaffolding for diverse students, improved safety with simulated lab environments, and decreased costs associated with engineering education.

5.1 Improved Spatial Skills

Juraschek et al. (2018) recognized MR's ability to extend the temporal, spatial, and functional scope of learning experiences. Learners can implement visual data with multiple spatial contexts, including actual machines, factories, processes, and systems. The learner is called to action coupled with digital augmentation. Deeper levels of transparency can reveal the actual inner workings of a machine. This unique ability enables the learner to view concepts, parts, and elements within the physical object that would usually be hidden from the human eye. Researchers studied the design of tools and methodologies for improving spatial abilities with multimedia exercises in reasoning and geometric transformations. Through the employment of MR tools, an improvement in learner retention by recognizing 3D objects from different angles, object position, and relation to other objects resulted in the study (Juraschek et al., 2018; Martin-Gutierrez et al., 2011).

For example, Tumkor's study of 217 students over the course of three years in engineering design and graphics courses was performed using various immersion levels of MR applications to aid object visualization of engineering drawings. For this subject matter, learning outcomes included a student's application of various visualization and graphical capabilities applied to geometry problems. After a traditional content lecture, assignments were made regarding definitions, procedures, and descriptive geometric problems' approaches. These assignments utilized spatial skills to solve problems, such as generating a digital model of a projected view into CAD software. The results of this study found that through the use of MR, the spatial visualization skills of learners without any CAD experience significantly improved. This further demonstrated that MR tools can be implemented into engineering drawing courses, particularly if the personalization of large classes is needed (Tumkor, 2018). Regardless of the level of mastery, MR empowers learners to develop spatial skills beyond their current understanding and use.

5.2 Improved Content and Concept Knowledge (Abstract to Concrete Understanding)

Many foundational concepts of engineering are abstract in nature. MR encourages dynamic contact and interactions with models (Tumkor, 2018). Students of art have often been expected to observe and act as a spectator. Yan & Chan (2019) studied the integration of VR in art courses by engaging learners in a sensory environment, participation in the virtual world, and unconscious engagement with inanimate objects of art. Learners become active participants through a dynamic change and the employment of MR tools (Tumkor, 2018; Yan & Chen, 2019).

Employee training time can be greatly reduced. For example, Boeing employees utilizing AR were able to assemble aircraft wing sections in 35% less time than those training by traditional methods (Zucchi et al., 2020). Porter & Heppelmann (2017) noted the workers able to perform this task correctly on the first attempt increased by 90%.

In another study (Frank & Kapila, 2017), 75 mechanical and aerospace engineering students at the NYU Tandon School of Engineering joined together in an educational platform using students' mobile devices directly connected with a laboratory testbed. This study found that participants significantly improved their content knowledge level in several topic areas of dynamic systems and control as a result of the MR learning environment (Frank & Kapila, 2017). A portion of this increased concept understanding is derived from the fact that students were already trained and comfortable in using their mobile device tools. This prior knowledge allowed them to focus on learning the desired topics instead of familiarizing themselves with new devices and navigation commonly used in regular laboratories.

5.3 Improved Focus and Interest

Dascalu et al. (2014) noted improved focus, increased interest, and motivation frequently resulted from MR encounters. Although robotics is a popular topic area, not all learners will remain engaged. For example, learners in one problem-solving approach incorporated building and programming robots. Learner's engagement increased not only in learning concepts but also in identifying the complexities of those concepts (Dascalu et al., 2014; Doswell & Mosley, 2006).

In a study of 20 randomized STEM professionals from an academic, biomedical research center at Harvard Medical School (Geller et al., 2021), the effectiveness of MR technology for Biomedical Engineering and Immunology disciplines was tested. According to Geller et al. (2021), participants learning about bioengineering utilizing MR was more engaged and found more enjoyment in their learning than those using only pdfs. The STEM professionals in the MR environment felt more strongly that they would continue to read on. This showed that MR environments improve the focus of students and interest in engineering topics (Geller et al., 2021). If students are more interested in engineering topics, it encourages them to explore their interests and increases their skill set before entering the workforce.

5.4 Improved Opportunities for Learners with Disabilities

MR, when properly employed, can provide a possible solution for learners with disabilities (Dascalu et al., 2014). In a meta-analysis study by Morimoto & Ponton (2019), autistic learners responded favorably to VMR in courses. The researchers noted a possible correlation between the positive response and the technology's inclusiveness of special needs learners. Through customized and intentional design efforts, MR places the learners in a personalized and immersive learning experience (Dascalu et al., 2014; Doswell & Mosley, 2006). The level of personalization, or differentiation, can address the specific challenges, disabilities, and preferences of the individual. For example, learners requiring specific colors due to color blindness can perform functions previously impossible. Tasks requiring all senses (sight, hearing, touch, smell) can be modified with braille, special lighting, auditory volumes and enhancements, language translations, and similar accommodations to promote learner success (Dascalu et al., 2014; Morimoto & Ponton, 2019). The inclusion and application of the Universal Design for Learning approach can be expanded to new levels.

Despite spatial ability being an important factor affecting students' learning outcomes, the use of MR technology as a supplemental enhancement can help low spatial ability students to reduce the gap between them and more advanced students, which allows them to obtain better learning results. For example, in a study of 80 students, the 3D-animated MR enhancements improved the grades of students with low spatial abilities. The scores of students with lower spatial abilities engaging in MR assignments were greatly improved. This mastery in skillset improves motivation and reduces attrition (Weng et al., 2018). Using MR in engineering education can help level the playing field for students with disabilities since their improvement in spatial abilities was greater than students without disabilities.

5.5 Increased Application Accuracy and Repeatability

A multitude of controls can be placed on the MR learning experience. Time limits, pressure tests, gaming levels, and complexity enhancement through machine learning can address a variety of educational needs. According to Dunston & Wang (2005), learners can benefit by utilizing specific points and phases of project development, evaluation, and inspection to pinpoint deficiencies in a process. This advantage is critical to reflection and growth (Dunston & Wang, 2005; Kolb, 1984).

In holographic design, fabrication and the assembly and analysis of woven steel structures employing traditional 2D structural representation can introduce inefficiencies, such as human error, in architectural experimentation. A three-day design-build workshop was performed by Frank & Kapila (2017) using unskilled construction teams. The effect of MR in these environments was explored in specific ways. For example, the Free-D was employed in the learning experience. This system was developed at the Massachusetts Institute of Technology as a hardware and software system that enlists a heads-up display to visualize an in-place digital 3D model for subtractive fabrication with a handheld computer numerical control mill. In projects like the Free-D, MR decreases fabrication time and increases the precision of nonuniform structures in the fields of architecture, engineering, and construction (Jahn et al., 2018). This decreased fabrication time creates a more efficient process that is now more repeatable and accurate.

5.6 Scaffolding for Diverse Students

MR can be tailored to meet the unique needs of global learners. The scaffolding approach creates multiple pathways for decision-making through collaboration and joint decisions resulting in a beneficial new level of transparency (Dunston & Wang, 2005; Martin-Gutierrez et al., 2011). Multiple tailored learning pathways can be created to clarify processes, methods, data, concepts, and systems. These areas can be enhanced not only visually but also with smell, sound, and animations (Juraschek et al., 2018; Yan & Chen, 2019).

As an example in the multimedia and architecture discipline, Birt & Cowling (2017) studied STEAM students learning spatial design at Bond University. Researchers reported that scaffolding was an essential component for learners utilizing MR in the course curricula. A segment of the students found it more challenging to employ MR due to a lack of comfort and familiarity the technology. Birt & Cowling (2017) emphasized that educational designers should build upon the learning from the visualization intervention by connecting the activity directly to the student's learning. The student should be assumed to have no prior technological skills and should receive detailed instructions to prime the student's knowledge (Birt & Cowling, 2017). Scaffolding can help students develop their skills based on their previous knowledge and skill level, which allows for a greater cognitive connection and increased inclusion in the learning component.

5.7 Improved Safety with Simulated Lab Environments

The safety of students is a critical component of the learning environment. Considering a diverse global setting, learners encounter a variety of situations that are not restricted by location, time, or frequency. Situations and environments that could be too costly, dangerous, complex, or requiring multiple attempts for skill mastery can be remedied through MR (Juraschek et al., 2018). Additionally, Morimoto & Ponton (2019) pointed out learners can experience hostile environments, adverse climate conditions, various time periods, partisan processes, and conflicts within the safety of a classroom. The flexibility is vast.

For example, there is an understanding that designing chemical reaction engineering labs is especially problematic while attempting to maintain tactical interaction. Sustaining this type of interaction is crucial because labs are an integral part of developing conceptual understanding, inquiry processes, and enthusiasm for certain topics in higher education. The difficulty that arises is commonly caused by the lack of safety protocols and practices with lab materials for chemical reaction engineering. In a study exploring MR use in a chemical engineering lab (Barrett et al., 2018), an MR design was found properly suited to simulate chemical reactors in real time and provide hands-on interactions. The MR simulation removes the danger of hazardous materials in these chemical reactors and simulates them at no physical risk to lab participants (Barrett et al., 2018). Barrett et al. (2018) asserted that MR should be imported into classroom environments because it allows learners to conduct labs that would not be safe otherwise. This allows learners to gain a deeper conceptual understanding of engineering concepts, because this increased safety allows any lab to be performed safely.

5.8 Decreased Costs Associated with Engineering Education

Mixing the technology with existing real-world objects creates a new level of reality and immersion. MR can reduce the learning cost by as much as 50% since the tangible learning objects already exist and do not require purchase (Juraschek et al., 2018). Remote laboratories permit the use of expensive, often large and immovable equipment by a large learner population. This reduces downtime and replicates the actual working environment. Furthermore, when information is provided in the appropriate time and space, the efficiency and usefulness of education are improved, resulting in the achievement of learning outcomes (Martin-Gutierrez et al., 2011; Yan & Chen, 2019).

MR learning environments have provided new solutions for engineering education that utilize students' personal devices. Applications have been created to transform student devices into assistive tools in engineering and science laboratories. These applications can often be downloaded at little to no cost. The MR approach to labs economizes in-lab activities and still provides hands-on experiences. This approach allows learners to continue working with real equipment without the cost of redundant laboratory-grade software and hardware (Frank & Kapila, 2017).

6. MOTIVATION

Motivation is defined as the learners' internal state, often identified as a want, need, or desire that initiates a behavioral or thought response (Kleinginna & Kleinginna, 1981). The presence of achievable goals can build levels of motivation (Elliot & Church 1997; Schunk 2016). Schunk (2016) emphasizes the importance of motivation as a key driver that engages students in their learning. The establishment of concise learning goals brings attention to the capabilities required to improve their skills. When the adult learner pursues his or her goals, the learner is empowered to learn and improve while generating a growth mindset (Yeager & Dweck, 2012). Ryan & Deci (2000) believe this new competence manifests through intrinsic motivation when the learner realizes the achievement as a reward in itself.

A recent survey (Essmiller et al., 2020) of 63 southern U.S. college students from a university in the southern U.S. showed no significant differences between motivation and self-efficacy as it relates to three selected activities: Robo-raid, Tutorial, and Freeplay. The Batra et al. (2020) mixed-methods study of 26 participants from chemistry and multiple engineering majors at a large university in the southwestern U.S. examined learners' motivation comparing a 15-min slides-based lecture to a 10-min VR-based lecture. Participants were surveyed after each lecture using Keller's ARCS model. The average motivation and positive change were experienced by 77% of the participants, while having a negative or null effect on 23% of the participants (Batra et al., 2020). A large percentage, 58%, of the learners believed that they were learning better using VR. Batra et al. (2020) further suggests that slides-based lectures could be enhanced with the supplemental use of VR-based manipulatives to improve learner motivation and satisfaction.

7. DESIGN OF THE MR EXPERIENCE

The design of MR learning experiences for engineering education requires a framework aligning the learning outcome of an engineering-related concept or skill to a real-world task or operation. Then, the task or operation is analyzed, broken down into composite tasks, which are matched to a specific MR technology (Dunston & Wang, 2005). This process is known as Technology-Task Mapping (Dunston & Wang, 2005).

Before the instructor can initiate the mapping process, engineering instructors need to identify the learning outcome their students will be required to meet. The concept or skill real-world contextual task is analyzed for influencing factors of media representation, input mechanism, output mechanism, and tracking technology (Dunston & Wang, 2005). Once the influencing factors are determined, the task can be matched to MR technology through device input/out taxonomies (Dunston & Wang, 2005; Pozzi et al., 2017).

Then, the production of the learning experience material (text, audio, images, video, etc.) and MR application begins. This phase of the process includes content creation or curation, design and development of situational context or scenarios, writing of instructions for the MR activity, etc. Next, the MR learning experience prototype is ready for review and evaluation of usability (Dunston & Wang, 2005).

Four learning theories comprise a pathway to utilizing MR in task-based curricula. Combining Fitts and Posner's three-stage model of motor skill acquisition, the Collins et al. (1987) model of cognitive apprenticeship and the Kolb (1984) experiential learning cycle encourage a learning experience comprised of modeling, coaching, live/recorded feedback, and reflection by the learner, peers, and faculty (Kumaravel et al., 2019). Even more impressive, the embodied cognition theory proposes an interwoven correlation between acting and thinking (Shapiro, 2019). When concepts are integrated, then their meaning is derived from a bodily experience, and Shapiro (2019) believed the collection of information is related to the body of origination. Embracing embedded cognition has been noted to improve communication skills, which could prove helpful for struggling engineers. The behavior is achieved not by changing the person in isolation, but rather by transitioning the dynamics of communication to reflect a negotiation and flow of conversational contributions over time (Shapiro, 2019). Options for the creation of the learning experience and corresponding technology are outlined in Figure 2.

FIG. 2: Off-the-shelf AR/VR systems and technology providers

Generally, designers and faculty may encounter challenges when developing courses. According to Martinez-Cerda et al. (2018), faculty may be unfamiliar and uncomfortable with the concept of collaboration. Conceptualizing the learning experience without identifying appropriate tools will have little impact on the learner (Tumkor, 2018; Pozzi et al., 2017). Tumkor (2018) noted learners in large classes do not normally have equal access to models and learning objects. When learners lack visualization skills, the problem is augmented negatively (Tumkor, 2018).

Müller & Ferreira (2003) noted the counterbalance between the novelty of MR and the unique realizations concerning guaranteed universality of outcomes and applications. Some practitioners may associate MR as a replacement for simulation or real experimentation. This posture should be considered too restrictive, since more significant pedagogical outcomes could benefit from MR's inclusion (Müller & Ferreira, 2003).

Underrepresented learner populations should be considered in the instructional design process. The recent advent of a methodology called critical digital pedagogy, rooted in social justice and concerned about the equitable distribution of power, creates a vast open educational network organically launched from the individual learners' perspective (Morris & Stommel, 2017; Müller & Ferreira, 2003; Tumkor, 2018). The method's collaborative nature and emphasis on community utilize the Internet and MR technology to remain open to the diversity of a global student body. This approach provides numerous opportunities to create learning experiences with the holistic aspects of the learner as the genesis. When learners are presented with MR coupled with the online and natural environment, instructional design enters a new space where distribution, partnership, participation, and learning become natural elements of the environment. McAuley et al. (2010) noted even if the learners attempted to disengage from the environment, the natural components of the environment would remain and learning could be a possibility.

Cowling & Birt (2018) cautioned designers to consider placing pedagogy before technology. The Sandoval Methodology places importance on embodied conjecture. Steps to this method are comprised of the analysis of a problem, developing solutions, testing and refining, and, finally, reflection. Faculty should consider pedagogy, or andragogy, as the foundational genesis while student success and a learner-centered focus are embraced as well (Cowling & Birt, 2018).

8. FUTURE WORK AND CONCLUSIONS

Reflecting on the review of the literature and current practices, the researchers propose a new MR Integration Course Design Process in concert with a new differentiated implementation model labeled the PuppetMaster Framework. The purpose is to analyze the impact of MR learning experiences on engineering education programs through the use of a specified design model and implementation of the differentiated application approach. The three variations of an MR learning experience would be designed for the topic of thermodynamics. The intended study would address 13 lessons experienced on a weekly basis of the 15-week course. The MR learning experience would be designed and developed to mimic real-world applications according to the following process.

9. MR INTEGRATION COURSE DESIGN PROCESS

Recognizing MR as a powerful tool for learning and teaching, Juraschek et al. (2018) noted an intentional process is necessary to implement the technology effectively. As previously mentioned, quality instructional design support is critical for the successful implementation of MR learning experiences and student success. A large Tier 1 research university is embarking on a pilot program to implement MR learning experiences into engineering courses methodically. The planning of this effort has involved consideration of gaining faculty buy-in and lowering barriers to integration. An initial step in the concerted effort is to develop a process for the college's instructional designers to collaborate with engineering faculty and to work with early adopters to illustrate successful MR integration.

The first step of the MR Integration Process (Figure 3) is a discovery meeting between the engineering faculty member and the instructional designer. In this meeting, the two collaborate to examine the course holistically in terms of learning outcomes and existing learning problems. Previous course learning analytics are reviewed to determine unmet learning objectives and outcomes of target populations. Learning misconceptions and gaps are analyzed, and abstract concepts related to those gaps are highlighted. Next, the designer and the instructor brainstorm ways in which the gaps could be addressed, including ways in which MR experiences could potentially augment learning outcomes through real-world application.

FIG. 3: MR integration process

After the initial meeting, the instructional designer creates a prototype for the MR learning experience and sends the prototype to the faculty member for review and feedback. Revisions to the prototype are made, and then the actual components of the MR experience are designed and developed, along with a user journey map to flush out potential procedural, technical, and conceptual barriers, as well as opportunities for gathering learning analytics and MR impact. The MR prototype is pilot-tested with a student focus group, and emerging technical and pedagogical issues and problems are documented. On the basis of the pilot test, the MR learning experience is further honed and retested until it is determined to be seamless for students and the faculty member (i.e., little to no support needed).

10. PUPPETMASTER IMPLEMENTATION MODEL

The PuppetMaster Implementation model is constructed with three variations: Stylized Expert, Jig-Saw Team, and Virtual Workstation. Each of the variations shares commonalities of an onsite practitioner-expert, remote student population, group collaboration, and MR elements. The variations and their applications are explained next and illustrated in Figure 4.

FIG. 4: PuppetMaster model variations

10.1 Stylized Expert

In this variation, an expert-practitioner is physically in a lab environment with an audiovisual connection to the remote learners. This individual can be a faculty member, a teaching or graduate assistant, or an industry expert. Real-time, two-way communication is prevalent throughout the experience. The remote learners utilize conferencing software, such as Zoom, to engage the practitioner-expert and each other. Learners direct the actions of the individual in the physical lab, which is located at an institution or actual industry/work location, while receiving immediate feedback and additional explanations of the learning. An additional caveat of this variation employs a blended learning experience consisting of a live physical lecture and a dialogue with an offsite lab component broadcast on a screen in the physical classroom and simultaneously in the online course environment.

10.2 Jig-Saw Team

Similar to the Stylized-Expert variation, an expert-practitioner is physically in a lab environment with an audiovisual connection to the remote learners. However, the identity of the practitioner-expert is different. This role is assumed by one of the remote learners who are physically present in a lab, which is located at an institution or actual industry/work location. The student physically in the lab is able to receive guidance and assistance from a faculty member in the lab. This variation is best suited for a long-term experiential learning event spanning multiple days, allowing each learner the opportunity to assume the role of practitioner-expert in the physical lab. Real-time, two-way communication is again prevalent throughout the experience. The remote learners may engage the practitioner-expert and each other using a conferencing software, such as Zoom. Learners direct the actions of the individual in the physical lab, while experiencing several physical components firsthand. Similarly, the extension of the Stylized Expert variation—employing a blended learning experience consisting of a live physical lecture and a dialogue with an offsite lab component broadcast on a screen in the physical classroom and simultaneously in the online course environment—can be extended for students. This would include a rotation of students between the physical lab and classroom, as well as the online environments.

10.3 Virtual Workstation

The third variation employs techniques and components of the previous two applications but adds a virtual workstation to the experience. An expert-practitioner is physically in a lab environment with an audiovisual connection to the remote learners. The identity of the practitioner-expert in the physical lab can be a faculty member, graduate or teaching assistant, an industry expert, or a student from the class. Real-time, two-way communication is again evident throughout the experience. The remote learners may engage the practitioner-expert and each other using a conferencing software, such as Zoom. Learners direct the actions of the individual in the physical lab with a workstation replicated from the physical lab version. The expert-practitioner utilizes a virtual workstation that manipulates the actual components of the physical lab.

11. PROPOSED RESEARCH MODEL

The analysis would be centered on the following three research questions:

1. Is there a difference in student performance between students in courses with MR versus students in the same courses without MR?
2. Is there a difference in student performance between students in courses influenced by the MR Integration Course Design Process versus students in courses not influenced by this deliberate instructional design process?
3. How do the students perceive their efficacy in courses with MR experiences?

The proposed study would take a quasi-experimental, nonequivalent group design to evaluate student performance in different modes of course delivery and the effectiveness of deliberately designed courses versus courses offered void of the design process. Student efficacy of these two experiences would be compared considering metacognitive factors on student performance.

Question 1 would examine student performance equivalency among the two versions of the thermodynamic courses. A comparison of grade distributions across the two groups would answer this question. A one-tailed t-test would be utilized to compare the grade distributions across the course pairs.

Likewise, question 2 would examine student performance equivalency between courses based on their design processes. To answer this question, a comparison of grade distributions between the two groups (MRICD versus non-MRICD) would be conducted. Researchers would use a one-tailed t-test to compare the grade distributions across the course pairs.

Altbach et al. (2009) recognized that grades are an imperfect indicator of learning. Research question 3 would examine the perception of self-efficacy by students in MR-enhanced courses and those without the technology elements. A metacognitive test utilizing questions derived from the Self-Assessment of Learning Gains (SALG) inventory would be provided to all learners in the sample. Reasoning that learners scoring high on the self-regulatory scale will have better grades, Pearson's R would be chosen as an indicator of the relationships between the scales.

The selected topic area is thermodynamics. Thermodynamics is commonly taken by biological and agricultural, chemical, civil, environmental, industrial, material science, mechanical, nuclear, ocean, and petroleum engineering students as an undergraduate course. This course contains some of the more challenging topics to understand in the engineering curriculum. The inclusion of laboratory demonstrations could help students understand fundamental concepts such as heat, work, entropy, and energy. Laboratory experiments could demonstrate lessons about conservation of energy and types of thermodynamic cycles, and thermal efficiency. These supplemental lab demonstrations could be crucial to students' knowledge and understanding of thermodynamics as engineers because of the nature of this course. An illustration of the models and their companion topics is found in Figure 5.

FIG. 5: Treatment model and topics

The physical motivations and connections of thermodynamics are difficult to grasp, unlike other engineering topics such as mechanics, which is evident in everyday life. Labs could allow learners the opportunity to understand the missing physical connections and motivations behind thermodynamics and could improve their performance in the classroom.

The proposed methodology of the study would include a SALG pre-assessment followed by the online course content. Then, at the conclusion of the MR experience, each student would receive a SALG post-assessment questionnaire. Quantitative data from each of the pre- and post-assessments will be analyzed along with a comparative analysis of learning metrics from companion courses of the same subject without the MR treatment. This study will look at the impact of MR learning experiences on pre- and post-assessment learning outcomes compared to course offerings of the same subject that did not contain an MR learning experience. The grades in each of the three model variations (Stylized Expert, Jig-Saw, and Virtual Workstation) and courses without the MR treatment will be compared. Then, through factor analysis and an analysis of variance, the quantitative data (grades) will be analyzed from the student pre- and post-tests to measure learning using grades as a proxy. This will serve as an indicator of the student's competency in successfully achieving the learning outcomes. Learners' self-efficacy would be measured through the SALG with a factor analysis of the responses. Finally, through the collection of learner demographics, the intended study would examine the target population data with previous learning analytics from comparable target populations over the prior three semesters. The data could be useful in learning if MR can be utilized as a tool for equity.

12. SUMMARY

Although the intended study focuses on thermodynamics, value can also be realized in nuclear engineering programs. Across the country, 25 universities have nuclear reactors at their schools. These reactors provide valuable experiences to learners studying nuclear engineering, because they can learn how a nuclear reactor works in real life. However, many universities that offer a nuclear engineering degree do not have a university nuclear reactor for the students' use. Through the use of the PuppetMaster implementation model, universities with a reactor can use the three different variations of the model to assist learners at academic or work locations without a reactor. In this way, those learners can be provided the opportunity to experience operating nuclear reactors in the real world rather than only using computer-simulated reactors.

This study addressed the current utilization of AR, VR, and MR in engineering education. Of equal importance is the implementation of these technologies to transform the learning environments for decades to come. MR provides a viable solution when posed with potential challenges and catastrophic events, such as pandemics, which could cause an immediate pivot of teaching and learning, MR provides a realistic alternative as well as a stable format for differentiation in online learning. A careful reflection and consideration of this work will assist engineering programs in determining the impact that MR learning experiences have on course design. Educators who readily embrace and adopt these technologies will have an edge in providing relevant, innovative, culturally relevant, and impactful learning for a global audience.