Jungwoo Ryoo Kurt Winkelmann *Editors*

Innovative Learning Environments in STEM Higher Education Opportunities, Challenges, and Looking Forward

# **SpringerBriefs in Statistics**

SpringerBriefs present concise summaries of cutting-edge research and practical applications across a wide spectrum of felds. Featuring compact volumes of 50 to 125 pages, the series covers a range of content from professional to academic. Typical topics might include:


SpringerBriefs in Statistics showcase emerging theory, empirical research, and practical application in Statistics from a global author community.

SpringerBriefs are characterized by fast, global electronic dissemination, standard publishing contracts, standardized manuscript preparation and formatting guidelines, and expedited production schedules.

More information about this series at http://www.springer.com/series/8921

Jungwoo Ryoo • Kurt Winkelmann Editors

# Innovative Learning Environments in STEM Higher Education

Opportunities, Challenges, and Looking Forward

*Editors* Jungwoo Ryoo Pennsylvania State University Altoona, PA, USA

Kurt Winkelmann Valdosta State University Valdosta, GA, USA

This book is an open access publication.

ISSN 2191-544X ISSN 2191-5458 (electronic) SpringerBriefs in Statistics ISBN 978-3-030-58947-9 ISBN 978-3-030-58948-6 (eBook) https://doi.org/10.1007/978-3-030-58948-6

© The Editor(s) (if applicable) and the Author(s) 2021

**Open Access** This book is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this book are included in the book's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the book's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specifc statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use.

The publisher, the authors, and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affliations.

This Springer imprint is published by the registered company Springer Nature Switzerland AG The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland

*For my parents, Kyung-sik and Jeong-Kang, wife, Hyeseon, and children, Hojin, Eugene, and Youngjin*

Jungwoo Ryoo

*For my wife Catherine and daughter Mary* Kurt Winkelmann

# **Foreword**

Imagine a world in which innovation is allowed to drive the educational learning environment. Whether it be the ability to reach out around the globe and have virtual access to the experts or to be able to feel like you are immersed into the actual environment, such a future can exist. It is important to keep these possibilities in mind as we continue to explore in-depth the concepts supporting Innovative Learning Environments.

My initial encounter with the authors of this book stemmed from an invitation to be the opening presenter for a workshop on Innovative Learning Environments. As I prepared the charts to present, I pondered on what would be the appropriate message to share with the participants of this workshop, individuals who have spent a lifetime grounded in integrating emerging technologies and innovations into learning spaces. In preparing the presentation, I thought it is best to share my experiences (as an educator, engineer, and STEM motivator) about learning environments that craved for change, that craved for fresh thought and new perspective, and environments that craved for innovation. The best way I could think of to impact and motivate the participants was to allow the voices of those students I had encountered to speak through me.

One story was that of Patrick who enrolled in my Summer Science Academy disinterested in science. His mother was trying to expose him to STEM to spark an interest while also seeking to improve his math and science skills. However, Patrick resisted applying himself. He hated math and science because (in his eyes) it was too abstract, boring, irrelevant, or as he says "stupid." Once I was able to create activities that simply connected science to everything in his life (his favorite foods, his clothes, his tennis shoes, his bicycle, his video games, etc.), Patrick adjusted his perspective and opened his mind to learning.

There are countless other stories such as the following: the stories of students who resisted group projects because of the frustrations from slackers or the inability to see the value of said projects; or the stories of STEM teachers who were struggling with fnding engaging and cohesive content and strategies designed to make learning fun; or even my own disappointment in knowing that the same challenges I faced in the engineering learning environment in 1982 were exactly the same as those faced by my daughter in 2017; and countless others who complained or were disinterested because the learning environment was simply too mundane, too disconnected from the real world, and lacked the benefts of modern technology. What I found in all those experiences was that the students/teachers were unable to connect what they learned/taught in class in a way that allowed them to value learning, to experience learning, to apply the knowledge gained to feed innovation. So, imagine how elated I was to be able to interact and engage with the participants, who are the experts and contributors of this book, as they worked so diligently to create this blueprint for the future of Innovative Learning Environments (ILE). Technologies included in that blueprint are personalized and adaptive learning, multimodal learning formats, extended cross reality, and artifcial intelligence and machine learning. These technologies are constantly pushing the envelope in their application in learning environments.

We, as educators and scientists, have to remain vigilant. It is not just about how we learn but also about how that learning impacts the educational environment, the work environment, and moreover the world globally. As a retired NASA engineer, I can envision students learning about the solar system and the stars while virtually existing in the world of outer space. I can also imagine them being able to virtually tour space vehicles and learn about the function of the many instruments and controls on the vehicle. I can even anticipate students being virtually present to learn about the sequences and operating procedures involved in preparing a vehicle for fight real time and so many other scenarios that positively impact learning.

With the recent invasion of the COVID 19 virus causing a push for more virtual learning spaces, the utility of Innovative Learning Environments can no longer be questioned or delayed. We are being thrust into this direction of virtual learning and cannot continue educating our students using traditional methods only. I am convinced that Innovative Learning Environments offer the opportunity for learning to be ever-reaching, richer, deeper, more expansive, and can serve to meet the needs of students and teachers in the future.

We already know the impact of learning experiences that are informative, engaging, and fun. We know that such environments are the foundations for creativity, productivity, and growth and allow learners to feel like they are a creative part of the

world that surrounds them. The building blocks for such an environment are what is detailed in this book and will have an impact on STEM learning for years to come.

NASA, STEM Virginia Cook Tickles Washington, DC, USA

# **Acknowledgments**

The editors of this book set out to explore the boundless possibilities available through Innovative Learning Environments (ILEs) in STEM Education in 2018. One major goal of ours was to develop a community of like-minded scholars and practitioners along the process. We are closer to that goal as of this writing, and this book is an embodiment of what such a community can accomplish when inspired and nurtured through a project like X-FILEs (sites.psu.edu/xfles/). We feel so honored and proud to be leading such a talented group of enthusiastic and passionate educators, administrators, and industry leaders.

X-FILEs organizers: from left to right, Lawrence Ragan, Lorraine Ramirez, Kurt Winkelmann, and Jungwoo Ryoo

This work was supported in part by the US National Science Foundation under Exploring the Future of Innovative Learning Environments in STEM Grant numbers 1848609 and 1848612. Any opinions, fndings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily refect the views of the National Science Foundation (NSF).

Publication of this book was funded in part by the Open Access Subvention Fund and the John H. Evans Library at the Florida Institute of Technology.

In addition to the chapter authors, many others contributed to this book project. These unoffcial contributors include those who participated as X-FILEs webinar and on-site workshop attendees, reviewers, and proofreaders.

We want to thank each one of the people below for their help with making this book possible.

### **Personalized and Adaptive Learning Chapter Contributors**

Ann Lindbloom, University of Kansas, Lawrence, KS, USA Dongho Kim, University of Florida, Gainesville, FL, USA

### **Multimodal Learning Chapter Contributors**

Sharon Lynn Chu, Department of Computer and Information Science and Engineering, University of Florida, Gainesville, FL, USA

### **Cross Reality Chapter Contributors**

Douglas North Cook, Assistant Professor of Immersive Media, Chatham University, Pittsburgh, PA, USA Lindsey Spalding, STEM Leadership Consultant

### **Artifcial Intelligence/Machine Learning Chapter Contributors**

Ronald Rusay, Chemistry Department, Diablo Valley College, Pleasant Hill, CA, USA Joe Russo, Learning Management and Curriculum Architect, Penn Medicine, University of Pennsylvania Health System, Philadelphia, PA, USA

### **Workshop Attendees**


We wish to acknowledge and thank Dr. Lorraine Ramirez Villarin for her help organizing the X-FILEs workshop and the X-FILEs Jam.

**Jungwoo Ryoo**, Ph.D., Professor of Information Sciences and Technology, Penn State University

**Kurt Winkelmann**, Ph.D., Professor of Chemistry, Valdosta State University

# **Contents**


# **Introduction**

#### **Jungwoo Ryoo and Kurt Winkelmann**

### **1 X-FILEs Approach**

The practice of educating students in college-level science, technology, engineering, and math (STEM) subjects is infuenced by many factors, including education research, governmental and school policies, fnancial considerations, technology limitations, and acceptance of innovations by faculty and students. Working together, stakeholders in STEM higher education must fnd creative ways to address the increasing need for a diverse US workforce with a strong STEM background (President's Council of Advisors on Science and Technology 2012) and the need for a more STEM-literate general population (National Research Council 2012).

In order to help researchers, developers, educators, and other stakeholders fnd these creative solutions, we conducted the eXploring the Future of Innovative Learning Environments (X-FILEs) project: a series of agenda-setting, interactive, online activities followed by a face-to-face workshop and a writing initiative which led to this book. Our participants considered the following question:

*What are the near-term and longer-term impacts, opportunities, challenges, and future research initiatives related to the development and implementation of innovative learning environments (ILEs) in higher education STEM disciplines?*

This project included three components: (1) Interactive, online discussions prior to the workshop introduced participants to various innovative learning environments (ILEs) and solicited feedback from a wide range of stakeholders. These discussions formed the basis of the workshop agenda. (2) Participants at our 2-day,

J. Ryoo (\*)

K. Winkelmann Valdosta State University, Valdosta, GA, USA e-mail: kwinkelmann@valdosta.edu

© The Author(s) 2021 1

J. Ryoo, K. Winkelmann (eds.), *Innovative Learning Environments in STEM Higher Education*, SpringerBriefs in Statistics, https://doi.org/10.1007/978-3-030-58948-6\_1

The Pennsylvania State University, Altoona, PA, USA e-mail: jryoo@psu.edu

face-to-face workshop engaged in creative activities that helped them envision how ILEs can transform STEM higher education. (3) Project leaders (editors of this book) and invited writers continued to synthesize the ideas, questions, challenges, and solutions proposed during the workshop into this book.

The workshop participants discussed and explored four main ILE categories: personalized and adaptive learning, multimodal learning, cross reality (XR), and artifcial intelligence (AI) and machine learning (ML). While a particular technology application may fall within one or more of these four categories listed in Table 1, each category is broader than any single educational innovation. Therefore, the usefulness and relevance of our book will endure beyond the shelf life of any current technology application.

We also solicited the views and ideas of subject matter experts and other stakeholders in STEM higher education, including those from traditional 4-year colleges and universities, online programs, and technical and community colleges.

In order to frame the examination of where and how the ILE category may impact the students' experience, we address the following framing questions (FQ) related to the four ILE categories. In particular, we examined the questions from the context of impact on the various aspects of teaching and learning: content presentation, interactions and communications, learner activities, assessment, and co-curricular activities.

FQ1. What are the *opportunities for gains* in the adoption of the ILE category?

FQ2. What *challenges/barriers* exist that may inhibit the impact of the ILE category on the creation of ILEs by 2026?

FQ3. What *implementation strategies* may be used to advance the ILE category in order to realize the impact by 2026?

FQ4. What *research questions* remain to be addressed in order to optimize the impact of the ILE category?

Below are our working defnitions of the four domains of teaching and learning:


**Table 1** ILE categories and sample technologies and pedagogies

*Content Presentation and Instruction*: The substance or content of a course refers to the information about the subject domain that will be conveyed to the learner. The breadth and depth of the course content are selected to meet the instructional needs of the learner in order to achieve the course objectives.

*Interactions and Communications*: In the teaching and learning context, interactions may be defned as the exchange of information between class participants. Interactions can be between one to many (teacher to class or student to students), one to one (teacher to individual student or student to student), or group-based (teacher to group or group to group).

Teaching, the act of guiding, illuminating, and explaining the course concepts, facts, and experiences to the learner, is principally a function of interactions and communications. Multiple methods are used to teach course content, including the delivery of lectures, readings, viewings, discussions, and hands-on activities. The media formats used to teach the course content are also varied and may include speech, text, video, audio, and hands-on exercises.

*Learner Activities*: The student activities are methods used to engage the learner with the course objectives. These engagement methods require the learner to apply, practice, and further seek mastery of the course domain. Methods may include assignments such as practice problems, responding to questions, developing or constructing output that represents an understanding of the course concepts. Methods may also include hands-on activities such as lab exercises and experiments, and feld studies.

*Assessment*: Assessing learners, progress in reaching the defned course objectives can be measured as the material is being delivered (formative evaluation) or at critical milestone markers (summative evaluation). Assessment strategies are viewed as instructional in nature as they may reveal to both the teacher and the student areas of needed improvement towards achieving the expressed learning outcomes.

*Co-curricular Activities*: The learner interacts with the course materials in many ways, internally to the course offering and externally through a variety of methods such as cultural and civic events, educational programs, internship and job experiences and even student-spirit and athletic events. The student experience consists of a wide range of interactions that may consist of athletic, academic, social, and service dimensions. Providing the student with access to a range of these dimensions is a critical link in the overall quality of the student experience.

For each framing question, we consider how the ILE category impacts the presentation of content, the interactions and communications between and among learners and instructors, and the methods and techniques used to conduct assessment of student progress. These four framing questions, combined with the four ILE categories, create a 4x4x5 3D matrix shown in Table 2.

In the following chapters, we will present our in-depth analysis of each of these combinations.


**Table 2** FQ-ILE category matrix

### **2 X-FILEs Workshop Activities**

Drs. Ryoo, Ragan, and Winkelmann hosted three online meetings in the months preceding the 2018 workshop. These introduced the technology categories and solicited feedback for each framing question from participants working in small groups. The online discussions were structured to stimulate broad ideas and visions of the potential for ILEs impacting STEM higher education in 2026. Participants included education administrators, instructional designers, STEM researchers and faculty, and industry experts. Some attended the subsequent workshop, while others could only attend the online meetings. A total of 100 people attended the online meetings.

Organizers prepared a list of potential workshop attendees based on members of their own professional networks, educators previously funded by NSF, online meeting attendees, and others who completed the online registration. Organizers selected a diverse group of attendees that included education researchers and practitioners in all four technology areas (Fig. 1). They adjusted the workshop's agenda and structure based on comments, ideas, and feedback from the online meeting participants.

The 2-day, face-to-face workshop in Melbourne, Florida, provided attendees a chance to collaboratively engage in creative activities that helped them exchange ideas and views on the use of ILEs to transform higher STEM education learning. The workshop commenced with a welcome dinner and two keynote talks. Dr. Virginia Tickles spoke about her professional life as a NASA engineer and her experiences as a strong advocate and promoter of STEM education. In the second talk, NSF DUE Program Offcer Dr. Pushpa Ramakrishna described the efforts of NSF to create a vision for the future of STEM education, to which this workshop was contributing.

Day 1 started with a "Lightning Strike" question session as an ice breaker activity. Resembling the World Café Method (Brown 2002), participants were encouraged to move between "question stations" that promoted discussions around the four ILE technology categories under study. Once this data was collected, a set of activities allowed for a deeper look of each technology category.

Dr. Larry Ragan, the workshop moderator, divided attendees into fve teams, one for each of the aspects of teaching and learning. Participants in each team explored the framing questions related to personalized and adaptive learning. A team lead and recorder compiled the team's thoughts and transcribed them in a pre-structured

**Fig. 1** X-FILEs organizers and participants

**Fig. 2** X-FILEs group activities

online document. At the conclusion of the discussion, all attendees reconvened and shared highlights of their discussions with each other. This same activity was repeated in the afternoon session for the second category, multimodal learning (Fig. 2).

Day 2 began with participants creating four narratives, each describing "A Day in the Life" of a higher education student in the year 2026. Stories were developed around one of these four assigned personas: (1) unconventional or alternative learner, (2) rural student attending a university fully remotely, (3) residential student, and (4) career-changing adult learner. These stories with "big ideas" were later shared, captured, and recorded.

To explore the last two technology categories, four posters for XR and four posters featuring AI and ML were created. Each contained one framing question along with a wheel depicting the fve aspects of teaching and learning. This gallery walk exercise (Carleton College 2018) encouraged all workshop participants to post ideas, reactions, and considerations within each section of the wheel. Teams then distilled and organized the contents of these idea sheets in order to reveal trends and themes and reduce redundant posts. Using this data, teams organized and structured a response for each of the aspects of teaching and learning. Teams reported results to other participants to foster comments, modifcations, and agreement.

This day culminated with a student-led discussion panel in order to gain insights into the student perspective of ILEs in STEM education in 2026. These four student panelists included a current high school junior, two undergraduate students, and a graduate student. Following a series of prepared questions, the workshop participants and panelists engaged in a conversation regarding the expectations and anticipations of the higher education experience in 2026.

Following the event, the workshop organizers invited participants to contribute to a white paper describing the workshop outcomes that later evolved into this book. When necessary, they invited other contributors not present at the workshop to help write and review chapters.

### **3 ILEs Addressed by X-FILEs Project**

Advances in educational technology must be coupled with a deep understanding of how students learn and retain information. Only then can developers create new, innovative ways for students to interact with their course materials that replace or augment traditional approaches inside and outside of the classroom. Although some of the ILEs discussed in this book are heavily dependent on technology, they are also part of sound pedagogical approaches that promote student-centered learning. Some major ILEs listed in Table 1 are briefy described below and may involve a physical space, a virtual space, or a combination of the two.

"Digital storytelling is the modern expression of the ancient art of storytelling" (Barrett 2006) and has become increasingly popular in recent years, as shown well in the examples of YouTube and podcasting. Digital stories are often in the form of multimedia movies that incorporate photos, animations, videos, soundtracks, texts, and narrations. College students are comfortable using diverse platforms and tools (e.g., Unity, OpenSimulator, Xtranormal, or Garry's Mod) to make, play, and share interactive digital stories (Pappas 2013). Many educators are already using digital storytelling in their classrooms to enhance the educational experience of their students. To be able to tell their own digital stories, students have to go through a rigorous process of researching a topic, developing a specifc point of view and insights, and fnally sharing them in creative ways. This experience is germane to one of the ultimate goals of higher education: producing independent learners.

The Internet of Things (IoT) is a growing feld of digital communications that allows objects to communicate with each other, creating smart environments that respond appropriately using data collected by ubiquitous sensors. It is expected that the IoT will have a signifcant impact on many economic sectors (Manyika et al. 2015). The IoT can impact education in two ways: frstly, students can access more data from interconnected physical objects, either nearby or in a remote location. This data provides students with a richer, more detailed understanding of their environment, and they can use the data to study real-world challenges (i.e., through problem-based learning) (Ali et al. 2017). Secondly, sensors can monitor a student's facial features and other characteristics. This data shows when a student is under stress or not fully engaged in the learning process (Farhan et al. 2018). A teacher could then call for a break or redirect the student's attention back to the activity.

Micro-credentials and digital badges acknowledge accomplishments and skill acquisition at a more granular level than college degrees. Many corporations and online programs already adopt micro-credentialing and badging. IBM offers digital badges for its data science training through its IBM Skills Gateway (IBM 2018). Their digital badges recognize the qualifcations of IBM's own employees as well as the public in general. edX, a nonproft online education consortium of higher education institutions, has introduced MicroMasters programs. Universities are also actively looking into ways to incorporate micro-credentialing and badging into their curriculum (Fain 2016). Some of the benefts include more explicit and organized guidance on students' academic achievements and higher motivation and engagement.

Online learning, in the form of massive open online courses (MOOCs), has existed for many years, but MOOCs remain an innovative method for learning due to their continuous incorporation of new technologies (e.g., wikis, credentialing, and personalized learning algorithms). MOOCs hold great promise for democratizing education, but this ILE has not reached its full potential. Only a fraction of science and engineering subjects are offered by MOOCs. Less than 10% of students enrolled in a MOOC actually fnish it (Lakshminarayanan and McBride 2015). Although MOOCs are available to groups who have been traditionally excluded from higher education, most users are employed, male, and live in countries of the developed world (Christensen et al. 2013). In a blended learning environment, the curriculum is presented in both an online format and in a traditional classroom space. For instance, students may complete the "lecture" portion of a chemistry class online but travel to campus to complete their laboratory experiments (Dalgarno et al. 2009).

Flipped classrooms are another example of blended learning. In a fipped classroom setting, students perform technology-rich exercises that prepare them for learner-centered activities during class. Pre-class activities can include videos, interactive computer-based simulations, and social media (Lundin et al. 2018). In-class activities require students to critically think about the subject matter and its applications (O'Flaherty et al. 2015). Flipped classrooms are growing in popularity, and they are used frequently to teach STEM subjects. Student acceptance varies; some enjoy their active role in the classroom, and others resist this format because it requires their greater preparation (McNally et al. 2017).

Game-based learning (GBL) leverages the entertaining experience of computer or noncomputer games to achieve an educational objective such as changing attitudes (Stewart et al. 2013) or improving knowledge in subjects such as math (Castellar et al. 2015) and physics (Hamari et al. 2016). Games may be designed with specifc learning objectives in mind, or they can be developed independently (e.g., a commercially available video game with no particular educational goal) that nonetheless can lead to users achieving desired cognitive or affective learning outcomes. Purposefully designed educational games are known as serious games. Their use has been shown to improve student learning (Clark et al. 2016; Vlachopoulos and Makri 2017), and best practices for studying their effectiveness are available. However, more rigorous assessment methods are necessary in order to clearly understand how GBL benefts students (All et al. 2016).

Virtual environments positively impact students' attitudes and learning (Hew and Cheung 2010; Kim and Baylor 2015; All et al. 2016; Winkelmann et al. 2017). Examples of virtual environments relevant to this book include virtual, augmented, and mixed realities and AI-enhanced virtual environments. Game-based learning environments may also have a virtual component.

Computer-based simulations and virtual lab experiments, such as PhET and Late Nite Labs, are effective supplements to many STEM courses (Moore et al. 2014; Pyatt and Sims 2012; Finkelstein et al. 2005; Reece and Butler 2017). Immersive virtual reality (VR) allows the user to experience virtual worlds more deeply by viewing the virtual world through stereoscopic visual displays within a headset. This blocks out other sensory distractions and provides a truly three-dimensional view of the virtual objects and other avatars within the VR program. Motion sensors track users' head and hand motions in real time to create an immersive experience in which users control their avatars and interact with each other in the virtual reality setting. One goal for VR-based education is for students to experience the psychomotor "hands-on" aspect of a real-world lab experience. VR technology is used or is being developed for aviation (Koglbauer et al. 2011; Yin et al. 2014), medicine (Buckley et al. 2012), and training in other felds (Kinateder et al. 2015). Virtual reality will provide students with more opportunities for learning both in and out of the classroom (Ma et al. 2014; Lakshminarayanan and McBride 2015). Augmented reality (AR) allows a user to view and interact with virtual objects layered over real objects when viewed through a headset or mobile device screen. It is a similar technology to VR, and its implementation is generating benefcial outcomes for students (Cai et al. 2014; Borrel and Fourches 2017). In a mixed reality (MR) environment, the virtual content and the real content coexist, and the virtual content responds to changes in the real environment. Collectively, these ILEs are referred to as cross reality (XR). Activities can be collaborative and inquiry-based, depending on how the virtual world is designed.

The use of smartphones and tablets for education is known as mobile learning or m-learning (Lakshminarayanan and McBride 2015). These devices enable students to participate in many other ILEs. For instance, smartphones are useful for collecting and processing IoT data. They provide a display screen for augmented reality devices and run educational game apps. Since mobile devices are now ubiquitous among college students, it is reasonable to expect students to have access to assigned digital media which they view outside of class, as is done in a fipped classroom setting.

Digital publishing refers to the dissemination of digital content, such as text, videos, animations, and interactive visualizations, to the public. Digital publishing has many advantages over its conventional counterpart. For one, it can incorporate diverse digital media into publications. One of the emerging trends in digital publishing is open and independent digital publishing of learning knowledge. For example, version-controlled git-authoring (github.com) and open educational resources (OER) are emerging technologies in digital publishing (OER Commons 2018). Many believe that OERs are the future of educational content due to their more engaging, customizable, and affordable nature.

Intelligent virtual environments that integrate artifcial intelligence techniques into the virtual environments may include XR and intelligent tutoring systems. For example, intelligent virtual reality systems (IVRS) incorporate AI algorithms into virtual reality (Aylett and Cavazza 2001). Advances in AI have allowed for the development of virtual environments that incorporate agents, eHealth-related devices, human actors, and emotions, projecting them virtually and managing the interaction between all the elements (Rincon et al. 2017). In these environments, the simulation and detection of human emotions can be used for the improvement of the decision-making processes of the developed entities. Machine learning occurs as the AI system adjusts its own algorithms in order to achieve greater success at its task. AI and ML have tremendous potential to enhance creative inquiry and informal learning, as well as adaptive learning through intelligent tutoring systems. One example is a web-based AI tutoring system in Tunisia that recognizes facial expressions as students progress through science experiments that they can access from anywhere (Adams Becker et al. 2016). Though most of the educational AI software is still in the development stages, advancing technologies could drastically alter the landscape of how students learn (Lynch 2016).

Stealth assessment is the evaluation of a learner's performance in a manner that is unobtrusive, using metrics embedded in the course material (Shute and Kim 2014). Students learn without realizing that they are being constantly monitored for their progress. Another beneft is the immediacy of feedback. The educational content can adapt to how well students are doing at a given moment. Technologies now allow educators to conduct stealth assessment without the concerns of costs or system performance. Educators can closely examine the learning process of individual learners in a group by observing a student's collaborations and contributions without disrupting the group.

### **4 Education Technology Trends Identifed Prior to the X-FILEs Project**

A discussion of the future use of ILEs in STEM education brings to mind a cautionary Danish proverb: It is diffcult to make predictions, especially about the future. Therefore, it is worthwhile to look back at past predictions in order to see how useful such predictions might be, which predictions were in fact credible, and how intervening events derailed expectations of other technologies.

A ready source of informed predictions is provided by the annual New Media Consortium (NMC)'s Horizon Report for the use of technology in higher education. In addition to identifying and analyzing challenges and trends in higher education (not just STEM education), the Horizon Report describes technologies that are poised for widespread adoption in the near, medium, and long term. The 2013 and 2014 Horizon reports (Johnson et al. 2013, 2014) predicted that 3D printing and wearable technologies in 2013 and quantifed self and virtual assistants in 2014 were emerging technologies that will be widely adopted in higher education within 4–5 years (2017–2019, during the time period of the X-FILEs project).

3D printing is the design and fabrication of three-dimensional objects by depositing successive layers of the printing material. A user frst creates or downloads a digital fle that the printer uses to build the desired object. Ceramics, metals, polymer composites containing metal particles or wood fber, and even paper are commercially available, but the most common printing materials are plastics (von Übel 2019). 3D printing is the most popular technology used in makerspaces—workshops where students and hobbyists collaborate, learn, and use low- and high-tech tools to create. Consumer-grade printers are inexpensive, and websites allow users to share designs. This technology and makerspaces in general are indeed becoming increasingly common in schools, as refected by the 2018 NMC Horizon Report which placed widespread adoption of makerspaces within a year (Adams Becker et al. 2018). Students can use 3D printing in many STEM activities for chemistry (Pinger et al. 2019), engineering (Chien et al. 2018), and biology (Gordy et al. 2020). A recent review article (Ford and Minshall 2019) describes the current state of this technology in education. The two most common ways of incorporating 3D printing in STEM education are rapid, inexpensive production of models that improve student learning and teach the skills associated with designing the virtual 3D model (e.g., CAD) and operating the 3D printer.

Wearable technology (wearables) is a category of the Internet of Things (IoT) devices that can incorporate other technologies: smartwatches, exercise tracking devices, AR glasses, VR headsets, smart hearing devices (hearables) that interact with virtual assistants or translate speech, and a growing number of proposed medical sensors that track a wearer's health conditions. Sensors within the wearable transmit data to a mobile computing device or to a remote cloud server for analysis and display for the wearer. This is desirable if the wearer wishes to improve quality of life and increase self-awareness. Real-time, passive, personal data collection is the basis for the quantifed self-movement noted in the 2014 NMC Horizon Report.

Wearables highlighted in this report include AR glasses and VR headsets. These technologies have shown the most growth in education during the past 5 years. AR and VR in the consumer entertainment industry increased students' acceptance, generated best practices for improving the student's virtual experience, and popularized development tools for creating educational VR experiences.

Potential wearable applications associated with the quantifed self have not advanced as rapidly due to a variety of limitations and challenges. These include the need for a continuous wireless connection to a user's phone and the diffculty of displaying information on small screens (Kemper 2019). Medical sensors must frst undergo rigorous study and review by healthcare regulatory bodies such as the US Food and Drug Administration (FDA). Collecting, transmitting, and analyzing a wearer's personal health data raise privacy concerns as well. It is not clear how these wearables would be used in educational settings, although students could use such information to improve their diet and sleep schedule, which could impact their ability to learn (Johnson et al. 2014).

Virtual assistants are now well known, thanks to Hey Google, Apple's Siri, and Amazon's Alexa. A user speaks (or types) to the virtual assistant using conversational language, which interprets the words and context and then gathers relevant information from databases or other apps. Virtual assistants use machine learning to improve their understanding of the user's speech and the context of questions. A particular brand of virtual assistant is designed to seamlessly interact with other technologies developed by the same company. These virtual assistants can answer questions about arithmetic, historical dates, and basic science facts. Some virtual assistants allow third-party development of add-on features that can provide more specialized information. A review of the research literature does not show signifcant implementation of commercially available virtual assistants in higher STEM education.

Conversational agents, more commonly known as chatbots, are another type of virtual assistants that are more widely used inside and outside of education. A chatbot answers questions or provides information about a specifc topic, such as a company's chatbot providing customers with frequently requested services. The most well-known example of this was Goel's use of Jill Watson, a virtual teaching assistant designed to answer students' questions about policies and assignments for his online AI course (McFarland 2016). This application has been employed by others as well (Chopra et al. 2016). Educators can teach a subject by employing a chatbot to conduct conversations with students within a virtual world (Heller et al. 2016). This is especially useful in medical schools to help students learn to converse with patients (Carrard et al. 2020). Virtual tutors can help students learn course content outside of class at their own pace (Coronell et al. 2019).

This brief and admittedly inexhaustive review of previous predictions about education technology use shows that many such predictions are accurate, even when made 5 years in advance. The most widely used technologies have commercial applications as well, such as 3D printing, AR and VR, and virtual assistants. Familiarity with technology can encourage its adoption by faculty and acceptance by students. Other factors that promote the use of new technologies are the ability

to lower costs, as in the case of 3D printed models and virtual patients, and provide more personalized learning outside the classroom, such as virtual assistants that save time for faculty and increase convenience for students.

### **5 Conclusion**

This book describes:


We hope that our book will help guide education policy-makers, researchers, developers, faculty, and practitioners to make informed decisions about the adoption and use of ILEs.

### *5.1 Overall Impacts*

Throughout our project, we take a holistic view of ILE implementation in higher education by considering not only the benefts of ILEs but also addressing the problems that they create and the challenges to their implementation. For instance, how can educators use technology-based ILEs without exacerbating the digital divide among students, and what policies (either within a school or at the state or federal level) must change in order to promote the adoption of ILEs? By thinking beyond the education research fndings about ILEs, we are hoping that this book is creating a solid foundation for developing a greater understanding of how society and educational innovations can infuence one another. These considerations lead to more challenging problems, but the solutions envisioned in this book will be more realistic and successful when implemented.

### *5.2 Limitations*

A challenge of integrating many of these ILEs into a curriculum is that the physical space is often set up for a traditional lecture or study area, but many academic activities require group work. This necessitates a rearrangement or remodeling of the classroom. Easily movable chairs, accessible electrical outlets for mobile devices, rearrangeable desk space, and whiteboards available for small groups enable students to engage in learning without the limitations imposed by a classroom design. Students value the functional attributes of a learning space and have defnite opinions about where they can effectively study (Beckers 2016). Proponents of new education technology must appreciate how it fts into existing physical structures and campus cultures.

### *5.3 Future of X-FILEs*

The X-FILEs project has been evolving and will continue to adapt to the newly emerging challenges. As we fnish our book, the world is in the middle of a global pandemic. COVID-19 caused college faculty to quickly change the way they teach traditionally hands-on STEM activities (e.g., lab experiments) to online instructional settings. This is an example of emergency remote teaching (ERT). Through a survey, online discussions, and interviews, the research team will document, curate, and study the ERT experiences of faculty who used innovative learning environments (ILEs) despite having little to no prior experience with them. Potential ILEs include synchronous and asynchronous teaching by video, use of virtual laboratory activities, and other emerging technologies. Selected faculty participants will have limited ILE teaching experience since this group teaches the majority of undergraduate STEM courses and is most in need of assistance implementing these technologies. It is expected that most participants will share the challenges that they faced; others may have discovered unexpected advantages of using ILEs. They will also convey how their experiences shape their attitudes toward future use of ILEs in STEM education.

This is what our focus is for now, but we are certain that our future projects will continue to take us to many other unexplored areas of the creative use of ILEs in addressing challenges in STEM education.

### **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# **Personalized and Adaptive Learning**

**Deborah L. Taylor, Michelle Yeung, and A. Z. Bashet**

Personalized and adaptive learning is currently implemented in a variety of ways. To explore the topic and how it might be applied to STEM disciplines, meet Keisha Johnson, our hypothetical student, and Professor Jones, who is implementing personalized adaptive learning techniques into the classroom. Using their shared story, we will provide defnitions, benefts, impact, opportunities, challenges, and future research initiatives while showcasing the choices required by the institution, faculty, and students.

*Keisha Johnson, a pre-nursing major, knows that she needs to earn high grades in all of her classes in order to be accepted into a competitive nursing program. Keisha is a scholarship student attending a large university in her state. She was a stellar high school student but is worried because she graduated from an urban school system that has a reputation of having underprepared students for the challenges of college. Her major concern at this point is that she is starting her second year of classes at the University and has enrolled in Human Physiology, a required but very challenging course that has a reputation as a weed-out class. Keisha is also worried that she won't be able to keep up with the work and might not be able to pass a large class with more than 200 students enrolled due to the possible lack of individualized attention from the instructor.*

D. L. Taylor (\*) University of Kansas, Lawrence, KS, USA e-mail: dtaylor@ku.edu

M. Yeung Loyola Marymount University, Los Angeles, CA, USA e-mail: Michelle.Yeung@lmu.edu

A. Z. Bashet Dallas County Community College District, Dallas, TX, USA e-mail: azbashet@dcccd.edu

© The Author(s) 2021 17 J. Ryoo, K. Winkelmann (eds.), *Innovative Learning Environments in STEM Higher Education*, SpringerBriefs in Statistics, https://doi.org/10.1007/978-3-030-58948-6\_2

*On the frst day of class, Professor Anna Jones makes an announcement to the class that they will be using an adaptive, personalized system for their assignments. She explains to her students how the system assesses each individual's prior knowledge before lecture and then directs content delivery and practice questions to create a learning path that is individualized for every student.*

*Keisha is intrigued by Professor Jones' explanation that adaptive learning is a type of personalized learning made possible by the use of computers. The adaptive learning system delivers assessments that evaluate students' prior knowledge and then using artifcial intelligence prescribes individualized learning content in a sequential manner that is designed to meet the learning needs of each individual. After hearing how the adaptive and personalized learning system works, Keisha becomes quite excited. This sounds like a game-changer to her. She knows that she is a hard worker and likes the idea of having resources personalized to her learning needs.*

### **1 Emerging Trends and Pedagogies**

### *1.1 Defning Personalized and Adaptive Learning*

In personalized learning, instructional approaches are customized to individual learners. Feldstein (2016) noted that personalized learning is what you do, not how you do it. While it is possible to achieve personalized learning in a classroom, it is quite a daunting challenge to personalize instruction when there are more than a few students in the class. Yet, it is now feasible to bring personalized learning to scale utilizing the affordances of adaptive learning technologies. As a result of the changing face of education, computer technology has greatly affected the pedagogy of higher education. The computer has gone from simply being a conduit for delivering course content to students to one that continuously identifes the learning needs of each student and provides individualized learning paths in real time.

Unfortunately, at this time, in the academic environment, the concept of adaptive learning is nebulous. Cavanagh et al. (2020) emphasize that the lack of clear and consistent terminology throughout the educational arena presents a stumbling block for adaptive learning implementation in higher education. There is an effort to clarify "what" adaptive learning is, and yet the technologies that provide adaptive learning vary so much that it is daunting to even identify what should be included in the defnition. Adaptive learning technology provides personalized learning at scale by assessing learners' current skills/knowledge, providing feedback and content, and then constantly monitoring progress by utilizing learning algorithms that provide real-time updates and the necessary tools to improve student learning (Educause Learning Initiative 2017). The Horizon Report (2018) explains that adaptive learning occurs when digital tools and systems are used to create individual learning paths for students based on their strengths, weaknesses, and pace of learning. While this defnition is accepted by many, others (Cavanagh et al. 2020; Pugliese 2016) note that the taxonomy is still quite fuid, thus making it diffcult to gain consensus on a working defnition.

Some adaptive learning systems include profle information from other sources, but most advanced systems create a learning path at the time of interaction with the student. The student's activity profle, learning analytics data, and machine learning then allow the tools to monitor progress and create continuous adjustment to learning paths in real time, in addition to providing personalized scaffolding to promote learning for each student and targeting individualized intervention for improving student success. Adaptive learning environments provide instructors the tools to utilize technologies and data to provide timely feedback on student performance. In describing adaptive learning systems, Mavroudi et al. (2018) quoted Froschl's defnition used in his masters' thesis: "in an adaptive system the needs of the learner are assumed by the system itself and, thus, it adjusts its behavior accordingly" (p. 2). In more formal language, Paramythis and Loidl-Reisinger (2004: 182) note that "a learning environment is considered adaptive if it is capable of: monitoring the activities of its users; interpreting these on the basis of domain-specifc models; inferring user requirements and preferences out of the interpreted activities, appropriately representing these in associated models; and, fnally, acting upon the available knowledge on its users and the subject matter at hand, to dynamically facilitate the learning process."

### *1.2 Pedagogies of Adaptive Learning*

Pedagogy is defned as the methods and practice of teaching (https://www.lexico. com/en/defnition/pedagogy). Personalized and adaptive learning are driving a change in higher education from instructor-centered pedagogies to student-centered pedagogies.

Student-centered learning (also known as learner-centered pedagogy) is based on constructivist learning theory and supports student learning by allowing students to make decisions in their learning (Goodman et al. 2018; Hannafn and Land 1997; Wright 2011). Dockterman's (2018) overview of the history of personalized learning informs us that students learn more effectively when instruction is individualized to the learners' needs and that a new pedagogy of personalization recognizes that each student is different. Bringing that pedagogy to scale, however, requires technological intervention that until now has not been available. Identifying students' needs and providing scaffolding for learning are in the heart of adaptive learning platforms. Scaffolding is defned as the support and guidance provided to the learner until the learner can accomplish a task or demonstrate competence independently (Wood et al. 1976).

"Chunking" of content is an online pedagogical approach guided by cognitive information processing (CIP) research that states, in order to reduce cognitive load and enhance learning, information must be broken down into small manageable "chunks" (Mayer 2005). Adaptive learning content delivery is driven by this online pedagogical approach.

Student learning is enhanced when learners are encouraged to evaluate their learning (metacognition), and many adaptive systems employ tools that allow the students to monitor their progress with some systems offering directed metacognitive experiences for the student as he or she moves through the lessons. Adaptive learning technologies promote learning by utilizing the method of retrieval learning which is often called "testing to learn" (Lindsey et al. 2014; Bae et al. 2019; Miller and Geraci 2016; Thomas et al. 2018). An initial assessment of prior knowledge directs the delivery of content and the scaffolding that addresses the learner's needs with continuous assessments in real-time updates and the learning data constantly allowing for the development of an individualized learning pathway for each student. Retrieval learning and metacognition are components of cognitive learning theories.

Online learning design plays a major role in the design of adaptive learning course development. Cavanagh et al. (2020) developed and shared a blueprint for a design framework as well as a pedagogical approach for adaptive learning that can serve as a starting point for institutions considering adopting an adaptive learning technology for course delivery.

### *1.3 Emerging Technologies*

Emerging technologies enabling personalized and adaptive learning include learning analytics, artifcial intelligence, machine learning, intelligent tutors, adaptive controls, and robust interactive learning content (Groff 2017; Mavroudi et al. 2018; Murray and Pérez 2015). Adaptive learning platforms provide students a fexible learning environment that can accelerate learning by creating an individualized learning path directed by prior knowledge and continuous assessment of performance. In the 2017 Horizon Report, Adams Becker, Cummins, Davis, Freeman, Hall, and Ananthanarayanan identifed adaptive learning technologies as one of the six developments of educational technologies that will have the greatest impact in institutions of higher education. However, the 2019 Horizon Report noted concerns that the expectations for progress were not yet met. Alexander et al. (2019: 35) suggest that the failure to meet expectations may be due to the fact that "Technology tools were felt to be in their infancy, creating a large investment from the institution of time, money, and resources." Yet, the rapid evolution of information processing and Internet technologies enables e-learning to provide personalization, interactivity, media-rich content, just-in-time delivery, and a learner-centered environment where students can take ownership of their learning. Advances in technologies present new opportunities for adapting instruction to individual learning paths. According to the 2017 New Media Consortium (NMC) Horizon Report–Higher Education Edition, "the increasing focus on customizing instruction to meet students' unique needs is driving the development of new technologies."

Adaptive learning technologies can be classifed as adaptive learning platforms and adaptive learning programs. The adaptive learning platforms exist as standalone systems, where all courseware functionalities are bundled into one robust working unit, whereas adaptive learning programs provide a component that can plug into an existing course in an LMS and deliver the adaptive and personalized experience. Either the adaptive systems can provide content developed by the vendor or the systems can merely serve as a framework for institutions to input their own content.

Pugliese (2016) noted that many promising opportunities exist where adaptive learning technology can enhance student learning and success. Adaptive learning technologies can support content created by a vendor, publisher, or institution. There are pros and cons to each of the options, and it falls to the instructor and institution to sort out which to use based upon institutional resources, availability, quality of the content, and usability of the technology. The adaptive mechanisms vary as well. Currently there are four categories of adaptive learning systems, and all integrate learning objectives, instructional resources, and assessments into modules for learning. The four frameworks (from simplest to most complex) are decision tree adaptive systems, rule-based adaptive systems, advanced algorithm-based adaptive systems, and machine learning-based adaptive systems. A short list of some of the emerging adaptive technologies currently used in higher education includes Smart Sparrow, Knewton, CogBooks, Cengage Mindtap, and Realizeit.

Personalizing learning at scale is made possible by powerful computer processing assessment data, interaction data, and learning behavior data for each student in order to create feedback, scaffolding, and continuous assessment that deliver the individualized learning paths. Adaptive learning systems can inform instructors early in the course, even before the frst exam, as the student's activity and performance in the course are continuously monitored and assessed in formative assessments. Information can be sent to the instructor, and early intervention methods can be utilized to catch an at-risk student before it is too late, thus reducing attrition and increasing student success. "Adaptive systems address the fundamentally different levels of prior knowledge, as well as course content progression based on students' skill and outcomes mastery measurement, decreasing faculty load in teaching and remediation to teaching and facilitating" (Pugliese 2016).

### **2 Use of Personalized and Adaptive Learning in 2026**

### *2.1 Content Presentation*

The adaptive learning system develops a learning path for each student and delivers individualized content based upon assessments of performance. One of the more critical requirements is that of delivering the content into "bite-sized" chunks that allow evaluation of bits of knowledge that are then compiled into a learning plan. The adaptive learning system tracks performance and monitors behaviors during the time of interaction and then delivers the adaptive content in an adaptive sequence individualized for each learner. A robust adaptive learning system offers multiple options for the modality of the content delivery, e.g., text, video, or interactive activities. Some adaptive systems have the capacity to analyze performance and determine if the student learns better by interactive activities, watching videos, or reading and then deliver the content in the ideal format that is associated with that student's improved performance.

#### **2.1.1 Opportunities**

In adaptive learning, content delivery is guided by student performance and prior knowledge, thus providing a more personalized experience and promoting student engagement (Dziuban et al. 2016). Kerr (2016) demonstrated that when the adaptive learning systems deliver content in the format determined by assessment to be the best for that student's learning, student learning is enhanced. Additionally, Meccawy et al. (2007) showed that integrating content presentation with learner interaction enhances student engagement and success.

Most adaptive learning systems are accessed through cloud services providing the students with access to course content when they are ready to learn thus enabling student-centered learning, promoting learning autonomy, and encouraging selfregulated learning. Another important component of an adaptive learning system is the ability to provide remediation should a student not have the prior knowledge needed to perform well in the class. This affordance assists underprepared students to gain the knowledge while still in the course, thus not slowing down their forward educational momentum (Dziuban et al. 2016).

#### **2.1.2 Challenges**

The adaptive learning technology feld is so young that there are no set standards for these tools. This can result in confusion for those wanting to implement this new technology. Some tools offer some adaptive opportunities, but the adaptive options vary considerably from system to system. Many institutions and faculty fnd it a very real challenge to select the tool that best meets their needs. While many would prefer to develop their own content, the time and costs to do so are a real challenge to institutions and faculty.

#### **2.1.3 Implementation Strategies**

The recognized value of adaptive learning courseware in the university system to increase student success has prompted numerous initiatives from the Association of Public and Land-Grant Universities (APLU). One initiative is the formation of the Personalized Learning Consortium (PLC), a membership organization charged with increasing information on using technology and personalizing learning in order to promote student success. In addition, the APLU, with funding from the Bill & Melinda Gates Foundation, created a grant opportunity for universities to accelerate the adoption of adaptive courseware by public institutions. Eight universities piloted the process of adopting, implementing, and scaling adaptive courseware. One byproduct of this initiative was the development of a tool for faculty, instructional designers, and administrators of postsecondary institutions to effectively navigate the market of courseware solutions. Recognizing that the selection of courseware is contextual, with the course context playing a critical role in guiding the selection and implementation of courseware, the Online Learning Consortium, Tyton Partners, and the Gates Foundation developed the Courseware in Context (CWiC) Guide (coursewareincontext.org) to assist institutions in the complex process of evaluating and selecting the appropriate adaptive learning courseware for their courses. SRI International aligned the framework for effcacy research. Education of stakeholders is a major component of the CWiC Guide, and suggestions for implementation are guided by the defned needs of that institution.

#### **2.1.4 Research Questions**

Several research questions were proposed at the X-FILEs workshop regarding content presentation and delivery. However, research answering all of the posed questions was found in current literature. This phenomenon supports the premise that adaptive learning is so new that current knowledge is somewhat nebulous. It appears that many, if not most, of the participants did not fully understand the current status of adaptive learning. One of the questions was "How are policies at various institutions determined for selecting and purchasing the content delivery tools?" which is addressed by the APLU's *Implementing Adaptive Courseware Guide* (2017).

*Each morning, Professor Jones pulls up the data on student performance on the assignment for that day and identifes topic areas where students performed poorly. She then adjusts her lecture later that day in order to focus on the content topics that the students found most challenging and reduces the amount of time for the content that the students already know to a minimum.*

### *2.2 Interactions and Communications*

Interactions and discussions have been shown to be a vital part of student learning. From the time of Plato up to the present, students have depended upon interactions and communication with one another and with their instructors to facilitate learning. The range and depth of interactions and communication and the modality vary greatly across the learning environment. Very few stand-alone adaptive learning systems include opportunities for students to communicate and work collaboratively with other students. Adaptive learning programs that only deliver adaptive learning activities are typically used in conjunction with other teaching modalities in order to facilitate interactions between and among students. The component systems delivering the adaptive learning activities can be used in online, blended, and face-to-face courses to enhance student-centric learning. Interactions and communications in most adaptive learning programs are limited to those between the student and the program, with the exceptions mentioned above offered by only a few robust adaptive learning systems.

#### **2.2.1 Opportunities**

Students can choose when they work, and because the assignments are automatically graded, they can receive immediate feedback and scaffolding as needed. A major component of adaptive learning tools is that of feedback and scaffolding. Initially the student is assessed for knowledge, and the results of this assessment guide the delivery of structured feedback and scaffolding as well as new content. Ideally this process offers a student the opportunity to move through the course content at a pace that is determined by his/her prior knowledge as well as the amount of time he/she puts into the learning process. However, typically these adaptive tools are used in conjunction with a regular class (online or face-to-face), but it is not beyond the scope of adaptive learning to develop courses that allow students to complete a course more quickly than a typical semester period. These self-paced courses offer fexibility of scheduling, but do present challenges as explained below. There are some institutions of higher education now offering self-paced courses, but these do not typically consist of adaptive learning software.

The instructor dashboards allow instructors to monitor student progress and identify students who are having problems early in the course so that intervention can occur in a timely manner. The student dashboards allow students to track their progress through the learning materials and promote self-regulation of learning, and some programs even offer metacognitive assessments to enhance the learning experience.

#### **2.2.2 Challenges**

For a student to beneft from an adaptive learning system, he or she must initiate interaction with the system, sustain engagement, and successfully complete the assigned activities. A student working in an adaptive learning environment brings a number of behavioral factors into play that affect the interactions with the course content. The internal student-level factors that infuence student behavior in an adaptive learning system can be examined from the perspective of motivational theory. It has been shown that engagement, self-determination theory, autonomy, self-regulation, and the level of internal motivation are positively associated with student learning. The action of initiating engagement is driven by motivational factors. Reeve (2012) notes "self-determination theory is unique in that it emphasizes the instructional task of vitalizing students' inner motivational resources as a key step in facilitating high-quality engagement."

As mentioned above, many fully inclusive adaptive software systems do not offer student-to-student interactions nor student-to-instructor interactions. This could result in the student feeling isolated if no compensating adjustments were offered.

#### **2.2.3 Implementation Strategies**

Adaptive software programs can be used to offer supplemental learning opportunities for the traditional classroom as well as in the fully online environment. Many of the fully adaptive platforms do not offer student-to-student interaction, and if this type of interaction is to be a part of the course, it is often necessary to utilize the communication opportunities offered by a traditional learning management system.

#### **2.2.4 Research Questions**

One research question asked in the workshop was: "Can personalized avatars be used to deliver the feedback and scaffolding to lessen the students' feelings of isolation?" Kim (2012) proposed guidelines for designing virtual change agents (VCAs) (avatars) that would promote student learning needs in a personalized manner in online remedial math courses. The strategies were based on motivational learning theories and the interactions between the student and the VCA.

*As the semester progresses, Keisha enjoys doing many of the interactive assignments. She likes receiving immediate feedback on her performance and guidance for her study in order to provide effcient use of her time. Keisha even enjoys the metacognitive component of each question and the ability to track her progress through the course. She recognizes the value of the learning resources and takes time to do additional review assignments that the program provides that focus on what she had missed before but limits material that she has already mastered.*

### *2.3 Learner Activities*

Prior knowledge assessments are typically the frst activity that the learners encounter in an adaptive learning environment. Results from these assessments guide the development of an individualized learning path that provides personalized content delivery for each student based on their strengths and weaknesses. This content can range from simplistic fll-in-the-blanks to more sophisticated scenario-based learning activities. The importance of engaging students in learning activities is paramount. In many of the adaptive learning systems, the content included in the system replaces a textbook, meaning that all of the learning occurs via the interactions with the adaptive learning system. The learning activities delivered to each student are determined by the results of assessments, thus creating differing learning paths. Students receive different learning opportunities by the adaptive learning system as it analyzes student knowledge and scaffolding needs (Meccawy et al. 2007). Learning activities that engage interactive animations of complex processes (e.g., cell division) with questions interspersed in the activities increase student engagement.

#### **2.3.1 Opportunities**

Adaptive learning systems have the capacity to provide targeted information based on individual learning needs (Peter et al. 2010), provide appropriate scaffolding (Raes et al. 2012), and personalize activities based on student responses (Chen 2011; Normandhi et al. 2019; Blair et al. 2016).

Gephardt (2018) studied the effect of utilizing adaptive learning courseware on student performance in an Economics course at Colorado State University as part of an Association of Public and Land-Grant Universities (APLU). She found that the "students who completed low-stakes adaptive assessments outperformed their peers who did not complete the adaptive assignment on easy and moderate questions on the exam that could result in a higher course grade. This suggests that if adaptive learning courseware is integrated as low-stakes assignments then student outcomes improved with relatively little effort on both sides of the instructor and the students" (p. 16).

#### **2.3.2 Challenges**

Providing engaging, robust, and applicable learning activities that address the learning outcomes is probably the most challenging aspect of building an adaptive learning experience (excluding building the learning platform). Following that is the ability to develop appropriate feedback and purposeful scaffolding that free the student from needing assistance from the instructor in order to move forward in his or her learning path.

#### **2.3.3 Implementation Strategies**

There is often resistance to accept this modality by the faculty due to training needs and time to set up the system (in those that are modifable). Alignment with course content is challenging since many faculty members determine the content for their

own courses. Therefore, it would be important that the system allows for adaptation by the instructor to give him/her ownership. The University of Central Florida modeled an excellent approach in developing and delivering adaptive content in their pilot program (Dziuban et al. 2016). However, it needs to be mentioned that the administration was behind the project and provided a great deal of fnancing, not only to pay the faculty to develop the content but also to pay graduate teaching assistants and instructional design support to reduce the workload of the faculty. Any institution considering building their own content would do well to explore what the UCF has done.

#### **2.3.4 Research Questions**

One question from the 3-day X-FILEs workshop was: "What learning activities are most effective in promoting student engagement and learning?" Linnenbrink and Pintrich (2003) determined that relevant and relatable activities promote student engagement and learning. Additionally, Van Lehn (2011) showed that utilizing an adaptive learning experience promoted student engagement and learning. Therefore, a valuable area of research would be that of examining student behaviors such as persistence, self-regulation, internal motivation, and engagement and see if the adaptive learning systems can capture that data correctly and determine if student learning is affected.

*Dr. Jones designed her class so that the adaptive learning system assignments are due the evening before each class begins. This prepares the students for learning to occur during class by remediating those who were lacking in knowledge needed to address new material.*

### *2.4 Assessment*

The most effective adaptive learning systems initially assess the student's prior knowledge followed by continuous formative assessment as well as guidance as the student moves through the course content (EdSurge 2016). Dziuban et al. (2018) proposed that adaptive learning acts like a GPS for students. It allows for personalized instruction by altering students' pathways through course objectives. In addition, adaptive learning systems continually assess students' knowledge, guiding them to effciently and effectively progress through the course.

In conjunction with the creation of a personalized learning path is the requirement of mastery of the material (Gebhardt 2018). Mastery learning requires clear measurable learning objectives, an idea of what mastery of that learning objective entails, learning activities that assess the mastery, and a means of tracking and sharing the information to direct learning. STEM courses depend upon learning new content based upon prior knowledge, and a student moving forward in a lesson without mastering the concepts is a recipe for failure. Adaptive learning provides the assessment tools and evaluation options that can assure mastery of content, not only at the time of new content delivery but periodically checking the continued mastery throughout the learning path.

#### **2.4.1 Opportunities**

Adaptive learning courseware can signifcantly reduce the amount of time a student takes to complete a course when evaluation of prior knowledge shows mastery of that content. The individualized learning pathway provided by the adaptive learning courseware can allow the student to move forward to the next module at a pace that is suitable for him or her. If gaps in knowledge are revealed, a robust adaptive learning platform provides remediation as needed, including appropriate feedback and scaffolding.

Grading open-ended questions such as essays, projects, etc. to assess higherorder thinking skills in didactic curricula of large classes can be onerous. Quite often, instructors use multiple-choice questions (MCQs). However, creating higherorder thinking questions in MCQ format is very challenging and time-consuming because typically a scenario, case study, or fairly complex problem needs to be described and three or four plausible, but incorrect choices created. Aguilar et al. (2006) proposed a computer-based adaptive assessment tool designed to use formative and summative assessments developed by teachers allowing them to use the information to perform real-time evaluations of the learners' levels of understanding. Adaptive learning systems take the process one step further by removing the instructors' tasks of evaluating the information to determine the levels of student understanding as well as providing scaffolding to support the learner where needed, thus freeing them up to have more personal contact with the students.

Formative assessments are effcient for both instructors and students to assess learning in personalized and adaptive learning environments. Instructors can check students' understanding through formative assessment and collect valuable data on student learning and then use that data to modify instruction. Godfrey (2006) found that incorporation of computer-based assessment increased student engagement in learning and instructors believed technology-enhanced assessment tools positively impacted their teaching.

*Technology-enabled assessments* including assistive technology emerged as potentially powerful mechanisms for measuring growth mindsets or behavioral attributes of students as they engage in the learning process (West 2011). The Information Research Corporation developed an integrated technology platform eTouchSciences to support STEM learning that includes devices that provide multiple forms of feedback, including tactile, visual, and audio, to the student (Thomas 2016). Examples of technology-enabled personalized and adaptive assessment tools used in higher education include Carnegie Mellon University's *Cognitive Tutor*  *Software*, Pearson's *MyLabs,* McGraw-Hill's *ALEKS Online Tutoring System* and *LearnSmart*, and Australia's *SmartSparrow*.

*Intelligent tutoring systems* are some of the earliest adaptive learning technologies, and they offer personalized and interactive learning experiences that help students engage in learning more effectively than traditionally based instructional methods. This technology tool enables personalization of learning and evaluation of performance in real time. Intelligent tutoring systems enhance students' engagement and individual learning experiences by providing immediate and adjusted feedback based on each student's learning progression and his or her actions and responses to given questions and lesson activities (Thomas 2016). Adaptive learning systems go one step further to supply scaffolding to add to those interactive experiences in order to enhance learning.

#### **2.4.2 Challenges**

Verifcation of identity is a must. It is very important to ensure that it is really the student doing the work. There are a number of methods to verify identity; however, many are quite costly to the student or institution.

Formative assessments composed of multiple-choice questions, including immediate feedback and scaffolding for learning, can be delivered via learning management systems or adaptive learning tools and can be readily directed by teachers. Formative assessment with feedback and scaffolding becomes more challenging when one incorporates critical thinking and complex problem-solving (Spector et al. 2016: 59). The value of formative assessment cannot be denied, yet the overemphasis on summative assessment has resulted in inadequate resources for formative assessment (Ecclestone 2010; Sadler and Good 2006).

#### **2.4.3 Implementation Strategies**

Instructor training and professional development opportunities are paramount to implementing a successful adaptive learning endeavor. The University of Central Florida (UCF) and Georgia State University provide excellent examples of instructor training to promote adaptive learning in their institutions. The UCF developed a support network of instructional designers, technical experts, and content experts to assist faculty in developing courses using adaptive learning, and to further enhance the onboarding experience, they developed a self-paced training course for faculty (Cavanagh et al. 2020). Georgia State offered a comprehensive adaptive learning workshop to foster faculty buy-in and commitment. They found that providing support as needed and building a community of inquiry enhanced the implementation of adaptive learning using the funds received from the APLU Learning Grant (Tesene 2018). Developing and delivering formative assessments require extensive time and resources, yet this is a must for a successful implementation if an institution hopes to build its own adaptive learning content.

### **2.4.4 Research Questions**

A powerful research question is addressed in the literature and is worth mentioning here. That one question was: "What pedagogical tools in adaptive learning environments best promote student engagement (and outcomes)?" Scaffolding is one of the most important pedagogical tools utilized in adaptive learning. Scaffolding is described as the structure of content and feedback in such a manner that the student can work through the lessons without requiring intervention from an instructor, thus promoting autonomy and self-direction. A well-designed adaptive learning platform offers a robust experience with numerous branching options supported by scaffolding to allow students to work independently (Raes et al. 2012).

*Professor Jones compared the performance of recent students in her course using adaptive learning with that of students from classes before she implemented the active learning component. She was able to quantify a signifcant increase in student pass rates and a decrease in course attrition after she integrated the learning activities into her course.*

### *2.5 Co-curricular Activities*

Co-curricular activities are learning experiences offered outside the classroom that allow students to expand their interests and perhaps even gain skills that would prepare them for their careers. Co-curricular should not be confused with extracurricular activities like sports and performance opportunities. Co-curricular learning experiences add depth to what is occurring in the classroom, yet are not typically graded. Co-curriculars include leadership skills training, service learning, and study abroad.

#### **2.5.1 Opportunities**

Co-curricular activities could easily provide personalization of a student's educational path and the chance to apply and implement lessons learned without interfering with the set academic path the most institutions require. A menu of adaptive learning minicourses could offer students opportunities to gain skills adjacent to their primary courses, thus providing a personalized option of learning opportunities that are not part of the program curriculum.

#### **2.5.2 Challenges**

One challenge is to ensure the safety of students participating in co-curricular activities while on and off campus. Another challenge is that many students at large universities have diffculties knowing what opportunities are available to them.

#### **2.5.3 Implementation Strategies**

Researchers at the University of Washington discovered that students had diffculty fnding co-curricular opportunities because they used many different sources to get information. A recommendation from the study was to address the student context and offer information at several different levels.

#### **2.5.4 Research Questions**

Many of the participants at the X-FILEs workshop were unfamiliar with what cocurricular offerings were, and all questions posed were already answered in the literature.

### **3 Conclusions**

Personalized and adaptive learning offers great opportunities to promote student learning, yet it presents a number of challenges that instructors and institutions will need to address in order for an adaptive learning implementation to be successful. The frst step is to identify what problems are being addressed in implementing an adaptive learning adoption and after that what "right" adaptive learning approach and system are needed to be determined. Will an add-on component to be used with existing courses within various learning management systems solve the problems that are driving the move to adopt adaptive learning, or is a fully independent adaptive learning platform needed? Questions of compatibility with the current learning management systems will need to be answered if the former option is chosen. Additionally, while decisions to implement an adaptive learning approach are typically made by the institution's administrators, yet without faculty buy-in, it is highly unlikely that a successful outcome will occur. Engaging faculty early in the process is advised coupled with institution-wide engagement increasing the possibility of a successful outcome (Implementing Guide 2017: 2).

*At the end of the semester, Keisha is one of the top-scoring students in the class and has received notice that she was accepted into the nursing program. She is hoping that the nursing courses she will need to take will provide learning resources that are adaptive and personalized like the physiology course she just completed. However, adaptive and personalized courses are limited by the resources at a school. And there are still some areas that require additional research to be sure that time and money spent on these courses result in better student outcomes.*

### **References**


Kerr P (2016) Adaptive learning. ELT J 70(1):88–93. https://doi.org/10.1093/elt/ccv055


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# **Multimodal Learning**

#### **Bettyjo Bouchey, Jill Castek, and John Thygeson**

### **1 Emerging Trends and Pedagogies**

The ubiquitous use of digital technologies is continually reshaping the ways individuals access information, share ideas, and communicate with one another. Doing so requires the nimble use of skills, strategies, and mindsets to navigate, communicate, and collaborate online and across multiple contexts (Leu et al. 2017). These changes have profoundly affected instructional choices in education. In today's education landscape, three key motivations challenge traditional notions of teaching and learning and set forth a strong case for multimodal learning as a critical pedagogy:


B. Bouchey ()

National Louis University, Chicago, IL, USA e-mail: bbouchey@nl.edu

J. Castek University of Arizona, Tucson, AZ, USA e-mail: jcastek@arizona.edu

J. Thygeson Drexel University, Philadelphia, PA, USA e-mail: jrt46@drexel.edu

#### 3. Students are increasingly diverse (National Center for Education Statistics n.d.).

The academy has long relied upon instructional approaches that favor text-based learning. These approaches are sometimes referred to as factory models of teaching because they include top-down management and favor outcomes designed to meet societal needs and rely on age-based classrooms (Bezemer and Kress 2016; Matusiak 2013; Phuong et al. 2017; Sankey et al. 2010). Researchers posit that presenting information in this one-size-fts-all way does not accommodate the widest possible range of learners' needs (Gee 1996; Phuong et al. 2017). As technologies continue to shape the ways individuals engage in learning, exchange ideas in networks, and communicate in multiple forms, educators must expand their ideas of how, where, under what conditions, and which tools people use to learn in a digital age. The changing nature of student demographics and shifts in the technological landscape call into question traditional ways of teaching and learning.

Technological advances, coupled with considerations of the changing needs of today's learners, call for exploring new directions for multimodal teaching and learning. Multimodal learning (MML) can be defned as "learning environments [that] allow instructional elements to be presented in more than one sensory mode (visual, aural, written)" (Sankey et al. 2010: 853). Multimodality looks at the many different modes that people use to communicate with each other and to express themselves. Multimodal learning is relevant as increases in technological tools and associated access to multimedia composing software have led to the ease of use of many modes in presenting, representing, and responding to information.

MML calls for sensemaking where learners take in information, process, and make personal sense of ideas to form deep learning patterns (Bezemer and Kress 2016; Moreno and Mayer 2007). MML expands the range of choices available in a learning environment so that learners can co-construct learning through their preferred mode while also being challenged by integrating the use of other modes (Nouri 2018; Phuong et al. 2017; Sankey et al. 2010). MML requires a high level of agency (self-discipline) by learners, who must have the metacognition necessary to understand *how* they learn and also *when* to challenge themselves to learn in ways that lie outside their preferred modes. Such sophisticated awareness requires a student to explore their metacognition and requires instructors to explore how assessments are administered and leveraged (Bezemer and Kress 2016; Moreno and Mayer 2007; Phuong et al. 2017; Sankey et al. 2010). MML promotes a greater emphasis on encouraging student voices during the learning process and calls for educators to listen to students' preferences. As such, multiple modes should be used to shape the content, response options, as well as the delivery of instruction (Nouri 2018; Phuong et al. 2017).

As a powerful means to customize and adjust learning strategies to reach diverse learners, MML leverages our technology-rich world, digital tools, and networks and can be used to address Universal Design for Learning (UDL) as well. UDL offers three basic principles that encourage the design of supportive learning environments. These principles call for instructional designs that build in multiple means and modes of (1) *representation* (offering fexible ways to present what we teach and learn), (2) *action and expression* (offering fexible options for how we learn and express what we know), and (3) *engagement* (offering fexible options for generating and sustaining motivation to learn) (Rose and Meyer 2002). Encouraging learners to compose ideas using different modes encourages connections to concepts and illustrates ways to employ the principles of UDL. Moreover, deep learning happens when students are given the opportunity to participate in a range of cognitive and social learning activities that are responsive to their preferences and needs or modes (Bezemer and Kress 2016; Moreno and Mayer 2007). Most importantly, MML serves as well-researched pedagogy that sets a platform for educators needing to shift their emphasis from the delivery of instruction to the engagement of their learner (Bezemer and Kress 2016; Tonsing-Meyer 2013). In this way, MML can be an equalizer for diverse students because the use of different modes encourages diverse ways of communicating. MML can be inclusive of multilingual students, English language learners, and those with learning differences and moreover can set the stage for an engaging, rich, and creative learning experience.

### *1.1 Aspects of Teaching and Learning Covered in the MML Literature*

An MML literature survey was conducted in conjunction with eXploring the Future of Innovative Learning Environments (X-FILEs) project. Several areas of teaching and learning were examined to create a framework to conceptualize new forms of learning in the next fve years, in other words in 2026. The areas of teaching and learning spanned (a) content presentation, (b) interactions and discussions, (c) learner activities, (d) assessment and evaluation, and (e) co-curricular learning and activities. The following is an overview of literature pertaining to each of these areas.

#### **1.1.1 Content Presentation**

Course content can be presented and delivered in multiple modes, such as visuals, auditory productions, text, and multimedia (Bezemer and Kress 2016; Matusiak 2013; Nouri 2018; Sankey et al. 2010). Through MML, content can be curated, either in advance or dynamically, to accommodate multiple access points. Multiple modes are offered so that students can gravitate toward the modes that best align with their individual styles and needs and to encourage the complementary nature of multimodal content (Nouri 2018; Phuong et al. 2017; Sankey et al. 2010). This section assumes that content is either contained in a learning management system for purely online coursework or blended for hybrid and face-to-face instruction and that multimodal content and delivery can augment traditional lectures.

#### **1.1.2 Interactions and Communications**

The use of MML in higher education creates a foundation for fexible, open, and multidirectional interaction and discussions with students, their peers, faculty, staff, and the business and community at large. As students embark on sensemaking and co-construction of knowledge in an MML-based classroom (Bezemer and Kress 2016; Moreno and Mayer 2007), they can use a variety of modes to engage with other stakeholders to shape their understanding. Using different modes, students can spontaneously engage in parallel and complementary dialog to deepen learning, answer questions, and engage in sensemaking of the learning objectives in and around a classroom.

#### **1.1.3 Learner Activities**

Jewitt (2008) suggests that how knowledge is represented, as well as the mode and media selected, is a crucial aspect of knowledge construction that makes the form of representation integral to sensemaking and learning, more generally. In other words, the way something is represented shapes *what* is to be learned, the curriculum content, and *how* it is to be learned. In our digital world, our efforts to design instruction must be responsive to the ways we intend learners to use the new knowledge they acquire in the workplace and beyond. MML calls for educators to shift their role from learning designer to learning facilitator (Bezemer and Kress 2016; Tonsing-Meyer 2013). Because MML sets a foundation for a co-construction of knowledge between faculty and students (Bezemer and Kress 2016), learner activities are available just-in-time, based on the student's preferred mode and personalized, drawing in the student's preferences for learning. In addition, the use and integration of different modal representations can reinforce ideas and make learning more memorable.

#### **1.1.4 Assessment**

Course assessment strategies must also be multimodal in order to ensure objectivity. Whether formative or summative, all course assessments are designed in a modally agnostic way so that the fnal product can be produced in the student's preferred mode and preferably in a way that demonstrates the intersectionality and complementary nature of different modes (Jewitt 2008; Nouri 2018; Phuong et al. 2017; Sankey et al. 2010).

#### **1.1.5 Co-curricular**

Co-curricular learning refers to activities, programs, and learning experiences that complement, in some way, what students are learning in formal classes, but also serve to expand the relevancy and contextualization of that learning in the real world. Since MML is pedagogical, it does not explicitly reference co-curricular activities or learning. Even so, opening up classroom interactions and discussions into a multimodal environment provides a foundation for co-curricular learning from stakeholders outside of the classroom, such as college and university staff, business and community leaders, friends, and family. Learning from those outside of the classroom should not be overlooked or undervalued.

### *1.2 Research Questions Answered*

The literature on multimodal instruction also answered research questions around design for multimodal instruction and how to use multiple modes to encourage deep learning. Moreover, the literature helped to contextualize many of the early research questions and provided a democratic, pedagogical framework within which to conceptualize content presentation, interactions and discussions, learner activities, assessments and evaluations, and co-curricular instruction in 2026.

### *1.3 Future Research*

As noted in the 3-day X-FILEs workshop, there remain areas that call for future research, specifcally in the areas of how MML can impact a student's identity development and self-initiative. Also discussed in the workshop, there remain some practical areas that would beneft from further formalization specifc to how institutions of higher education support the technology needed to implement multimodal instruction. More inquiry into the effcacy of some modes over others in terms of deep and sustained learning may also be needed, perhaps as a way of creating synergy across modes as well. Workshop participants also noted the importance of developing a framework for the ethical use of MML that would suggest appropriate use and rules of engagement that support students, faculty, and staff in setting healthy boundaries for immersive learning in an information- and technology-rich society. Further research is also needed to better understand the affordances that multimodal instruction can create for students, especially as it pertains to students with disabilities. Lastly, as co-curricular learning takes hold in the use of multimodal instruction, workshop dialog indicated external validation, perhaps by higher education professionals or other community and business partners of learning credentials, will need to be investigated.

### **2 Use of Multimodal Learning in 2026**

This chapter brings MML into focus through the work of the eXploring the Future of Innovative Learning Environments (X-FILEs) project spanning a series of focus groups, 3-day in-person workshop, and further investigation through a team of researchers, industry partners, and faculty brainstorming on how this democratic pedagogy can shape (a) course content presentation, (b) interactions and discussions within a course, (c) learner activities, (d) assessment and evaluation, and (e) co-curricular aspects of education. Each section outlines opportunities for using MML as a framework, implementation strategies, and key questions that have been answered or left open in the literature.

### *2.1 Content Presentation*

Content presentation has progressed from an educational format in which the instructor is charged with passing on his/her information to students via lecture, recitation, and associated activities into an experience in which information is presented to learners in many ways. While lecture is still used, it is frequently supplemented with additional educational content in additional formats, both in class and away from the classroom.

#### **2.1.1 Opportunities**

Several affordances exist in providing multimodal course content, chief among them being the self-directed nature of students selecting content delivered in the most familiar or comfortable mode, which is when the deepest learning occurs (Bezemer and Kress 2016; Moreno and Mayer 2007). Moreover, traditional classroom lectures can be augmented by multimodal content and delivery in the learning management system or can provide the opportunity for classroom time to be used for group- or project-based learning. Augmenting content in a learning management system or the traditional classroom lecture can create an active learning space where students are focused on discovery, rather than consumption of information. Discussed in the X-FILEs workshop, this content created by faculty could be commoditized, whereby students could subscribe to certain modes based on their individual preferences for a small fee, rather than relying on traditional per-credit tuition models. In an online course, MML course content could also spark faculty and course designers to create and curate content in new and innovative ways. Additionally, the multimodal nature of virtual environments such as lab simulations, virtual reality, and augmented reality provide experiential opportunities that would otherwise be very diffcult to provide in "real life."

Another opportunity exists to deliver MML course content to those who require assistive technologies. Assistive technology (AT) can be defned as any item, piece of equipment, software program, or product that increases, maintains, or improves the functional capabilities of people with disabilities (Assistive Technology Industry Association 2019). The number of people who could beneft from AT is not insignifcant. Currently, 15% of the world's population has some form of disability, and 20% of those people experience signifcant disabilities. As the world's population continues to grow, so too will the total population of those that experience disabilities; furthermore, this population is aging, and there is an increase in chronic health conditions (World Health Organization 2011).

In addition to reaching individuals who would normally have diffculties accessing education, using these technologies can help educators stay compliant with the law. The Rehabilitation Act of 1973 and the Americans with Disabilities Act addressed workplace support for people with disabilities and were later adapted for use in educational situations (U.S. Equal Employment Opportunity Commission n.d.; U.S. General Services Administration 2018). Providing individuals with disabilities with the opportunity to the same education and in the same time frame as students without disabilities will beneft the education provider by staying compliant with laws and regulations.

#### **2.1.2 Challenges**

Workshop participants agreed that chief challenges in designing courses and delivering them to incorporate MML are cost and speed of design. It may be cost- and time-prohibitive to create content in all learning modes. For example, one course module on calculating averages in a statistics class could contain the following content items:


In addition to the time and cost of curating MML content, faculty may fnd it diffcult to switch between modes while in the "live" classroom environment, suggesting more of a need to augment the lecture through MML content in a learning management system as well. Faculty and instructional designers may also fnd it diffcult to fnd or create content in several modes, especially if they fall outside of their individually preferred modes.

Another challenge of using MML in course content is relying on a student's selfawareness and self-direction to explore and represent content in their preferred mode or combination of modes. Accessing all course content in one mode, even if preferred, might not be the most effective approach as combining modes may serve as reinforcement and encourage connections. There may also be implications with respect to accessibility where some modes might be richer than others. For example, a student who only accesses materials in a preferred mode may not beneft from deeper learning utilizing content presented in different modes or ones that incorporate many modes such as virtual reality.

#### **2.1.3 Implementation Strategies**

Incorporating MML in course content and delivery of this type of instruction in "real time" by a faculty member requires a symbiotic relationship with the student so that the faculty member can match instruction to the learning needs of their students. In thinking through online courses, curating content requires examining instructional design frameworks that guide the thoughtful assembly of course content. The seminal and informative ADDIE model of design, which includes the elements of *a*nalysis, *d*esign, *d*evelopment, *i*mplementation, and *e*valuation (Molenda 2003), can still be used, perhaps with iterations derived from rapid prototyping.

#### **2.1.4 Research Questions**

Literature on MML clearly answers the research questions set forth by the X-FILEs team in this area of how to choose the best mode for a subject or student group. Through its very nature, MML provides content that addresses all modes regardless of subject matter or student group characteristics.

*As Juan accesses his online class for his Introduction to the Fundamentals of Science, he is faced with many choices that he will navigate throughout the term. He has a selection of content available to him to support his frst week of learning, including chapters from an online textbook, a series of videos and podcasts, and a set of keywords he can use to locate his own or additional content—he is also asked to register for his asynchronous virtual reality classroom where he can access his book, the videos, and podcasts and perform and post his research. His university has obtained a grant to fund the augmented course design, and Juan has a low-cost technology fee that he pays each term to offset the cost of the virtual classroom licensing, hosting, and support to the institution. The virtual classroom is where the majority of his course will actually take place throughout the week, though he will have several opportunities to participate in the learning "live" or through other modes as well. To provide the students with a sense of familiarity, the virtual classroom is designed just as a school is; each room corresponds to a different subject or module. Juan navigates his avatar into each room to view content and participate in class as though he was sitting in a physical classroom.*

### *2.2 Interactions and Communications*

Interactions and discussions around academic coursework have evolved from episodic, in-class lectures and peer dialog to full-scale, immersive, and 24/7 opportunities for dialog and deep learning. While in-class lecture and dialog remain central ways to build foundational learning, the co-curricular opportunities for interactions and discussions serve as a platform for more immersive learning.

Using the principles of UDL (Gordon et al. 2014) and frameworks that encourage the wide use of MML, students, peers, faculty, staff, and community/business stakeholders can engage in open and fexible and dialogic interactions and discussions. Dialogic interactions create space for shared leadership in learning in which learners and instructors become co-inquirers; in this way, a more egalitarian classroom environment can emerge, devoid of traditional teacher/student power dynamics. Together, they collaboratively engage in creating and evaluating new interpretations in order to gain insight into the world, themselves, and each other (Reznitskaya 2012). Learners can and should be viewed as active drivers of their own learning (Bezemer and Kress 2016; Sankey et al. 2010). In this way, students can choose multiple modes within which to engage in learning and receive just-intime or real-time information to help contextualize their learning. Providing an opportunity for multidirectional modes of communication enables students to be resourceful and frees up faculty time as well.

#### **2.2.1 Opportunities**

As students shift from consuming knowledge to actively driving and constructing it, they can enhance their understanding by using different modes to represent their understanding. Interacting across modes frees students to think more creatively and integrate knowledge faster. By opening up meaning-making (Postman and Weingartner 1969), the process by which people interpret situations, events, objects, or discourses, to expand communication, MML may also empower students to generate meaning and make sense of themselves, their experiences, and their relationships through their learning. Moreover, MML affords students who may have trouble communicating in a particular mode with the opportunity to express themselves in another. In this more fexible and dynamic environment, students with disabilities can freely operate within their most capable mode. MML also provides new avenues for student collaboration, beyond the confnes of the traditional classroom or learning management system.

#### **2.2.2 Challenges**

Despite the benefts and possibilities that MML can make possible, there are also challenges introduced. For example, the use of images to convey meaning requires faculty to develop visual literacies required to infer how others may interpret their creations. Not all viewers walk away with the same meaning from visuals, and thus other modes may need to be integrated to convey a common meaning. Identifed in the workshop, another challenge may be introduced when initializing interactions exclusively via technology among learners who come from different cultural backgrounds. In the absence of context clues that come from facial expressions, body language, and gestures, miscommunications may occur. Pairing MML strategies with opportunities to connect through two-way videoconference technologies and to openly dialog about different communication norms across cultural groups may support overcoming these challenges. A fnal challenge with MML identifed in the workshop centers around limiting opportunities for face-to-face communication due to technological dependency. Relying solely on technologies for bridging distances between learners may in fact create more distance and feelings of isolation. MML is intended to provide options for learning and instruction and does not intentionally aim to create distance-only options. Discussing ideas face-to-face, in combination with the use of different modes of meaning-making, can provide fexible options for learning engagement. We aim to support educators in their implementation of MML in support of designing learning interactions of all kinds, including face-to-face.

Another challenge in the use of MML in discussions and interactions is the level of ability in technology use among faculty, staff, and students. The challenge stems from the concept that today's students are digital natives, while most of today's teachers are digital immigrants (Bezemer and Kress 2016; Blum and Barger 2018; Kirschner and De Bruyckere 2017; Lambert and Cuper 2008; Moreno and Mayer 2007; Phuong et al. 2017; Picciano 2009; Prensky 2001; Sankey et al. 2010; Wang et al. 2014). A less rigid defnition of the digital natives/immigrant is now used in which the technology exposure is more a factor of a variety of reasons, such as demographics, opportunity, social infuence, cognitive knowledge, and socioeconomic status (Kesharwani 2020). A key concept to take from these studies on digital natives/immigrants is that a primary focus should be placed on reaching across the technological divide between varying technology levels when designing course content. Relatedly, it is important to think through access to technology among different members of the population and how this relates to the value of MML to discussions and interactions.

Lastly, as technology evolves, communication channels must be evaluated for privacy and security. Workshop attendees stressed that it is critically important for faculty and students to know what modes and channels may be open for public viewing. In fact, faculty professional development on tool privacy and accessibility will be important and ever-changing year to year or perhaps term to term as the speed of technological advances and tool creation continues to increase. At the same time, students need orientation to tool privacy and accessibility and experience with the types of tools the university provides, as opposed to ones they are not. There may be tools that do not meet privacy or accessibility standards that the university would not support. The choice of tools and the benefts and drawbacks, should be communicated clearly to students.

#### **2.2.3 Implementation Strategies**

To motivate students to interact beyond the traditional faculty-student relationship, faculty can provide for multimodal communication either in advance, while a course is in progress, or following learning activities to promote refection (Dewsbury 2019). For example, a classroom meeting or online discussion question could have a companion virtual world classroom where students interact in real time through avatars or a backchannel for a face-to-face meeting where students can contextualize the lecture with peers or other educational stakeholders. Course participation could also be linked to time-limited assignments in the workplace or community in person, through videoconference, or virtual networking to encourage idea-sharing and real-world application of learned content.

#### **2.2.4 Research Questions**

While research in MML is not entirely new, research that addresses challenges identifed in the workshop concerning second language learners are an important direction to pursue. Offering multimodal texts is often benefcial for second language (L2) learners as multiple modes can scaffold understanding (Plass et al. 1998; Royce 2002). This is because, as Farias and Abraham (2017) state:

multimodal texts more effectively support second language reading by providing input that caters to different learning styles and that they are familiar, authentic, and contextualized to the learners' lives. Moreover, these texts facilitate learners' meaningful interaction not only intratextually, by exploring the text/image semiosis, but also intertextually, by allowing readers to become literate in the different genres that are constructed multimodally.

Thus, while L2 learning is supported by the use of multimodal texts, it can also develop learners' broader understanding of concepts. Examining identity and literacy development surrounding the creation of multimodal texts both within and beyond course objectives is an fruitful area in need of additional research. These directions can further support MML for L2 learners across felds. While MML seems to indicate there are benefts associated with the use of multiple modes of communication for students, less is understood about the benefts of students' choices; research into this area is still needed.

*In the virtual classroom, Juan can engage through phone, text, email, instant messaging, or other means with his professor, a group of scientists in industry and other colleges, as well as all his peers and advisors. Professor Aranda also actively engages in the virtual classroom and spends time cultivating personalized relationships with each student—in addition to maintaining the relationship with the community of scientists and supporting the classroom. During the Tuesday evening session, she carefully plans a short didactic lecture that is then supplemented with a series of small group virtual breakout rooms facilitated by other scientists where students and scientists work through the theory and discuss ways to demonstrate mastery of the learning objectives for the week.*

### *2.3 Learner Activities*

Learner activities can take many shapes and forms and may connect resources available through coursework to online and community resources. The choices faculty make include multimodal options that build toward applying what is learned within and beyond the classroom. However, choices often depend on individual faculty and the pedagogical disposition of a department, college, or university. In this way, learner activities can be varied and may involve multiple formats and response options that call for different ways to illustrate learning.

MML calls for educators to shift their role from lecturer to learning facilitator and designer (Bezemer and Kress 2016; Tonsing-Meyer 2013). MML encourages fuidity between accessing information using multiple modes and representing knowledge through the integration of multiple modes. This approach encourages personalization as well as creativity and calls for a co-construction of knowledge between faculty and students (Bezemer and Kress 2016). In an MML environment, learner activities can be more collaborative, designed to require real-world problemsolving, and created to apply knowledge based on a compelling purpose for learning.

#### **2.3.1 Opportunities**

In shifting the focus from lecturer to learning designer or facilitator, faculty are free to work with individual students to understand their current state of learning and to work collaboratively on learner activities that will enhance their mastery. Students are thus empowered to fnd the best ways to integrate the subject matter alongside faculty. Faculty can also learn from student-designed learner activities when they use modes and tools unfamiliar to the faculty.

#### **2.3.2 Challenges**

When students drive their own learning, they are able to beneft from the multiple modes that are available to them. Working between and across modes may serve to support deeper learning and improve retention of key concepts. MML requires a high degree of agency, and provides opportunities for learners to select a focus or direction for themselves. This requires that learners develop the metacognition necessary to understand *how* they learn and also *when* to challenge themselves (Bezemer and Kress 2016; Moreno and Mayer 2007; Phuong et al. 2017; Sankey et al. 2010). The time needed for faculty to engage with students on learner activity design may be prohibitive in larger courses. It also may be diffcult to objectively assess learning when activities are varied across a student group.

### **2.3.3 Implementation Strategies**

Learner activities in an MML environment are student-driven, based on preferred modes. Still, faculty could develop a range of learner activities, mapped to course learning objectives, that can be completed in several modes to enable students to select an appropriate activity. These activities may be experiential and hands-on or operate in a virtual environment. For example, an assignment either could be openended in terms of how the learning is demonstrated such as a document, a presentation, or a video or could have a range of possible formats the students could choose from.

### **2.3.4 Research Questions**

Research questions in this area rest on the opportunity that exists to operationalize MML-based learner activities to promote student engagement. The MML literature clearly supports the shift from subject matter delivery to that of engagement, as well as the positive benefts of this. Workshop participants aptly pointed out that it will also be important to compare MML-based learner activities to other types of pedagogical frameworks for effcacy.

*All of Juan's learning activities are "hands-on" in the virtual classroom and include a series of science experiments that he can perform in the virtual space, at home (where he can record and post in the virtual classroom), or at a series of physical labs his university has partnered with. Throughout the course, Professor Aranda is working to ensure that the course content, learning activities, and support from other scientists and peers all work together to further his individual learning. Professor Aranda is aware that Juan's preferred modes of learning are in visual and auditory realms so she takes time to ensure that he is challenged to incorporate writing and presentation skills, as well as to spend time in the virtual space to drive the intersectionality of multimodal learning and to create more strength in other modes of learning and information consumption.*

### *2.4 Assessment*

Traditional means of assessment and evaluation have often revolved around tests and written assignments. Over time, educators have embraced new ways of assessing competency through action projects, multimedia presentations, and other types of practical application strategies. MML can be a helpful framework in the assessment strategy in a course or classroom environment that frees up the faculty and students for more creative ways to show mastery. No matter the type of assessment, formative, summative, or performance-based, designing assessments that invite different response options may free students from a specifc way of illustrating their knowledge. In this way, through MML options, learners are encouraged to produce demonstrations of their learning in modes that they have an affnity for. Moreover, providing suggestions for multiple ways to illustrate learning may open the doors to creativity and deeper levels of learning (William and Flora Hewlett Foundation 2016).

#### **2.4.1 Opportunities**

By designing assessments so that they can be completed in multiple modes, students can draw from their modally-driven learning to produce artifacts in the mode that they are most comfortable and familiar with. By reducing the cognitive load associated with using an unfamiliar or uncomfortable mode, learning and mastery can be more easily conveyed. Students could avail themselves of technological advancements to capture their learning as it evolves, ostensibly faster than higher education curricular design cycles. In addition, by opening up the classroom co-curricularly as discussed in section 2.5, assessments can be conducted by stakeholders outside of the classroom walls, providing helpful feedback from employers, community partners, and others that can help shape the student's learning in ways that may not be within the scope of what a faculty member knows or has experienced, personally. Ultimately, the opportunity to have multiple stakeholders providing evaluation and feedback affords faculty with more data to gauge learning and to plan to reduce gaps and increase levels of mastery. With more and consistent data streams through multimodal evaluation, the possibility for machine learning to aid in faster and more effcient grading also presents itself.

#### **2.4.2 Challenges**

Students might not choose a mode that effectively demonstrates their learning, or they may select a mode that the faculty member is not familiar with. Diverse outputs may challenge the use of rubrics or programmatic assessment plans that rely upon standardized artifacts or other uniform, objective measures given to students. Moreover, it may be very time-consuming for a faculty member to grade assessments produced in many modes because mastery may "look" different based on the mode in which it was produced. Students may also choose to demonstrate their learning in a mode that lacks adequate privacy or accommodations for students with disabilities—presenting concerns around assisting their selection of modal delivery as part of the assessment directions.

#### **2.4.3 Implementation Strategies**

In higher education, a culminating project or fnal assignment often requires learners to synthesize, analyze, and create content using a variety of modalities. Such projects, when closely aligned with learning outcomes, support relevant and authentic assessment that leverages digital communication strategies. An emerging practical model designed specifcally for multimedia instructional strategies, CASPA, contains fve components: *c*onsume, *a*nalyze, *s*caffold, *p*roduce, and *a*ssess. This model can be used to promote curriculum-based integration of multimodal projects for assessment in higher education courses. The CASPA framework for assignment design asks students to iteratively attempt and create culminating assessments throughout a course (Blum and Barger 2018). The framework allows students to experiment with different modes as they complete a culminating assignment. Faculty feedback and direction throughout the term could be used to enable learners to choose the best modal demonstration of their learning by the time the end product is due. Faculty will design assessment artifacts that are open-ended and driven by course learning objectives. No formal description of the end products or artifacts is provided to the student, though it may be appropriate to provide options to help students choose effective outputs or ones that protect the student's privacy and that are accessible to all students, regardless of their abilities. Faculty may also consider creating standardized metrics that would allow for cross-modal comparisons of learning products illustrating that there are multiple ways to achieve course aims.

#### **2.4.4 Research Questions**

Research questions in this area center on the affordances of some modes over others, and this remains an area for further research as identifed by workshop participants; perhaps some modes are more effective than others in the demonstration of learning. Furthermore, faculty modal preference is rarely discussed in the literature. Faculty who are evaluating artifacts outside their modal preference may not be able to objectively assess them due to the cognitive load required to adapt to new or unfamiliar modes.

*Juan and his peers are encouraged to locate and post content that aligns to each week's learning objective through a framework of information validation provided by the university. Students are encouraged to comment on these new content postings as a way of sensemaking and co-construction of knowledge. Experiments are the main source of assessment and evaluation in his course, but Juan also has several opportunities to identify and capture scientifc theories in his everyday life through a system of digital storytelling. This can include taking a picture and posting about an experiment, recording audio or video about seeing science in nature, or providing a summary of a scientifc experience through video or a text-based summary. Professor Aranda awards points to every learning capture that he is willing to submit in the course; in fact, all of the assessments in the course are documented as merely a list of learning objectives so that each student can demonstrate mastery through modes that they have an affnity for or modes that lend themselves to the objective being assessed.*

### *2.5 Co-curricular Activities*

While the idea that learning does not always take place in the traditional classroom setting has been accepted for many years (Dewey 1997), educators and students are now embracing the concept. Co-curricular learning refers to activities, programs, and learning experiences that complement, in some way, what students are learning in formal classes, but also serve to expand the relevancy and contextualization of that learning in the real world. Co-curricular contexts enable the learner to interact with material outside the classroom in such a way that they acquire new knowledge or apply a skill from one area to a new area. Increasingly, institutions of higher education are embracing co-curricular learning and its benefts to their students (Stirling and Kerr 2015; Turrentine et al. 2012).

#### **2.5.1 Opportunities**

By broadening learning to extend beyond the academy, students can build professional networks, enhance their learning through multiple perspectives, and more quickly build workforce-ready skills. Through co-curricular access to other educational stakeholders, students can engage and consume new learning in the mode they most prefer. Students could also participate and drive change in their community by working directly with business and community leaders to couple their learning with practical applications.

These co-curricular activities could take many forms, and in one scenario, educational stakeholders such as a venerable scientist or engineer would be purposefully contracted by the institution to participate in the virtual world associated with the course to:


Each scenario further extends the student's learning, infused with the relevancy and currency of a working professional in their feld of study.

#### **2.5.2 Challenges**

Many institutions of higher education do not have formal connections with the private sector that underrepresented students' need to beneft from co-curricular learning. Faculty may also be challenged with assessing which learning should happen outside the classroom and what should be inside the classroom (face to face or online), may struggle with how to integrate or sequence this type of learning with classroom-based learning. Federal credit hour defnitions may also limit co-curricular learning or make it diffcult to "count" it, so it would be additive, perhaps contributing to cognitive overload for students. Validation of co-curricular learning may be diffcult to assess as well.

### **2.5.3 Implementation Strategies**

Faculty can design course learning objectives tied to business and community interaction, where the choice of modalities would be driven by a student's individual preferences. For example, a course learning objective could read "Students will consult with a scientist to discuss what types of careers a science graduate can pursue upon graduation." Faculty could design course content and set the stage for interactions and discussions that engage educational stakeholders from outside the walls of the classroom.

### **2.5.4 Research Questions**

Research questions in this area center on the benefts and consequences of students' learning outside of the classroom. MML literature is seemingly quiet in this area, though it was a theme discussed in the workshop. There does remain a risk to the fdelity, meaning the degree to which application of MML can be replicated, across iterations of co-curricular learning programming. As co-curricular learning increases, capturing the outcomes and related learning that stems from MML will need to be developed and well understood by faculty and students as well.

*Each Tuesday night, Juan's whole class, his professor, and a group of scientists from all over the world gather to sit in the virtual classroom to discuss vital scientifc theory that will enable him to be more successful when he is attempting his experiments. These same scientists are working with Juan and Professor Aranda throughout the term to customize the learning and to be sure Juan is engaged in the most up-to-date practice. Juan is developing a strong network of future colleagues as he interacts with these external educational stakeholders.*

### **3 Conclusion**

While there are clear opportunities and affordances offered through the use of MML in the classroom, there are also inherent limitations to MML, particularly related to cognitive load. Too many modes and too much information can overwhelm learners (Moreno and Mayer 2007; Sweller et al. 1998). Furthermore, fundamentally shifting traditional teaching would require intentional, responsive, and on-going professional development so that teachers can develop multimodal literacies. One must also consider the cost and complexity (Bezemer and Kress 2016) of curating learning environments so that learners can choose from all of the varied types of modes. Though challenges do exist in the use of MML, the opportunities and affordances that exist through incorporating facets of MML in classroom teaching and learning far outweigh the challenges. Moreover, as student demographics continue to evolve into more and more diversity and technology continues to reshape learning interactions, instruction will inherently take on a multimodal nature.

*In the end, Juan fnishes his Introduction to the Fundamentals of Science course with a deep understanding of scientifc theory and its application in the real world. He also leaves the course with a newly formed group of mentors which includes Professor Aranda and the scientists who supported the course. He has also formed deep bonds with his peers and plans to connect with them in future courses. Lastly, he leaves the course with a new sense of how improving his writing and presentation skills can provide value not only in deepening his learning but in how he presents himself to the world outside of the university.*

### **References**


William and Flora Hewlett Foundation (2016) What is deeper learning? http://www.hewlett.org/ programs/education/deeper-learning/what-deeper-learning

World Health Organization (2011) World report on disability. WHO Press, Switzerland. http:// whqlibdoc.who.int/publications/2011/9789240685215\_eng.pdf?ua=1

**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# **Cross Reality (XR): Challenges and Opportunities Across the Spectrum**

**Cindy Ziker, Barbara Truman, and Heather Dodds**

### **1 Emerging Trends and Pedagogies**

Cross Reality (XR) refers to a group of emerging technologies such as virtual reality (VR), augmented reality (AR), and virtual worlds (VWs) that involve the use of 3D models/simulations across physical, virtual, and immersive platforms. The path to optimizing the use of XR in education is not always easy to navigate. However, with adequate support, XR has the potential to help faculty and students transcend the boundaries of the classroom by providing new types of environments for presenting and delivering instructional content and creating learning experiences with the power to develop unique communities of inquiry and practice.

Throughout this chapter, we will explore scenarios populated by a typical future student named Andi and share examples of her engagement with Cross Reality throughout her educational career. These scenarios are designed to illustrate what lies ahead in education, as these technologies become more and more ubiquitous, allowing students like Andi to move seamlessly within the Reality-Virtuality Continuum.

C. Ziker (\*) Ziker Research, San Jose, CA, USA e-mail: cindyziker@yahoo.com

B. Truman University of Central Florida, Orlando, FL, USA

e-mail: btruman@ist.ucf.edu H. Dodds

Independent Researcher, New York, NY, USA e-mail: heatherelizabethdodds@gmail.com

© The Author(s) 2021 55 J. Ryoo, K. Winkelmann (eds.), *Innovative Learning Environments in STEM Higher Education*, SpringerBriefs in Statistics, https://doi.org/10.1007/978-3-030-58948-6\_4

### *1.1 Defnitions*

For the purpose of this chapter, Cross Reality or XR refers to technologies and applications that involve combinations of mixed reality (MR), augmented reality (AR), virtual reality (VR), and virtual worlds (VWs). These are technologies that connect computer technology (such as informational overlays) to the physical world for the purposes of augmenting or extending experiences beyond the real. Especially relevant to the defnition of XR is the fact that this term encompasses a wide range of options for delivering learning experiences, from minimal technology and episodic experiences to deep immersion and persistent platforms. The preponderance of different terms for slightly different technologies indicate that this is a growth area within the feld. Here we provide a few defnitions of these technologies.

MR—Mixed reality refers to a blend of technologies used to infuence the human perception of an experience. Motion sensors, body tracking, and eye tracking interplay with overlaid technology to give a rich and full version of reality displayed to the user. For example, technology could add sound or additional graphics to an experience in real time. Examples include the Magic Leap One and Microsoft HoloLens 2.0. MR and XR are often used interchangeably.

AR—Augmented reality refers to technology systems that overlay information onto the real world, but the technology might not allow for real-time feedback. As such, AR experiences can move or animate, but they might not interact with changes in depth of view or external light conditions. Currently, AR is considered the frst generation of the newer and more interactive MR experiences.

VR—Virtual reality, as a technological product, traces its history to approximately 1960 and tends to encompass user experiences that are visually and auditorily different from the real world. Indeed, the real world is often blocked from interacting with the virtual one. Headsets, headphones, haptics, and haptic clothing might purposely cut off all input except that which is virtual. In general, VR is a widely recognizable term, often found in gaming and workplace training, where learners need to be transported to a different time and place. VR experiences in STEM often consist of virtual labs or short virtual feld trips.

VW—Virtual worlds are frequently considered a subset of VR with the difference that VWs are inherently social and collaborative; VWs frequently contain multiple simultaneous users, while VRs are often solo experiences. Another discrimination between virtual reality and virtual worlds is the persistence of the virtual space. VR tends to be episodic, with the learner in the virtual experience for a few minutes and the reality created within the experience ends when the learner experience ends. VWs are persistent in that the worlds continue to exist on computer servers whether or not there are active avatars within the virtual space (Bell 2008). This discrimination between VR and VW, however, is dissolving. VR experiences can be created to exist for days, and some users have been known to wear headsets for extended periods of time. Additionally, more and more VR experiences are being designed to be for game play, socialization, or mental relaxation. The IEEE VR 2020 online conference and the Educators in VR International Summit 2020 offered participants opportunities to experience conference presentations in virtual rooms as avatars while interacting with presenters and conference attendees (see Sect. 2.5 for more information).

Relevant to defning VWs, Correia et al. (2016) conducted a meta-analysis on the potential of using virtual worlds for learning and training (p. 407), while Mann et al. (2018) proposed a different defnition of XR with variations of real and synthetic applications that make up their Multimediated Reality Continuum (p. 12), for another term. According to Mann et al. (2018, abstract), "As a new feld of study, All Reality is multidisciplinary. We must consider not just the user, but also how the technology affects others, e.g. how its physical appearance affects social situations, and how sensor-based reality (e.g. wearable and implantable cameras in the smart city) affects privacy, security, and trust. All Reality includes Virtual Reality (VR), Augmented Reality (AR), X-Reality (XR), X-Y Reality (XYR), and Mixed, Mediated, etc. realities (MR)."

CVEs—Collaborative virtual environments are communication systems in which multiple interactants share the same three-dimensional digital space despite occupying remote physical locations (Yee and Bailenson 2006).

Embodiment—Embodiment is defned by Lindgren and Johnson-Glenberg (2013) as the enactment of knowledge and concepts through the activity of our bodies within an MR (mixed reality) and physical environment (p. 445). Embodiment can also be experienced as a suspension of disbelief while using avatars (digital individual representations) in a fully online virtual world. In fact, embodiment can be experienced as group phenomena that may lead to the development of communities of practice (CoP). Truman (2014) studied the relationship of embodiment in collaborative virtual environments (CVEs) and its refexive properties of the primary avatar (learner/user) related to the theoretical framework of transdisciplinarity (p. 59). Attachment to avatars resulting in embodiment was not always found, suggesting that some individuals may be incapable of embodied experience (Truman 2014: 232).

The deeply immersive nature of some forms of XR has had a powerful effect in studies of multifaceted empathy. Manipulating the full environment around a learner, including sight, sound, smell, taste, pressure, heat, and texture promises to be signifcant in impact. "VR feels real, and its effects on us resemble the effects of real experiences. Consequently, a VR experience is often better understood *not as a media experience, but as an actual experience,* with the attendant results for our behavior" (Bailenson 2018: 46). Studies have shown that learners can quickly adopt an avatar as a personal representation of their own bodies; this effect is known as body transfer. When learners accept a digital object as a real object, the Proteus effect (Yee and Bailenson 2007) has been achieved, and manipulations of the digital object are biochemically accepted as real (Fox et al. 2016). Each of these defned forms of technology, MR, AR, VW, and XR, provides learners with real experiences.

### *1.2 Key Trends*

XR is becoming ubiquitous across society in domains such as entertainment, healthcare, government, military, education, and industry training for manufacturing and automation. Three key trends that are infuencing the adoption of XR include:


Taking these trends into consideration, the following section explores XR learner activities with respect to STEM content and instructional design.

### **2 Use of XR in 2026**

During a 2018 X-FILEs Workshop, educators, researchers, and professionals gathered to discuss and hypothesize about the future of XR within STEM higher education. Ideas generated from this workshop formed the basis for this chapter.

### *2.1 Content Presentation*

Faculty and instructors typically think about instructional content to present, chunk, and sequence in their courses. Incorporating XR for graded assignments requires departmental and ideally institutional support to make the most of purchasing and design decisions. Appropriate XR applications can provide the foundations for new types of learning environments and experiences. XR can also bring users together creating new communities of inquiry and practice. With adequate support, XR can help users transcend the boundaries of classroom and web-based instruction. XR applications are discussed in this section in terms of the context of faculty demands and institutional resources based upon variables of size, staffng, need to scale, and emphasis on research focus. Taxonomies, toolkits, and means to obtain data valuable for creating baseline analytics using XR all support scholarship and research regardless of institution type.

#### **2.1.1 Opportunities**

XR presents an opportunity for an entirely new approach to learning design that can leverage the principles of transmedia learning. Transmedia learning provides a guide for educational experiences in XR, primarily because it offers the learner a real or simulated world with extra layers of overlaid information. As the learner is often at the center of these XR experiences, transmedia learning builds upon creating stories across experiences and devices where the learner is the hero possessing self-determination and self-regulation. The use of game-based learning, serious games, and gamifcation is included under Raybourn's (2018) transmedia learning framework, which provides heuristics for designing and personalizing transmedia learning. According to Raybourn (2018), "The transmedia learning framework (TLF) is meant to provide ways to think outside the box about learning, or what we normally consider education. The TLF is a way to employ new media in a way that is going to augment your learning experience, to include engaging yourself at the neurological level" (Raybourn 2018: 1).

### **2.1.2 Challenges**

Authoring 3D content has become easier, but is not easy enough for fast production by most users and especially faculty. Library leadership is needed to track 3D open educational resources and provide authority for managing data into existing information schema and standards. Staff support for matchmaking of needs and content will require signifcant commitments of time to experiment and play with XR possibilities. Partnering with innovators on campus already doing XR can result in valuable informal professional development offerings that include panel discussions, hands-on demos, and tracking of effective uses.

### **2.1.3 Implementation Strategies**

Instructors should be very clear about why they are selecting XR learner activities as opposed to other modes of teaching and learning. Apparent gains in learning, or increased learner preference, by utilizing XR can be attributed to the novelty effect (Clark 1985) if compared to non-robust, non-immersive learning choices. Aldrich (2009) suggests that higher interactive virtual environments (HIVE) do work as learning activities because immersive activities stimulate emotional involvement, which is necessary for learning to move from working memory to long-term memory.

### **2.1.4 Research Questions**

Future research relevant to content preparation in XR should address the following:


### *2.2 Interactions and Communications*

#### **2.2.1 Opportunities**

Some of the early evidence that interactions with XR applications are effective comes from corporate training (Maxwell and McLennan 2012), which suggests that adult learners experience enhanced transfer, possibly due to situated learning and because such training often has an immediate real-world application. This point should not be lost; Adult learners appear to be the frst to have larger-scale learning success when focusing on the application of learning.

Collaborative virtual environments in the workplace allow for co-design and codevelopment to embed scenarios or situational problem-solving questions into immersive 3D environments. Microsoft's HoloLens 2 is innovating in the workplace environment by bringing what they call "instinctual interaction" into XR use (Upload 2019). The company is currently expanding into the workforce market, hoping to tap up to two billion frontline workers with HoloLens (Weise 2019).

Researchers are still in the nascent stages of discovering how the power of XR can be optimized. While studies by Yee and Bailison and others have examined the role that XR activities can play in behavioral skills like eliminating negative stereotyping, fostering empathy, and increasing scientifc inquiry thinking processes, research on the concept of mirror neurons as the conduit for behavioral skills is maturing past conceptual to actual stages (Rizzolatti and Craighero 2004; Fox et al. 2016). Early data is pointing to the fact that the brain believes that XR activities really happen and as such, the internal biochemical changes of learning are the same as the real experience (Bailenson 2018). For example, XR can provide the benefts of feld trips without requiring travel. Further XR ideas include: manipulating space and time; such as looking inside of a supernova; using expensive lab equipment at little cost; and scenarios as yet unimagined that are too dangerous or diffcult. Thus XR can bring a new range of experiences to the learners of tomorrow.

For health, XR can also have a positive impact in the form of aiding cancer treatments (Chirico et al. 2016). The arrival of the COVID-19 pandemic and the closing of contact sports are causing a rise in the use of eSports. If, following the mirror neuron research from Rizzolatti and Craighero (2004), the mind believes that what the eyes see is actually happening, it is logical to assume that engaging in eSports could have a positive impact on physical health. During eSports, the body is being fooded with the same biochemicals as traditional athletes. Increased physical health could be measured with two blood pressure tests, one at each end of an academic semester for eSports athletes.

Research fndings on empathy studies using XR are mixed at this time. Herrera et al. (2018) found that there are positive long-term impacts via VR experience on the topic of homelessness. However, a VR experience with color-blindness resulted in no short-term empathy increases. In some cases, exposure to VR decreases empathy. Companies like Equal Reality (https://equalreality.com/) are using virtual reality for diversity training as a way to impact perspectives on privilege and power.

#### **2.2.2 Challenges**

XR platforms also have signifcant accessibility concerns that need to be addressed with both hardware and software. Many experiences require full 360° movement and motion controller input and do not have any accessibility features built in to allow for a wider variety of users. There are a variety of ways that this affects users with a range of abilities (Ryan 2019). Opportunities to create compelling social XR applications are exciting but must not come at the cost of alienating users.

When we introduce multiple users into an XR experience, we introduce the possibility for a whole host of issues that are not encountered in single user educational experiences. We encounter signifcant issues related to safety, accessibility, and privacy that must be accounted for very early on in the design process of any multiuser educational experience (Reilly et al. 2014).

When using existing social virtual reality experiences, female users reported struggling "in a social sense, dealing with strangers and encountering behaviors that they equated to real life scenarios where they had experienced harassing behavior. In the social VR worlds they visited, they encountered firting, a lack of respect for personal boundaries, socially undesirable behavior and in the end, most were not interested in using these platforms to meet strangers" (Outlaw and Duckles 2017). The social problems we deal with in location-based educational environments follow us into XR. Outlaw and Duckle's study of women in social VR experiences highlights the issues of personal and social safety faced by those who experience a lack of social safety outside of VR. In VWs, the phenomenon known as griefng could negatively impact digital systems in the form of attacks that reduce digital commerce (Bakioglu 2009).

#### **2.2.3 Implementation Strategies**

Educators should be looking for ways to leverage existing tools to create meaningful interactions using XR technologies while continuing to explore new opportunities that have not yet been developed. "Remote real-time collaboration with socio-technical systems and dialogue tools aimed at promoting collaborative learning and deepening the space of debate and producing epistemic interactions is in the interest of designers, engineers and educators around the globe. This calls for enabling more platforms for real-time collaboration between teams and networks" (Wendrich et al. 2016). XR technologies will also provide new forms of leadership engagement between students, faculty, and institutions.

Using commercially available applications, like Altspace or Bigscreen, educators already have the opportunity to present more traditional digital content (video, photos, presentations, etc.) to students in an XR environment. Utilizing these types of systems allows for a low barrier to entry that requires minimal customization or onboarding. Educators can quickly adapt existing content to be used.

For universities that already have existing XR equipment, labs, and studios, there are great opportunities to invite students deeper into the design process and to create collaborations with other universities so students can interact with other students who might have similar access. It should be a top priority for institutions with these resources to leverage them, in order for students to connect with their peers at other universities in XR experiences.

#### **2.2.4 Research Questions**

Future research regarding interactions and discussions using XR should address the following:


### *2.3 Learner Activities with XR*

Prior to 1910, STEM classes were taught primarily with direct instruction (Olson and Loucks-Horsley 2000). This started to change with John Dewey's emphasis on active learning activities that occurred outdoors, outside the science classroom. Over time, however, STEM teachers wanted activities that would be active in cognitive load but could also be done indoors to eliminate problems of weather, distance, and availability (Brown 2003). The space-era Commission on Science Education of 1964 has strongly infuenced STEM instruction for the past half-century by requiring the inclusion of experiential learning approaches, known as laboratory courses (labs), in addition to rote learning of the scientifc method (Livermore 1964). Virtual labs provide equal learning gains when compared to in-person labs (Faulconer and Gruss 2018). The emphasis in XR, however, is less on content knowledge and more on inquiry scientifc thinking processes and behaviors and then collaborating and communicating. Consequently, VR, VW, AR, and all forms of XR have a history of providing the valid and immersive active learning activities required within STEM domains (Nelson and Ketelhut 2007).

#### **2.3.1 Opportunities**

Designing with XR requires a fundamental shift to a Human-Centered Design philosophy that involves putting human needs, capabilities, and behavior frst (Jerald 2018: 15). XR provides the opportunity to experience just-in-time *immersive*, experiential learning that uses concrete yet exploratory experiences involving senses that result in lasting memories. Here we discuss opportunities for social applications with XR.

XR learner activities are usually created for individual use, which may or may not need to be simultaneously experienced as a class together at the same time or place with the instructor. Activities can be designed into instruction with VR headsets, high-resolution screens, smartphones, or other solo technological devices for use inside and outside of the classroom. Feedback is also often individualized (Lynch and Ghergulescu 2017). Multiple characteristics contribute to the growing ubiquity of XR.

STEM XR learner activities often have these characteristics:


Some published examples of the use in XR learner activities within STEM include:


#### **2.3.2 Challenges**

The technological challenges of the use of XR in learner activities should be acknowledged. There will always be frst-use hesitation from learners, and they may tend to prefer more traditional activities until the newer technological interface becomes intuitive (Waldner et al. 2006). VR head-mounted displays can be burdensome with tethering cables and tight-ftting headsets, and if technology is shared or passed from learner to learner, there are the added challenges of the sharing of germs and body moisture. Further developments already coming in VR equipment include the use of temperature, pressure, and scent which can potentially disturb the learner. Bailenson aptly identifes the VR dangers of poor behavioral modeling, simulator sickness, eyestrain, and reality blurring (2018). VW can be addictive and can contain signifcant non-educational activities not suitable for young learners. AR can cause distractions if used walking down the street or while driving.

XR learner activities should be designed to increase accessibility to all learners by utilizing the XR strengths of multi-layered information display. However, some learners will be held back from full XR activity by visual, physical, and social abilities such as stroke, vertigo, epilepsy, or age-related reaction time. It should also be noted that the encompassing nature of VR headsets might create some discomfort or danger for any learners as they can no longer fully see and control their body and body space.

Keeping young learners focused while using technology can be a challenge. Tallyn et al. (2005) found that AR could bring media-based learning to life when used in concert with paper-based learning as an instructional theme through a learning experience. Learners focused on working through a worksheet or workbook could reap the benefts of AR while not going off-track with their learning.

#### **2.3.3 Implementation Strategies**

Combining VR applications such as virtual worlds with other XR applications can serve as a multiplier effect for research where communities of practice span across international boundaries. User support communities are robust in the virtual worlds of Second Life and OpenSimulator where institutional representation often occurs at the grassroots level. Immersive conferences such as the Virtual Worlds Best Practices in Education and the OpenSimulator Community Conference enable sharing of R&D across XR applications. Network members in organizations such as the Immersive Learning Research Network and Educators in VR overlap the disciplines of education, technology, and industry in order to further the research-based effective application of XR technologies.

Examining how augmented reality and virtual reality social systems differ, how they may become blended, and the necessary design considerations, we can see how as these technologies converge there will be new possibilities for exploration and collaboration (Miller et al. 2019). Already there are social XR platforms, like AltspaceVR and Bigscreen, that allow users with access to a variety of XR devices to share virtual spaces and content from their own devices with each other.

In juxtaposition to the experiences of users in 3D worlds using 2D screen, or those using video-based telepresence systems, XR social platforms allow for users to feel as if their "virtual self is experienced as the actual self" (Aymerich-Franch et al. 2012). This sense of social presence (Oh et al. 2018) offers new areas of exploration for users of this technology. We can now consider not just adapting existing educational content, but conceiving of entirely new ways of educating that leverage an embodied experience and allow for a huge range of augmentation.

The affordances of XR environments include opportunities for geographically dispersed learners to learn in an environment similar to their traditional classrooms without forfeiting the ability to learn at their own pace and in their own time zone (Olasoji and Henderson-Begg 2010). Computer-supported collaborative work (CSCW) or team-based projects involve looking at the individual and collaborative potentials based on individual and shared contributions. Ward and Sonneborn (2009) note that methods of assessing creative problem-solving in groups will need to include measures of how people personalize their learner group contributions and the effect that such individualized collaboration has on the quantity and quality of ideas produced.

#### **2.3.4 Research Questions**

Future research in learner activities using XR should include:


*Since her earliest school years, Andi has been going on virtual feld trips that utilize basic VR headsets in the classroom and immersive rooms at museums. Her initial experiences involved little interaction and were more look-and-see experiences, such as taking a feld trip to the Great Wall of China while using a VR headset. Gradually, Andi's teachers added interactive experiences during which Andi learned to navigate controllers to draw, assemble, and bounce virtual objects in small group activities with her classmates. By middle school, Andi was a confdent presenter in virtual reality platforms and often played social VR games outside of school. In 6th grade, Andi wrote a report on her virtual visit to Mars, ending with her resolution to become an astronaut.*

*In high school, Andi enjoyed participating in eSports and played on an all-female team with the potential to win \$30,000 in college scholarships. She also engaged in weekend virtual science workshops on biofuel creation sponsored by corporations looking to increase students' scientifc collaboration skills of observation, cause-and-effect monitoring, and communication of results. Andi regularly joined virtual check-ins with scientists at international museums and connected virtually with astronauts at the National Aeronautics and Space Administration and the International Space Station.*

### *2.4 Assessment*

#### **2.4.1 Opportunities**

The immersive nature of XR has the potential to provide engaging assessment experiences that represent real-world activities more accurately than traditional paper and pencil measures. Still, assessment of learning in virtual environments typically employs the use of external instruments that are administered outside of the virtual experience. This review examined the literature related to the types of assessments used in the context of XR, with a focus on methods for assessing skills, evaluating work products and performance, and analyzing log fles.

Conventional assessments, such as pre and post, multiple-choice items, short answer items, and open-ended questions, are frequently used to assess the effectiveness of learning in a virtual environment. Ketelhut et al. (2006) created a pre-post, self-designed content test to assess knowledge of science inquiry and process skills during an investigation of a science curriculum, implemented through a virtual environment called River City. Labster's HMD and desktop VR lab experiments include embedded multiple-choice questions and a point system to track students' scores as they progress through experiments. Metrics, such as assessing the number of questions asked in a lab experience within and without a pre-lab XR experience, offer additional possibilities for collecting performance measures.

In contrast, evaluating learner-created work products developed directly within immersive virtual environments provides opportunities for assessing authentic learner performance. Rose (1995) leveraged the psychological theories of information processing and constructivism to identify specifc approaches for measuring learning in VR that included performance tasks, such as world building, problemsolving, and the evaluation of the quality of fnal products. Olasoji and Henderson-Begg (2010) studied a Second Life course that required learners to produce a summative assessment containing a scientifcally accurate depiction of a biological molecule or bioinformatic concept. Work products such as these can be evaluated to produce data concerning the learner's performance through a rubric, a scoring guide, or an automated scoring procedure (Mislevy et al. 2017).

The Virtual Performance Assessment (VPA) Project, created by the Harvard Graduate School of Education, is an open-ended 3D immersive virtual environment designed for performance assessments of science inquiry skills in multiple virtual scenarios. Students engage in authentic inquiry activities and solve scientifc problems by navigating around the virtual environment as avatars, making observations, interacting with non-player characters (NPCs), gathering data, and conducting laboratory experiments (Baker and Clarke-Midura 2013). VPA enables the automated and non-intrusive collection of process data (event logs or logged actions and behaviors) and product data (students' fnal claims), facilitating the capture and assessment of science inquiry in situ (Jiang et al. 2015). These examples represent advancements toward relevant methods for assessing learning within virtual settings.

A frontier of educational assessment is the development of automated methods of evaluating log fles, through which cognitively meaningful patterns and features of work are detected and characterized as observations (Mislevy et al. 2017). The dynamic nature of the computer system allows recording of learner interactions and data gathering in the background as the learner moves through an XR experience (Rose 1995). Log fle analysis of this type of data is often referred to as stealth assessment, which has been used to measure problem-solving and spatial skills (Shute et al. 2015), assess causal reasoning in the World of Goo (Shute and Kim 2011), and assess systems thinking in Taig Park (Shute et al. 2010). An advantage of stealth assessments is that data can be collected without disrupting learner fow within the XR experience (Shute et al. 2015).

#### **2.4.2 Challenges**

Assessments in XR present several challenges that can impact the integrity of the inferences that can be made from results. Examples include interference due to headset discomfort, lack of familiarity with navigating the virtual world, and gaps in alignment between the assessment method and the content being assessed. Construct-irrelevant variance caused by distractions in the virtual environment can negatively infuence learner behavior. In multi-user environments, issues around trust, security, and identity can arise, given that users can create multiple accounts and avatars (Warburton 2009).

In the context of stealth assessments, the complexity of log fle coding schemes can make analysis diffcult, while a lack of aligned external validation instruments can pose additional reliability challenges (Wang et al. 2015). According to Wang et al., "If a researcher wants to create stealth assessments within an existing commercial game, the frst step is to make sure that either the coding in the log fles is simple enough to understand, or the coding scheme is available from the game developer so that changes can be made to the information that is being captured" (2015: 5). It is important to note that not every logged interaction or tracked gaze represents evidence of learning. Validating accurate scoring systems for virtual activities that have the ability to predict performance in real-world settings presents a formidable challenge for XR assessment developers.

#### **2.4.3 Implementation Strategies**

Implementing assessments that have the capacity to elicit evidence of what learners know and can do requires close alignment between learner interactions and the expected outcomes (Code et al. 2012). By leveraging the power of evidence-centered design (ECD) (Mislevy and Haertel 2006), assessment developers can create an evidentiary assessment argument supported by a conceptual assessment framework that includes task specifcations, evaluation procedures, and measurement models. In a virtual environment, provisions for accommodations and the use of the principles of universal design for learning (UDL) are critical for ensuring accessibility and equity during assessment implementation.

#### **2.4.4 Research Questions**

Overall, the development of reliable assessment methods within XR is still in the early stages. Future research is needed to examine the following research questions:


### *2.5 Co-curricular Activities*

Here we explore the experience of learners in the context of academic endeavors beyond classroom interactions that include socialization, employment, and responsibilities that provide opportunities, challenges, and potential for future research *To mitigate safety risks during Andi's frst XR experiences, her teachers limited the viewing to under 2 min. Teacher assistants served as spotters, and the space for engaging with the hardware was cleared of hard surfaces. Alternative experiences were set up in 2D for any learner.*

*One of Andi's teachers wisely included safety precautions in her XR choices and picked a platform where the students were not tracked and did not need to log-in with authentic credentials. When the learner's session ends, the entire session is deleted from the host's servers. At the high school computer lab, the instructional technology specialists set up all of the VR devices to be controlled and monitored from a main station and ensured that devices could not be accessed outside of the school network without specifc teacher permission. Andi's instructors selected learning experiences where teleportation was used rather than abrupt motion, to reduce incidences of motion sickness.*

*Routinely, XR devices were disinfected daily during the school year. Learners were taught to wipe down the headset and controllers before using them. All parents were informed of the planned instructional use of XR through the year, and opt-in signatures were requested.*

*Andi's early choices within XR environments were limited to looking at a spot, clicking on something, a small range of "right click" alternative choices, and click and drag. As she gained confdence, she navigated multiple screens (e.g., in XR and in browser) at the same time, learned fy commands, combined virtual objects, and panned around 3D depictions. Due to products like Unity and other 3D content creators, Andi was able to create unique XR experiences with new textures and objects. Her instructors used her content creations within XR as evidence of learning, rather than relying on traditional forms of assessment outside of XR.*

using XR. Testing and adoption of commercial XR applications by learners enables instructors to harness new data that is useful for providing analytics of learner performance while supporting motivation to engage in STEM. The National Academies of Sciences, Engineering, and Medicine (2016: 95) reported that the use of cocurricular programming can affrm students' self-perceptions of competence to mitigate impacts of a stigmatizing STEM academic culture.

#### **2.5.1 Opportunities**

The pervasive use of XR is leading to the creation of new forms of partnerships and the potential for mass collaboration. More sustainable open-source software communities are examples of collaborations that build ecosystems for virtual world platform development. Further expansion of these collaborations is recommended to advance the feld of XR.

Obtaining value from the use of XR applications requires new interdisciplinary collaboration across departments, campuses, and partnerships with industry to manage complexity in pursuit of improved learning outcomes. Virginia Tech created a report for envisioning the campus of the future where organization and reorganization will be required among stakeholders including the greater community to fuse intellectual and co-curricular life as Human-Centered Smart Environments (HCSE). Figure 1 illustrates a vision for technology-enabled, interdisciplinary participation in societal needs that create new forms of learning opportunities. The emphasis in this diagram is that collaboration from multiple sources is necessary in humancentered environments. This collaboration needs to be made and remade in successive cycles in order for the positive impacts to be effective. For example, maker-spaces and research efforts on college campuses need to collaborate with industry to make the results widely available.

Virginia Tech's HCSE plans include Living Laboratories that are "... experiential environments wherein students and researchers can engage in learning, discovery, and innovation through interactions with real-world systems and communities. Living labs incorporate 'smart' autonomous systems that range from furniture and wearable technology to buildings and the surrounding landscape. These systems will assist humans in multiple ways that include supporting wellness, collaboration, solitude, inspiration, and disruption" (Hundley 2016: 11). The Virginia Tech HCSE

**Fig. 1** Hundley (2016: 8) integrated innovation hubs

model represents the framework for a type of immersive, digital learning commons that can incorporate existing instructional and information software systems, designed for the most common XR applications used across the institution.

#### **2.5.2 Challenges**

The feld of XR is in need of the creation of standards and norms for implementing technologies in educational settings, which could provide support for and accelerate their adoption by higher education institutions. XR applications currently do not have universal standards that enable interoperability and seamless integration into existing software. Professional associations such as the IEEE's IC Industry Consortium on Learning Engineering (ICICLE) (https://www.ieeeicicle.org/) XR working groups (https://www.ieeeicicle.org/xreality-sig) are investigating how to make the user experience more seamless and transparent. Potentially, if universities coordinate to design and develop communities of practice using XR, standards will develop faster for interoperability, building a robust ecosystem. The pervasive use of XR is leading to the creation of new forms of partnerships and the potential for mass collaboration. More sustainable open-source software communities are examples of collaborations that build ecosystems for virtual world platform development. Further expansion of these collaborations is recommended to advance the feld of XR.

#### **2.5.3 Implementation Strategies**

Recommendations for higher education institutions and faculty seeking to advance the use of XR in co-curricular instruction include:


During the COVID-19 pandemic, educational institutions of all sizes implemented XR as a social connection strategy. Platforms such as Discord, Mozilla Hubs, AltspaceVR, and VirBELA which were previously used only for co-curricular meetings and game spaces became more popular. The power of these platforms *As a college senior, Andi has experienced several courses in virtual worlds and designed her dorm room using the Walmart AR app to make furniture choices. Training for her current job has included virtual simulations that address complex problems through global collaborations and negotiations among individuals and large groups in virtual settings. She regularly attends virtual conferences that connect her with thousands of attendees in virtual spaces, such as Altspace, Mozilla Hubs, and VirBELA*. *Utilizing industry and media contacts, Andi can include any of her family and friends in XR activities because she can fnd XR resources that work for a range of ages and abilities. Many XR experiences have text, as well as icon labels so that Andi can engage with learners that need language-fexible choices. Andi's father often engages in VR training before he visits a new building site, while her mother uses XR training to practice empathy skills with her fellow emergency medical technicians. For Andi and her family, XR has become a common part of daily life.*

was leveraged to conduct virtual global conferences, such as the Immersive Learning Research Network 2020 Conference, the IEEE VR 2020 Conference, and the Educators in VR 2020 Conferences. These events engaged thousands of students, educators, and industry practitioners from around the world in virtual convenings that demonstrated how XR resources can be used for the purpose of sharing knowledge and professional networking, when traditional conference formats are impossible.

#### **2.5.4 Research Questions**

Recommendations for areas of future research related to co-curricular XR include the following research questions:


### **3 Conclusions**

As posited in the chapter introduction, the infuence of the entertainment sector, new collaborations between technology and business, and the ubiquity of the Internet of Things all indicate that the required technology for XR might already be in American homes, classrooms, and workplaces in the form of smartphones, computers, or game systems. "VR alone is expected to reach \$60 billion in 2020" (Bailenson 2018: 9), and AR is expected to reach \$60 billion in 2020 (Porter and Heppelmann 2017). With the COVID-19 pandemic, institutions of all sizes are considering the incorporation of XR experiences.

In conclusion, while there are many opportunities for students and faculty who are able to leverage the benefts of XR to enhance coursework, a variety of obstacles may hinder scaling. Optimizing the use of XR in higher education requires the support and resources of an interdisciplinary community of committed professionals from education, government, and industry who will work together with researchers to overcome the existing challenges that limit adoption. The development of common standards could advance this effort. There is also a need to invest in future research regarding the implementation and the assessment of the effectiveness of XR on learning. Technological and practical solutions are possible through the collaboration of experts and the fnancial investment for research in this feld.

### **References**

Advanced Distributed Learning Initiative (2019) About. https://www.adlnet.gov/about


Association for the Advancement of Computing in Education (AACE), Denver, pp 1604–1616. https://www.learntechlib.org/primary/p/40960/


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# **Artifcial Intelligence and Machine Learning: An Instructor's Exoskeleton in the Future of Education**

**Stephanie E. August and Audrey Tsaima**

### **1 Emerging Trends and Pedagogies**

### *1.1 How Do Artifcial Intelligence and Machine Learning Appear in Interactive Learning Experiences?*

Technology is transforming how we solve complex problems, as well as how we share information. In this chapter, we look at an innovative learning environment from the perspectives of an enrolled student, a teaching assistant, and the professor of a fuid dynamics course with 100+ enrolled students. The scenario and research provide insight into the value of incorporating artifcial intelligence and machine learning into the learning experience.

#### **Student**

Halfway through the pursuit of their undergraduate degree in chemical engineering, Alex Rhimes, age 20, from Baltimore, Mary's Lake, was planning on taking the foundational fuid dynamics class—the most notoriously diffcult class in the major. The learning management system and recommendation engine used by the university suggested taking this course early based on Alex's good grades and internship experience. Like most students, Alex logged into ratemyprofessors.com before selecting the class. With ratings in

S. E. August (\*)

A. Tsaima BetterUp, San Francisco, CA, USA e-mail: audrey.tsaima@betterup.co

Department of Computer Science, Loyola Marymount University and California State University, Los Angeles, CA, USA e-mail: august@acm.org; saugust@lmu.edu

<sup>©</sup> The Author(s) 2021 79

J. Ryoo, K. Winkelmann (eds.), *Innovative Learning Environments in STEM Higher Education*, SpringerBriefs in Statistics, https://doi.org/10.1007/978-3-030-58948-6\_5

the high 4s, Alex tabbed over to the university course site and clicked the big blue "Register" button on the screen. According to the reviews, Prof. Gomez went above and beyond to create a highly personalized environment for each student. As soon as Alex registers, an email notifcation is received: Pre-Course Simulation Game.

#### **Instructor**

In Charrysville, Virgonne, Dr. Riley Gomez, second year associate professor, wakes up early on the frst day of class to check new emails. Rolling over in bed groggily, Prof. Gomez reaches for the phone on the bedside table, scrolling past the unfltered junk emails. Prof. Gomez is anticipating notifcations from students submitting their last-minute survey responses to the self-assessment exercise shared a week ago. As student enrollments have increased and class sizes increased, Prof. Gomez started incorporating artifcial intelligence and machine learning methods into the classroom environment. It was the only feasible way to reach the 100+ students enrolled in the fuid dynamics class.

"Lecture-style classes with the sage on the stage are a thing of the past," explains Prof. Gomez. "Using algorithms is the most effective way to manage classes in the face of increased enrollment."

Prof. Gomez swipes left, left, and down in order to load the most recent results. It looks like the class is spread all over the place with experience levels and interest in fuid dynamics. It is not uncommon for students to drop out of this class and fail to persist. However, Prof. Gomez is adamant about ensuring that every student feel supported and hopeful that the simulation assessment provided an opportunity for students to learn some basics before the frst session. The left chart on the dashboard shows passive traits that were monitored during the simulation. Indicators such as eye movements and facial expressions are tracked in blue and orange. The middle chart on the dashboard illustrates anticipated knowledge gaps and opportunities for support based on data mined from last year's class.

The role of artifcial intelligence (AI) in US education is continuing to expand (*Artifcial Intelligence Market in the US Education Sector 2018-2022—Key Vendors are Cogni, IBM, Microsoft, Nuance Communications, Pixatel, and Quantum Adaptive Learning—ResearchAndMarkets.com* 2018). As education moves toward providing customized learning paths, the use of artifcial intelligence (AI) in learning systems increases, creating scaffolding that extends the ability and reach of an instructor (Tsinakos 2006) much as a physical exoskeleton combined with augmented reality enables a worker to see more than what is in front of them, and accomplish tasks they are not able to complete on their own (Srinivasan 2018). Chatbots (Bradeško and Mladenić 2012; Fonte et al. 2016; Albayrak et al. 2018; Eicher et al. 2018), autograders (Wang et al. 2018; Kyrilov 2014), and systems that passively monitor and then direct student progress (Paaßen et al. 2018) use AI,

machine learning (ML), and deep learning technologies to store and process data and then communicate it to students and instructors. This exploitation of AI in education requires substantial funding and time for research, implementation, and assessment for the education community to understand the effcacy of the technology and its role in student persistence and subsequent on-the-job performance or success in graduate studies (Marr 2018; Polachowska 2019).

**Artifcial intelligence** is the study of how to make computers perform tasks that appear to require intelligence when performed by humans. Machine learning and deep learning fall under this broad defnition of artifcial intelligence. **Machine learning** focuses on parsing and analyzing data in an automated fashion, without human intervention, to learn models for decision-making. Machine learning is considered a data mining technique. An algorithm that clusters data according to its similarities and differences is an example of machine learning. **Deep learning** is a subset of machine learning that relies on networks that mimic the way the human brain processes data and creates patterns to acquire decision-making ability.

A **chatbot** is a software program that converses with a human user. Chatbot ability ranges from those that conduct a shallow dialogue over a broad range of topics to those with deep knowledge and conversational ability over a wellscoped domain of discourse. The best are diffcult to distinguish from human conversants. **Autograders** are software programs used to evaluate work produced by students with little or no human intervention. They can perform tasks ranging from scoring multiple choice tests to analyzing and grading essays. The fndings produced by autograders range from binary (correct/ incorrect) to conceptual feedback. Automatic review of essays is often combined with human review of the essays, with subsequent closer human examination of an essay if the automated and initial human results disagree. **Passive monitoring and guidance** can be integrated into an online learning system to compare a student's activities to expected behavior. An instructor might learn through the system that several students engaged in online activities appear to be making similar errors, or when a particular individual appears to be lagging behind. They can also suggest interventions to the instructor, tailored to the diffculty encountered. Likewise, these systems can provide students hints about what they might try or modules to review, as well as feedback regarding how the student is progressing relative to the rest of the class.

Applications of AI-based education technology support learning in four ways: through automated tutoring, personalizing learning, assessing student knowledge, and automating tasks normally performed by the instructor (Lu and Harris 2018). Intelligent tutoring systems (ITS) produce statistically signifcant improvements in student learning outcomes, such as mastery and retention, when compared to traditional classroom teaching, independent textbook use, and non-AI computer-based instruction (Ma et al. 2014). However, experts point out that ITS curricula are rather infexible due to technical challenges in accommodating user feedback, modifed core standards, or content changes.

In addition to supporting improved student learning outcomes, the use of AI and ML in education has the potential to lead to improved teacher satisfaction (VanLehn et al. 2019a, b; Dietrich 2015). AI coupled with ML can provide 24/7 student support. It supports tracking student performance and aggregating student concerns. It can facilitate personalizing and adapting learning materials to individual students. These automated tools enable timely and passive assessment and more fnely grained tracking of student knowledge and skill gains (Aleven et al. 2010; Arroyo et al. 2014). This assistance empowers the instructor, who can feel more confdent in student opportunity to succeed, knowing that the students are receiving needed support that the instructor might otherwise struggle to provide. The instructor is able to devote time to creative activities and feedback beyond what the automated systems can provide, such as affective feedback and support (Wu et al. 2016; Duo and Song 2012).

The implementation of an online learning system requires a sophisticated digital ecosystem that incorporates the complex interactions among students, instructors, and content. It must include a sophisticated human-computer interface that supports access, monitoring, feedback, and assessment (Reyna 2011; Rezaei and Montazer 2016). The system can be built upon an existing e-business solution or learning management system, or arise from an array of independent modules. These systems are often cloud-based, providing services over the Internet, to provide maximum accessibility.

### **2 Use of AI and ML in 2026**

### *2.1 Content Presentation*

#### **2.1.1 Opportunities**

1. **Multifaceted presentation**, such as mobile computing, the Internet, natural language interfaces, gesture-based interfaces (Audinot et al. 2018; Bowman et al. 2008; Case 2018), silent speech recognition (Waltz 2019), and other advances in tools for user interface development have led to a richness in the modes of communication between humans and machines. Guided learning (Chi and Barnes 2014; Price et al. 2016; Ontañón et al. 2017) automates feedback tailored to student needs and gives students control over their learning (Zhou et al. 2016; Harackiewicz et al. 1987). Brownfeld programming (Baley and Belcham 2010; Vujičić et al. 2018) allows students to learn programming by studying legacy

systems. Simulation-based learning (Lateef 2010; *Offcial Site | Second Life— Virtual Worlds, Virtual Reality, VR, Avatars, Free 3D Chat* 2019; *OpenSimulator* 2019; OPNET Optimum Network Performance 2020) offers alternatives to realworld experiences. Content can be presented through multiple modalities, such as text, video, audio, or simulation. These advancements are extending our ability to interface with machines and each other. However, more work is needed for such systems to become highly reliable, robust, and widely available (Case 2018).


#### **2.1.2 Challenges**


learning experience (Kyrilov and Noelle 2016). There is evidence that instant binary feedback increases the likelihood that students will cheat on assignments (Kyrilov and Noelle 2015). Conceptual feedback can be provided through casebased reasoning. A case-based reasoner can analyze student errors, compare them to previously recognized errors, and retrieve and tailor specifc guidance that proved useful to other students (Wiratunga et al. 2011; Kyrilov 2014).

#### **2.1.3 Implementation Strategies**

Content delivery can be cloud-based, server-based, or a combination of both. Fuad, Akbar, and Zubov present Dysgu (Fuad et al. 2018a, b) as an example of a cloudbased interactive learning environment. Dysgu personalizes and adapts out-of-class activities to satisfy individual student needs. Dysgu employs mobile technology to present activities that are smaller than traditional out-of-class activities. It incorporates social networking, which supports anonymous interaction and allows students to gauge their progress relative to the progress of other students.

In contrast, Isomöttönen, Lakanen, and Lappalainen's TIM (The Interactive Material) is a document-focused system (Isomöttönen et al. 2019). TIM's documentoriented user interface supports creating and editing learning materials, rather than managing course content. Instructors are able to track document sections (un)read by individual students. It also supports automated assessment, gamifed learning through performance monitoring and display, and comprehensive tracking of submissions and user interactions with the system.

#### **2.1.4 Research Questions**

The appropriate use of online learning systems, their curation, and assessment are open areas. Several concerns need to be addressed on the path to full and effcient integration with learning experiences:


### *2.2 Interactions and Communications*

#### **Classroom**

Five minutes before the frst class, Dr. Gomez logs into room system to turn on the affective computing machine. The cameras in the classroom quietly swivel toward the students' seats and begin populating data to the instructor screen regarding individuals' moods. Taylor Speek, the 27-year-old teaching assistant, starts meticulously drafting learning plans for students that are showing signs of poor engagement and diffculty grasping knowledge during the frst session, making decisions based on an analysis of previous courses related to fuid dynamics. This class is particularly rich in information because all the students have signed waiver forms to be recorded and have their physiological and physical traits captured to monitor and notice patterns and trends in their progress. Looking at the pre-course assessment, Taylor notices that 83% of the learners showed signs of stress according to the data on their wearables, particularly in the area on Bernoulli's principle. Taylor adds demystifying Bernoulli's principle to the class agenda and gives Prof. Gomez a heads up.

During the class, Alex runs into more confusion about Bernoulli's principle and makes mouth movements so that her silent speech recognition device picks up the question without disrupting the class. "It makes you feel less self-conscious," argues Alex. "My older sibling went into college thinking they would also pursue chemical engineering but then kept failing classes because they couldn't get help and felt too embarrassed to ask." The question is funneled to the class chatbot, which provides helpful resources curated by Prof. Gomez. The affective computing software on Alex's computer registers that although the resources alleviate some of the stress, this student will need more support after class. The same question which has been asked in similar ways throughout the class is anonymously posted as a single topic to the class community for discussion afterward. Prof. Gomez proceeds with the fnal activity seamlessly, knowing that the peer learning forum used will foster more opportunities for students to reinforce what they learned from each other today. The chatbot reports back to the mainframe and automatically schedules a session between Taylor and Alex when they are both available.

#### **2.2.1 Opportunities**

1. **Interactions and communications** in an intelligent learning environment span user-initiated information retrieval and question answering, facilitation of informal or directed peer-to-peer communication, and student-facilitator interactions for guided learning. The facilitation of natural language interactions with systems, from storytelling to question answering, has been studied since the birth of artifcial intelligence in the middle of the twentieth century (Winston 2016). Theories were devised for inferring the appropriate response to a question (Lehnert 1977; Wilensky 1977), and systems were constructed to allow novice users to ask about complex systems, such as Wilensky's UNIX Consultant (Wilensky et al. 2000). Later systems supported natural language interfaces to databases (Androutsopoulos et al. 1995) and ultimately IBM's Watson suite, designed for building conversational interfaces into any application, device, or channel (IBM 2018). Despite these advances, these interfaces lack deep knowledge and instead rely on massive amounts of data to train the system. While this is useful in most situations, being able to handle novel queries or mundane queries posed in an unusual fashion remains a challenge.


#### **2.2.2 Challenges**

Affect is an important component in human-human communication and can signifcantly infuence learning experiences (Wu et al. 2016). Affective computing has three components: detecting the emotions of the user, expressing what a human would perceive as an emotion, and actually experiencing an emotion (Picard 2003).


**Fig. 1** An example of Chen, Ciborowska, and Damevski's automated refection-in-action system. A student (ronnie c.) developing a secure mobile app is encouraged by the recommender system (grey) to learn about the security principle of chain of control (Reproduced from Chen et al. 2019)

thought to lessen learner satisfaction with e-learning systems and lead to a higher drop rate of online courses. The challenges lie in the sophistication of the systems, the complexities of intercultural communication, and developers' awareness of affect and ethics (Cowie 2014).

### **2.2.3 Research Questions**


### *2.3 Learner Activities*

#### **Intervention**

During Alex's support session, Taylor comments on an interesting tidbit highlighted by the data dashboard. "Alex, it looks like when you took Process Control, the only aspect of the subject where you struggled was momentum balances, which relates to Bernoulli's principle. Do you think that might be why you're struggling?" Alex's face lights up with this revelation and head nodding ensues. "Let me assign this additional activity you can do during your free time. I think it'll help a lot with coming back to basics and slowly ramping back up. I think you should try and do this before the next session. It scaffolds the level of diffculty as you go and should challenge you in the correct areas. This app will customize the content to be more video-based since I know that's what you seem to prefer. Is that correct? And don't worry, a lot of students struggle with Bernoulli's principle." Taylor spends the remaining 15 minutes they have together providing emotional support, answering questions, adding points of clarifcation, and ensuring that Alex receives the additional materials. As the student leaves the room, another one takes his place, and Taylor's dashboard changes to the appropriate student. When meeting with her master instructor, Taylor refects, "These tools have made my work so much more meaningful and effcient. I get to support more students than I did in the past and really target the key areas of need."

In the past decades, student activities and course assignments have evolved in new directions, from traditional take-home work to fipped classrooms (*Flipping the Classroom* 2019), from one-size-fts-all curriculum to personalized learning, and from paper and pen to digitized submissions requiring broadband Internet and access to personal devices. With these advancements, institutions have adapted their curriculum to more fnely deliver and assess learning experiences more focused on personalization than in the past—creating an environment that is ripe for leveraging machine learning models. As such, the lines between activities and assessments have also blurred over time. In some cases, instructors become facilitators of experiences as opposed to disseminators of information. In this section, we explore the current opportunities, challenges, implementation strategies, and further research questions through the lens of uncovering best practices in intelligent instructional design.

#### **2.3.1 Opportunities**

1. **Interactive** and adaptive game-based learning is an increasingly popular area of investment for researchers focused on designing new digitized and innovative learning activities, which are often examples of constructivism in education. Constructivism explains that humans acquire and gain knowledge through experiences (Wadsworth 1996). LewiSpace by Ghali et al. (2016) is an example of an exploratory educational game that was developed with Unity 4.5 to teach a college-level chemistry lesson on drawing Lewis diagrams, which are structural representations of molecules. LewiSpace captures learners' physiological traits such as electroencephalography (the measurement of electrical activity in different parts of the brain and the recording of such activity as a visual trace), facial expression, and eye movements throughout the game. LewiSpace also pulls data from a personality traits questionnaire in order to determine a learner's performance and potential need for help. Experimenting with multiple machine learning algorithms, Ghali et al. (2016) found the highest level of accuracy in predicting failure rates using a logistic regression model.1 Future versions of LewiSpace will incorporate real-time measurements to adapt the experience to learner's needs.

Similar to Dysgu (Fuad et al. 2018b), an interactive mobile game that is currently manually scaffolded by instructors, and Epplets (Kumar 2018), an interactive tool for solving Parsons puzzles (Kumar 2017), LewiSpace presents a typical case study of the potential for simulations to be designed in more adaptive manners for the enhancement of constructivist learning. To support such efforts, Li et al. (2010) developed an adaptive course generation framework, which extracts course materials and learner profles to help instructors design courses that are more advanced in handling multiple learning characteristics such as style and preference. This tool helps facilitate the alignment of course curriculum with the creation of adaptive activities.

2. **Supportive** elements in learning activities are crucial to ensure a relatable experience. Particularly in problem-based learning, enhancing critical thinking skills outside of the classroom may take the form of discussion-based activities. In such instances, intelligent learning tools such as MALESAbrain (Chiang and Fung 2004) encourage learners to judge their peers' solutions before exploring further content. This information is used to rank and arrange learning issues in an effort to transform obligatory forums and chat rooms into rich discussion opportunities. This provides an opportunity to improve the quality of conversations inside and outside of the classroom, which may increase the opportunities for peer-to-peer learning.

<sup>1</sup> In statistics, the logistic model is used to model the probability of a certain class or event existing such as pass/fail, win/lose, alive/dead, or healthy/sick.

#### **2.3.2 Challenges**

There are many remaining challenges toward leveraging artifcial intelligence in regard to learner activities. Important challenges include:


#### **2.3.3 Implementation Strategies**

Throughout the US education system, learner activities are often disseminated as instructors see ft to fulfll curriculum requirements and learning objectives. As such, instructors are a crucial part of the process for successfully deploying, tracking, and triaging these activities. As learning activities become more adaptive and personalized than instructors themselves, human instructors often become facilitators of experiences as opposed to disseminators of information.

Epplets (Kumar 2018) are a examples of software assistants designed to help students working alone learn good programming principles and algorithm design. The instructor retains the role of passing along knowledge to learners before the activity has begun. This small-scale integration of intelligence is an approachable frst step in observing the outcome of allowing students to practice and assess at their own pace and level of rigor. Epplets enables a teacher to maintain a relatively hands-on approach in monitoring progress and providing aid where necessary. In more immersive experiences such as LewiSpace (Ghali et al. 2016), which replace an entire lesson, including the process of disseminating knowledge, instructors should be prepared to smooth the transition between classroom and simulation. In-person class time should be focused on deconstructing the virtual interaction, facilitating discussion, and processing learnings into applicable knowledge outside of the experience.

Instructors within institutions should provide a fuid and seamless experience inside and outside of the classroom for learners, particularly for fipped experiences.

#### **2.3.4 Research Questions**

There are a number of opportunities for further study. A few key examples include:


### *2.4 Assessment*

A crucial component of determining the effectiveness of any learning program,

#### **Assessment**

At the end of the semester, Prof. Gomez wraps up the last class with an essaybased exam. Students are asked to answer questions based on their greatest opportunities for improvement. Alex receives three prompts on inviscid fow and one on Bernoulli's principle. The machine learning model is fne-tuned to Prof. Gomez's competency-focused goal of measuring improvement to ensure that the students are well-rounded and confdent in their abilities. "Why test a student on a topic that I know they're an A+ on?" shares Prof. Gomez, "Let's get straight to the point, what I care about is whether all of their skills are up to par and that we're being effective in our delivery of learning. Throughout the course, we should have caught all of the pain points and now we're just confrming." As soon as Alex clicks "Submit," the essays are automatically graded with a natural language processing tool. Prof. Gomez and Taylor also receive a new report on their dashboard, indicating that this fnal piece of data from the course has increased Alex's likelihood of successfully graduating in chemical engineering.

intervention, and/or activity is being able to meet or exceed anticipated student learning outcomes. In optimal situations, all students would be equipped to successfully accomplish a variety of learning goals within and outside of traditional learning environments. However, it is common knowledge among students, educators, and parents that this is simply not the case. Leveraging artifcial intelligence and, more specifcally, machine learning methods can illuminate opportunities to quickly assess student performance, provide accurate feedback, proactively engage with students to an extent not possible without the intelligent intervention, and predict likely student outcomes. In the succeeding paragraphs, we explore the existing literature and opportunities within four major categories: (1) predicting performance, (2) reading and writing tasks, (3) zone of proximal development, and (4) personalized learning.

#### **2.4.1 Opportunities**


useful in primary education, these principles may still support higher education English-as-a-Second-Language (ESL) students in all felds.


Finally, affective computing provides a promising avenue for machine learning applications as learner engagement, knowledge retention, and many other components of learning are infuenced by emotion. Wu et al. (2016) provide a review of current trends and challenges with affective computing in education and learning. The authors identify common data collection and machine learning methods used. They also highlight opportunities to leverage insights to intervene at appropriate moments to improve learner trajectories.

#### **2.4.2 Challenges**

There remain a number of outstanding challenges toward fully developing artifcial intelligence within synchronous and asynchronous environments. Key challenges in the four sections outlined above include:

1. **Predicting performance** (Ogor 2007; Ciolacu et al. 2017) presents ethical challenges if learners are given opportunities and support based on their anticipated grades and test scores from machine learning algorithms. Measures of performance on tests and grades are limited defnitions of student achievement and real-world outcomes in STEM felds (Spector 2017). This approach limits the way in which we defne learner success and evaluate whole-person skills that produce effective STEM graduates such as resilience, grit, or lifelong learning (Strauss 2017).


Several important questions remain, especially in the area of societal considerations. Can an AI be programmed to accurately account for racial, cultural, and religious differences? Can this be accomplished without controversy or running the risk of inappropriate racial or other profling? Are there basic, humanistic beliefs that can be integrated into these systems to ensure equitable treatment and respect for all? Finally, there is the issue of consent. What challenges does a mixed classroom of those who consented and those who did not create? Does such a mixed classroom affect outcomes on both sides?

#### **2.4.3 Implementation Strategies**

In many cases, AI for assessment and evaluation are embedded within standard synchronous and asynchronous activities, e.g., evaluating test scores or written work. Data mining techniques are leveraged with machine learning models to passively determine desired predictions. As universities invite more remote students into their programs, there is an increased amount of potential data to be gathered, tested, applied, and analyzed. In instances such as Dysgu (Fuad et al. 2018b), instructors have manually assigned learner activities (replacing the judgment piece of what may become AI in the future) in order to increase student engagement

outside of the classroom. This experiment provides a low-fdelity frst step for institutions to test whether or not implementing AI in certain areas would be impactful to a student's learning outcomes.

Additionally, the partnership between AI and the instructor is a key relationship to balance. Fully applying AI in a classroom or within institutions will require clarity in terms of role defnitions and leveraging each party's strengths. For instance, an instructor may be more effective in providing an emotion-based intervention after a machine learning algorithm has identifed a disengaged learner, rather than having the instructor initiating an emotion-based response without understanding the current affective state of the learner.

#### **2.4.4 Research Questions**

There are a multitude of opportunities for future study. A few crucial examples include:


### *2.5 Co-curricular Activities*

Learning is being transformed by intelligent systems (Schmelzer 2019). Co-curricular activities are those which relate to and support an academic course of studies. AI-driven co-curricular activities are an opportunity to support online learning, the various ways people learn, and the rates at which they learn. Machine learning-based development of student profles and customization of training materials allow instructors to draw upon a single curriculum while modifying content and presentation for individual users. Online textbooks and their interactive interfaces further support tailoring of content and personalization of delivery and feedback. They facilitate providing students hints as they work through assignments and conceptual feedback and as exercises are automatically assessed.

A learner interacts with material in many ways external to the formal educational event. Thus, the student experience consists of a wide range of interactions that may consist of athletic, scholarship, social, and service dimensions. Providing students with access to a range of these dimensions is a critical link in developing the overall quality of the student experience. Workshop participants identifed several forms of interaction. They called for the augmentation of tutoring with personal, conversational education assistants, often referred to as autonomous conversational agents (IBM 2018). These and other integrated intelligent autonomous education agents must respond effectively to a learner's questions and provide assistance with learning or assignment tasks. Learning systems need to reinforce concepts in a personalized fashion with additional materials to reinforce the curriculum. Furthermore, they need to allow students to learn at their own pace, to satisfy their own goals. Co-curricular education involves many opportunities and challenges. Co-curricular activities are implemented in a variety of ways and result in several open research questions.

#### **2.5.1 Opportunities**

X-FILEs workshop participants identifed several characteristics of future learning and future students. Their consensus was that the basic pedagogy will still be delivered with new technology, and predicted that the best pedagogical ideas will be more fully realized. The Internet facilitates delivery of content to large numbers of individuals both synchronously and asynchronously, and the scale of delivery is expected to change. Feedback will become more automated, and content and delivery will be tailored to instructional objectives and the individual learner's interests and needs. As institutions are called to do more with limited budgets, the importance of virtual environments over brick-and-mortar settings will increase.

Participants were asked how the teaching and learning process might be different in the future. Four responses stood out:


A classroom environment that is conducive to learning entails staging the physical space, creating a communal environment, maintaining a positive climate and culture, and, most critically, convincing the students to become cooperative, active learners (Lynch 2016).

#### **2.5.2 Challenges**

Several challenges must be addressed to implement widespread adoption of automated intelligent co-curricular activities. These can be categorized as seeing a need for the systems, as well as acceptance, availability, assessment, and robustness.


#### **2.5.3 Implementation Strategies**

AI-based co-curricular activities range from games to tutors to immersive simulations. Gamifcation platforms such as Classcraft, Rezzly, Seppo, Youtopia, and Kahoot! offer external motivation, such as rewards and leaderboards, and internal motivation, such as autonomy and mastery, to engage students in meaningful learning experiences (Goshevski et al. 2017). The instructor is able to provide or tailor content to increase relevance to the target content. AI chatbots, such as Jill Watson at Georgia Tech (Eicher et al. 2018), as well as Mtabe from Tanzania and LangBot in Ethiopia (Nsehe 2019), provide students personal tutors, tailored to instructional and learner needs.

Adaptive learning has the potential to promote access and quality at scale in higher education (Becker et al. 2018). Cavanagh et al. (2020) lay out the design of framework for adaptive learning and best practices for its use. The features of the design framework include objective-based learning knowledge units, personalized assessment and content, adaptive learning paths, alternate content, and procedurally generated questions. These systems can be implemented as standalone components or integrated into an existing learning management system. ALEKS (*Overview of ALEKS* 2020; Boyce and O'Halloran 2020) is one widely used for algebra. M-Shule from Kenya (Haba 2017) is an example of a data-driven personalized learning system for K-12 education.

Virtual labs present another opportunity for co-curricular activities that involve immersive simulations. Labster (*The Complete Guide to Virtual Labs* 2020) and PraxiLab (*Virtual Science Labs at Your Fingertips* 2020) offer commercially available virtual labs for secondary and higher education, as well as other informal learning needs.

Immersive platforms such as SimCity®, Second Life®, and the *Unity* real-time development platform are additional opportunities for implementing co-curricular activities that allow 24/7 access to engaging activities that support informal learning (August et al. 2016); Winkelmann et al. 2017).

#### **2.5.4 Research Questions**

Workshop participants offered a number of open research questions related to intelligent autonomous education agents in co-curricular activities:


### **3 Conclusions**

Development of robust, engaging, effective digital systems for learning must engage all classes of stakeholders from conception through implementation and evaluation. Such systems need to be integrated into the learning environment and into the routines of the instructors and students. Discussants at the 2018 X-FILEs Workshop identifed several concerns:


Addressing the frst concern requires broad studies of innovative learning environments over diverse demographic groups and a range of higher education institution types. These will become more feasible over time as interactive learning environments are more widely adopted. Addressing the second concern requires looking to the longer term beyond gains in content knowledge and examining the affective impact of these learning opportunities, as well as development of critical thinking skills and fostering independent learning.

Many other questions remain to be considered:


A rollout of the Summit Learning Platform (2017), a Chan Zuckerberg Initiative, points to multiple areas for future study, including integration of student/instructor integration, appropriate limits on screen time, parent acceptance, access to vetted resources, and controlled access to non-vetted resources. Community experiences in a Kansas school district refect the need for more thought on these points before successful integration and achievement of learning objectives can be achieved (Bowles 2019).

### **References**

Ahadi A, Lister R, Haapala H, Vihavainen A (2015) Exploring machine learning methods to automatically identify students in need of assistance. In: Proceedings of the eleventh annual international conference on international computing education research—ICER '15, pp 121–130. https://doi.org/10.1145/2787622.2787717


conference on programming language design and implementation, pp 481–495. https://doi. org/10.1145/3192366.3192384


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# **Emergent Guiding Principles for STEM Education**

**Lawrence C. Ragan and Lorraine J. Ramirez Villarin**

### **1 Motivation**

STEM-related jobs are on the rise. Workers are required to exercise critical thinking and decision-making skills while being knowledgeable and competent in domains related to Science, Technology, Engineering, and Math (STEM). The eXploring the Future of Innovative Learning Environments (X-FILEs) Workshop, hosted by the Florida Institute of Technology in November of 2018, allowed stakeholders to make informed decisions about the adoption and use of innovative learning environments (ILEs) in higher STEM education. Participants had the opportunity to consider four emerging technologies that could assist in this effort: personalized and adaptive learning, multimodal learning formats, cross reality (XR), and artifcial intelligence and machine learning. The research team gathered shared ideas through online meetings and collaborative activities that refect on the opportunities and challenges to expect while implementing ILEs in higher education STEM curricula.

Since these immersive technologies are quickly evolving, it has been challenging for institutes to implement integrated STEM education programs that utilize them (Blackley and Howell 2015; Briener et al. 2012). Therefore, these guiding principles are a good starting point.

L. C. Ragan (\*)

Ragan Education, State College, PA, USA e-mail: lcr1@psu.edu

L. J. Ramirez Villarin University of North Georgia, Dahlonega, GA, USA e-mail: Lorraine.RamirezVillarin@ung.edu

© The Author(s) 2021 107

J. Ryoo, K. Winkelmann (eds.), *Innovative Learning Environments in STEM Higher Education*, SpringerBriefs in Statistics, https://doi.org/10.1007/978-3-030-58948-6\_6

### **2 Recurring Themes**

A series of recurring themes emerged from the data collected at the 2018 workshop. These themes appeared across most or all four technology categories and served as the basis for the development of guiding principles for the future of STEM education. In most, if not all cases, the identifed themes were interconnected and related to each other. For example, providing students a personalized learning environment based on personal interest and preferences is dependent upon having collected data about each student. In collecting data on student preferences, concern for student rights to privacy must be evaluated.

The nine emerging principles were classifed into four clusters to better understand their correlation and importance within the academic instructional design feld. The four clusters encompass the student experience, the learning community, its availability, and the learners' protection. While considering the affordances of the supporting technologies in the generation of ILEs, the limitations of these technologies must also be weighed.

### *2.1 Student Experience*

**Student-Engaged Learning Environment** While the lecture format of instruction has been the norm in higher education for years, it is time for the students to play a more active role in their educational experience. STEM education requires learners to solve real-world challenges through the application of scientifc and mathematical principles, the engineering process, and the use of current and emerging technologies. To help retain learners who choose STEM programs in college, active learning approaches are a must (Olson and Riordan 2012). Carefully crafted ILEs allow students to construct representations of their own knowledge and practice applying their skills to new and novel problem scenarios. Donally (2018) suggests students create monuments for a history project, design worlds with unique ecosystems for science, or bring a literary reference to their reality for a reading assignment.

Personalized learning provides the opportunity to tailor instruction to the individual's needs, skills, goals, and learning preferences while consistently monitoring progress (Sampson 2001, cited in Sampson et al. 2002). This approach promotes improvement while enabling participants to take control of their learning. Nonetheless, student-centered environments like multimodal learning require students to have a high degree of agency and metacognition to recognize when learning is happening and when to challenge themselves (Bezemer and Kress 2016; Moreno and Mayer 2007; Phuong et al. 2017; Sankey et al. 2010).

Virtual reality provides an opportunity to keep the student as the center of the learning environment. While being "mistake-free" (Bailenson 2018), participants actively explore virtual worlds or encounter scenarios where learning takes place through trial and error. Cross reality experiences like these have shown equivalent learning gains (Madden et al. 2018; Winkelmann et al. 2017) or lesser learning gains than traditional instruction (Makransky et al. 2019). However, cross reality experiences focus more on scientifc thinking processes and behaviors than it does in content knowledge.

Applications of artifcial intelligence help craft these learning environments through automated tutoring, personalized learning, student knowledge assessment, and automating mundane instructors' tasks (Lu and Harris 2018). Devices like chatbots and auto-graders enable progress monitoring which may pinpoint strengths and weaknesses that can steer students and instructors on the proper academic path.

**Personalized Learning** Students do not learn in the same way. They learn better when instruction is individualized to the learner's needs (Kerr 2016). However, factors like academic background, culture, language, or disability may infuence the manner an individual learns. Having the advantage of a personalized or customized learning environment can be benefcial for students, as well as instructors. Dashboards that monitor individual activity, provide feedback, alert instructors of students' diffculties, and suggest other approaches are likely to enrich instruction. For example, adaptive learning environments are based on individual students' profles created upon their strengths, weaknesses, and pace of learning (Becker et al. 2018). The technology tracks progress and adjusts the learning path enabling instructors to deliver timely feedback. These adaptive learning systems promote self-regulated learning which has been shown to be a predictor of academic success (Boekaerts 1997; Miltiadou and Savenye 2003; Pintrich and De Groot 1990; Rosen et al. 2010). A study by Prain et al. (2013) reported that math students participating in a personalized curriculum got better scores and were more engaged than those who were not.

A "one size fts all" curriculum may not fulfll everyone's needs (Gee 1996; Phuong et al. 2017). While studying augmented reality applications for cross reality, one recommendation made by Radu (2014) was that experiences be designed based on curriculum and pedagogical needs while providing the opportunity for faculty to customize them accordingly. For example, an AI-driven personalized learning system would adjust the context of the content based on the learner's interest and preferred learning modality while still meeting desired learning outcomes.

In adaptive learning systems, students work at their own pace. Since learner activities are auto-graded, they can receive feedback and scaffolding when needed. It is important that the student does not feel isolated. Communication should not be a concern between students and instructors. As in any learning experience, support should be available at any time in the form of live instructors or automated systems to provide guidance or answer questions.

**Flexible, Fluid, and Evolutionary** Current and emerging technologies are transforming constantly tending to learners' and instructors' needs while also considering academic institutions' demands. The required technology for most of these ILEs can already be found in many homes, classrooms, and workplaces in the form of

smartphones, computers, or game systems. Bailenson (2018: 9) predicts virtual reality to be "a mainstream technology, worth an estimated \$60 billion, in the next decade." As for augmented reality, it is expected to reach \$60 billion in 2020 (Porter and Heppelmann 2017). This fexibility of access provides the opportunity for informal learning environments almost anywhere. Multiuser 3D virtual world environments allow geographically dispersed learners to participate in a traditional classroom-like environment while learning at their own pace (Olasoji and Henderson-Begg 2010) and in their own time zone. This provides instructors the opportunity to convey fuid and seamless learning experiences inside and outside the classroom.

The fexibility of content and context is another advantage of ILEs. Having the malleability to enhance content by presenting it in more than one sensory mode or incorporating text-based information and multimedia using a diversity of modes can generate multiple access points for learning (Bezemer and Kress 2016; Matusiak 2013; Nouri 2018; Sankey et al. 2010) as those provided through multimodal formats. Well-designed courses like those for adaptive learning foster the learning of processes and strategies that promote self-regulation in students (Dabbagh and Kitsantas 2013; Hense and Mandl 2012; Shea and Bidjerano 2009). On the other hand, higher interactive virtual environments work through learning activities as immersion stimulates emotional involvement which is essential in the relocation of learning to long-term memory (Aldrich 2009).

For measuring student progress and performance, formative and summative assessments inform the student of advancement toward the learning objectives and provide a fnal grade of achievement. However, many methods do not objectively address student achievement or translate into real-world outcomes in STEM felds (Spector 2017). Using "real-to-life" and situated learning can enable students to practice or be measured within the context of the domain. Simulations, virtual reality, and augmented reality can recreate "real-world" experiential opportunities for learning that would otherwise be impossible in a regular classroom. These immersive environments promote learning without any hazards to students or others. For example, surgical procedures may be practiced safely as no "real" patients will be in danger (Health Scholars n.d.). Chemicals can be mistakenly combined or spilled and cleaned up by pressing a recycle button (Faulconer and Gruss 2018) with no harm to participants. These experiences can be paused, reset, and available ondemand (Lynch and Ghergulescu 2017), while the dynamic nature of the computer system records and gathers data in the background as the learner moves along (Rose 1995). Learner activities like these bring relevance to context while replicating many tasks professionally encountered in STEM-related felds.

STEM educators need to be open to a variety of technologies and recognize the interoperability of various systems. Learning management systems (LMS) may not provide the instructor with all they need to support a synchronous or asynchronous learning system. In some cases, there are add-ons to the LMS that may be used to expand the system's functionality. Social media platforms, cross-platform messaging applications, and other application software may facilitate instruction even if these programs' main duty is not related to academics. For example, Twitter can be used to promote interaction among peers and instructors, while Facebook and WhatsApp are fexible enough to create groups or chats for which you can share video and audio recordings and get immediate feedback.

### *2.2 Learning Community*

**Include Both Individual and Group Interactions and Input** Students who select STEM programs in college might be looking forward to engaging in active learning approaches. To challenge these learners, instructors may want to alternate between some of these technologies for individual and collaborative tasks. For example, computer-supported collaborative work assesses individual and collaborative potentials based on individual and shared contributions. Ward and Sonneborn (2009) indicate that creative problem-solving in groups must consider the learner group contribution and the quality and quantity of ideas produced through individual collaborations. Other communication systems like collaborative virtual environments allow users to share a three-dimensional digital space while occupying remote physical locations (Yee and Bailenson 2006). With dozens of multiuser experiences available (JuegoStudios 2019) like social virtual reality instruction, facilitated discussion, and project collaboration, instructors and researchers have the freedom to incorporate these for either learning or research purposes.

As for personalized and adaptive learning, interactions occur between student and content and between student and instructor. Student-student interactions are not typical. Moore (1989: 1) describes student-content interaction as that which "results in changes in the learner's understanding, the learner's perspective, or the cognitive structures of the learner's mind." Student-instructor interaction, often based on performance, usually takes place through the learning management system.

Besides promoting collaborative learning, interactions that can foster communication skills and cultural enrichment could enhance these learning experiences. Donally (2018) claims that augmented reality, virtual reality, and mixed reality allow students to shape how they view others around the world through interaction and collaboration. It should be a top priority for institutions with access to these cross reality resources to have the opportunity to stimulate interaction and promote collaboration while inviting students from other academic institutions with similar access. These immersive technologies should expand to engage global interactions and feedback.

Multimodal learning environments facilitate interaction since those who might have trouble communicating in one mode can interact in another. For example, students can arrange to connect with culturally diverse students who might be wellversed in the digital world or other multilingual or English language learners or others who have similar learning disabilities, creating a rich, comfortable learning experience. Considering the evolving demographics of students expected in higher education, multimodal formats can help customize student-centered experiences and address diversity issues.

**Co-contributed and Multiply Sourced** Increasingly, with access to sophisticated online content development tools, the instructional material used in a course may be generated from multiple sources. Students, generating content through social media or self-publishing, perceive a class assignment as an opportunity to create, generate new information, or assimilate existing data into new perspectives. This studentgenerated content makes an increasingly critical contribution to the course experience. Indeed, with the trend toward active and engaged learning, students are being encouraged and directed to create new works, products, and ideas that may have added value outside of the course.

The content for today's classes is unlikely to be singularly sourced, that is, comes from a single text or course pack. Content from multiple sources can be presented in different modes (e.g., gestures, visuals, multimedia, text-based information) (Bezemer and Kress 2016; Matusiak 2013; Nouri 2018; Sankey et al. 2010) and greatly enhance the learning experience. The richness of content digitally accessible is extensive and more likely to be integrated into the learning materials than other reference sources. In many cases, live data streams provide the most current data source. For example, assembling a virtual expert panel of industry leaders directly linked into a live lecture can further approximate reality and enrich the experience for all participants. With this access to a plethora of content comes the responsibility to ensure the accuracy and validity of the information. This alone becomes a new skill the lifelong learner will need to master!

### *2.3 Availability*

**Equity of Access** As new learning systems emerge, the concern of equitable access for all learners needs to be reviewed. These ILEs are crucial learning instruments, and all students should be provided equal access regardless of privilege, ability, or economic situation. More sophisticated adaptive learning systems, often developed commercially, may also increase the cost for engagement for the learner. Even as the cost of cross reality devices may decrease, the technical requirements may include higher-speed bandwidth, device batteries, and add-on components. In some locations, for example, broadband access may limit student interaction with cross reality systems.

Technologies such as artifcial intelligence can require resources from machine learning models for each assessment and grade level (Balfour 2013; Shermis et al. 2010; Nehm et al. 2012). Curating learning environments for different modes can also be quite complex and demand additional allocation of resources (Bezemer and Kress 2016). Add to this fnding the preparation of the physical space including enhanced lighting and audio, upkeep of devices, and the time and money it takes to train instructors and technicians. This will make it challenging for certain academic institutions who lack the resources. A digital divide still exists between certain educational institutions and communities (ACT Center for Equity in Learning 2018).

**Accessibility** Assistive technology refers to any item, piece of equipment, software program, or product that increases, maintains, or improves the functional capabilities of people with disabilities and is in the form of equipment, software, or hardware (Assistive Technology Industry Association 2019). Immersive learning environments such as those made possible through virtual worlds simulations can provide participants with a sense of physical movement which can enhance their experience.

This same advantage, however, in some circumstances, may deprive an individual with disabilities from this enhanced experience (Ryan 2019). The Americans with Disabilities Act prohibits discrimination toward this population by allowing support in the workplace and academic institutions (U.S. Equal Employment Opportunity Commission n.d.; U.S. General Services Administration 2018). Unfortunately, most personal devices, nowadays, do not accommodate the emerging technological demands for replicating experiences for this group and other diverse learners. Therefore, engagement will be interrupted as you exit the classroom. Providing this population with the same opportunities as traditional student has is not only keeping compliant with the law but also the right thing to do to achieve equity and inclusiveness.

### *2.4 Learner Protection*

**Safe and Secure** Keeping the privacy of users seemed to be a recurring concern across three of the four technologies explored. Unfortunately, the digital world is not exempt from inappropriate or unlawful actions. Users may be exposed to threats like cyberbullying, griefng, and security attacks. For example, research done by Outlaw and Duckles (2017) refected that female participants in social virtual reality experiences had encountered "firting, a lack of respect for personal boundaries, and socially undesirable behavior," similar to those in real life, leaving little desire to return to these platforms. As for multiuser environments, the number of accounts and avatars that can be created can increase trust and identity concerns (Warburton 2009). In order to minimize this, proper training would be required for instructors and learners since a lack of technical skills could make them more susceptible to this kind of action. Faculty and students need to understand that some channels or modes may be "open" for public viewing. Professional development can provide orientation on what university tools are available to ensure students' safety and privacy. Nevertheless, training is time-consuming and costly.

For example, during the COVID-19 pandemic, Zoom became one of the most popular platforms for teleconferencing. However, "uninvited guests" would ruin the

video conference experience using shocking imagery, racial slurs, or profanity leading to a phenomenon known as Zoom bombing (Lorenz and Alba 2020). These "trolls" (disruptive, uninvited strangers) were hard to identify and, therefore, suffered little to no consequences for their malicious actions. Zoom implemented stricter measures of access control enabling the host to have more restrictions over the meeting. Nonetheless, these incidents may still happen in other platforms and without the user's knowledge. For example, spyware can be installed remotely through an email, a photo download, or an instant message. This type of malware records screenshots of information typed (e.g., passwords, usernames), and media accessed.

Access control measures can be a challenge, but there are certain options that can improve security. More and more organizations are adopting multifactor authentication which requires users to use more than one physical device to verify their identity. The second factor device can be your mobile phone in addition to a personal computer. Facial recognition and fngerprint readers are also common biometrics mechanisms used for identity authentication. Other options include the power to access your information from another device which gives you the fexibility to receive notifcations when changes are made in your accounts. To protect your information, data encryption allows the system to scramble the data and share a secret key or key pair with those who access it. However, as technology changes every day, access control measures need to evolve to outsmart attackers.

**Ethics** The increasing technical dimensions of developing STEM curriculum demands a critical review of the ethical aspects of the use of these tools in future educational systems. As presented previously, the safety and privacy of each learner are paramount. Although proper netiquette is expected among users, hacking and cyberbullying are dangers to which everyone is exposed. As users create multiple accounts and avatars, for example, in a multiuser environment, trust and identity may be threatened (Warburton 2009). These hazards can be intimidating for some and may limit use especially if an individual is not familiar with the different technologies. To minimize falling into these negative experiences, rigorous training on the different modes and technologies is a must for instructors and learners going through this digital transition. However, this may not be enough. Certain corporations, like Microsoft, are adding face-to-face interactions to virtual and persistent collaboration rooms to enhance trust as they hope to expand to the workforce market with a cross reality platform called Hololens (Weise 2019).

As technologies involving cross reality evolve bringing features like sight, sound, smell, taste, pressure, heat, and texture to the learning environment, the danger exists in disconnecting the user from reality. Bailenson (2018: 46) indicates that "VR feels real, and its effects on us resemble the effects of real experiences. Consequently, a VR experience is often better understood *not as a media experience, but as an actual experience,* with the attendant results for our behavior." If virtual experiences are perceived as real, for example, it raises the question of ethics in conducting research with "virtually real-life" specimens and "real-life" specimens.

Another ethical challenge relates to the way algorithms work in predicting performance. Learners might be presented with opportunities or support based on their anticipated scores or scores provided by a machine algorithm (Ogor 2007; Ciolacu et al. 2017). Users may want to "play the system" to deceitfully display to have a better performance. Instructors may feel uncomfortable trusting results from an algorithm and may resort to other forms of assessment. However, FERPA requires student information to be kept private, including scores. To keep compliant, strict rules of engagement, regulations, and set boundaries should be established for users and offenders.

The use of increasingly intrusive technologies that collect, store, analyze, and manipulate personal data presents a myriad of ethical decisions that must be addressed and defned. Although AI and machine learning has the capacity to track and adjust the course content to the interest and preferences of the learner, they also present the opportunity to negatively infuence students' behaviors. Defning the boundaries of appropriate social behavior when presented such powerful data gathering tools is a crucial step in the development of future learning systems.

### **3 Conclusion**

The guiding principles for STEM education emerging from the X-FILEs Workshop provide a foundation on which to consider the design and development of future STEM innovative learning environments. These principles are not exclusive to STEM education but are also critical to the construction of all course instruction in the near and future education systems. These guiding principles refect the capability and promise of the technologies that make it possible as well as the challenges and threats presented by the use of these technologies.

The guiding principles emerging from the X-FILEs Workshop aggregated into four clusters representing the student experience, the learning community, availability, and learner protection. All of the clusters refect some aspect of the learner's response to the design of the learning system. Many of the principles are interrelated and, in some cases, represent "the other side of the coin" of a stated principle. For example, a personalized learning system described in Sect. 2.1 also presents the challenges of protecting personal data gathered from profle information of the student as referenced in Sect. 2.4.

Forefront in these principles is a focus on the changing role and experience of the learner. These principles embrace an engaged and active learner as both a recipient and a contributor to the learning system. Capitalizing on the capability for all class participants to contribute to the construction of the course content, the principle stated in Sect. 2.2, empowers students to improve and enhance the course materials. In increasing the responsibility of the learner as a co-contributor, the act of generating, analyzing, and publishing course content also prepares them for controlling and directing their own future learning opportunities.

Perhaps the most challenging ideas and unanswered questions evolve from the ideas in the Sect. 2.4 principle. The promise and power of the previous principles raise the awareness and concern for addressing how to protect the individual and group rights of learners. The unprecedented affordances and opportunities enabled by the technologies and methodologies explored in the X-FILEs research create a multitude of ethical questions regarding harvesting, analyzing, and manipulating student data and input. Where, when, and how to use student data within the learning system need to be defned and appropriate guidelines and protections established.

The guiding principles emerging from the X-FILEs Workshop can serve as the basis for the design and development of student-centered, interactive, and increasingly effective innovative learning environments. These principles need to be further vetted, refned, and improved in order to address all dimensions of the learning ecosystem. Input and responses from all stakeholders also need to be garnered in order to ensure as thorough and comprehensive of a set of guidelines as possible. The future of STEM education is exciting and creates the responsibility of doing so thoughtfully and intentionally in order to maximize learning and preparing students for lifelong learning.

### **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# **X-FILEs Jam: Ideation Process and Outcomes**

**Lawrence C. Ragan and Lorraine J. Ramirez Villarin**

### **1 Motivations**

In exploring the future of STEM education in 2026, the research team recognized the importance of gaining insights from a variety of stakeholders including administrators, faculty, instructional design and technology professionals, and visionaries who scan and identify trends and forces affecting higher education. Perhaps the most critical perspectives, however, were the insights, thoughts, opinions, and hopes of the user population, the students. The 2018 X-FILEs Workshop included a student-led discussion panel providing their perspective of STEM education in 2026. Four students, representing high school through graduate students, served as panelists and shared their hopes and expectations of the higher education experience. The X-FILEs research team recognized the value of this panel to the X-FILEs project and determined to structure an event that would stimulate and harness ideas directly from undergraduates and graduates. The target population was expanded from students representing STEM felds to students representing a range of backgrounds and academic disciplines.

The purpose of the student-based event was to gain complementary input that would enhance the data gathered at the 2018 X-FILEs Workshop. The form and structure of the 2018 Workshop would not be appropriate as it did not match the interest and logistics of a 1-day format. The project team settled on the "jam" model of ideation to organize and operate the event. Jams, sometimes referred to as "ideathons" or "designathons," focus on idea development rather than the produc-

L. J. Ramirez Villarin University of North Georgia, Dahlonega, GA, USA e-mail: Lorraine.RamirezVillarin@ung.edu

© The Author(s) 2021 121

J. Ryoo, K. Winkelmann (eds.), *Innovative Learning Environments in STEM Higher Education*, SpringerBriefs in Statistics, https://doi.org/10.1007/978-3-030-58948-6\_7

L. C. Ragan (\*)

Ragan Education, State College, PA, USA e-mail: lcr1@psu.edu

**Fig. 1** X-FILEs Student Panel Session

tion of a product. Jams are team-based, loosely structured exercises conducted in a face-to-face environment designed to bring out participants' creativity for developing innovative solutions to complex problems (Morrison 2009).

Jams engage students and promote creativity by giving them a challenge statement and structured guidance for solving it. The most creative jams require a multidisciplinary team in order to achieve the desired outcomes. Challenges require participants to apply knowledge to solve real-world problems. The X-FILEs Jam focused on a modifed challenge statement derived from the 2018 X-FILEs Workshop (Fig. 1).

"Using what you've learned about innovative learning environments, create a solution that improves or enhances the student experience for a challenging dimension of college-level STEM education."

### **2 Goals**

Applying the challenge statement to the X-FILEs Jam program, the goal was to generate a collection of student idea-solutions where one or more of four technology categories may be applied to an aspect of teaching and learning to address a specifc problem in STEM education. During the process, the research team gathered the participants' ideas and brainstorming results through a series of written tasks to track common patterns and keep a record of students' thoughts. Findings refected what students envisioned of their STEM education experience in the near future.

### **3 Program Summary**

### *3.1 Technology and Teaching and Learning Framework*

In order to maintain connective themes with the original 2018 X-FILEs Workshop, the same four technology categories and the aspects of teaching and learning with their defnitions were incorporated. Technology categories included personalized/ adaptive learning, multimodal learning, cross reality, and artifcial intelligence/ machine learning. The aspects of teaching and learning encompassed content presentation, interactions and communications, learner activities, assessment, and cocurricular activities. The student teams were encouraged to address at what point in the course experience these technologies (individually or a creative mix) may be applied. The project's principal investigators, Drs. Ryoo and Winkelmann presented the defnition and descriptions of these categories as they applied to the Jam. A video recording is available on the Jam website (Ryoo n.d.).

### *3.2 Design*

Students, working in teams of three or four individuals, were guided through the day's ideation process in order to facilitate the successful completion of their ideasolution addressing the X-FILEs Jam challenge statement. Team idea-solutions were incubated and developed over a period of 6 to 8 h and presented at the conclusion of the Workshop. Award categories and prizes were described at the beginning of the program with the fnal awards being presented during the wrap-up activity.

### *3.3 Jam's Summary of Events*

The X-FILEs Jam began with a tour and hands-on experiences guided by Martin Gallagher, director of the Evans Library Digital Scholarship Lab on the Florida Tech campus. He introduced the students to the four categories of emerging technologies discussed in the 2018 Workshop. This presentation was followed by participants exploring, interacting with, and refecting on how today's technologies can become vital resources on future STEM higher education instruction.

Lawrence C. Ragan, the day's facilitator, introduced the X-FILEs research team and sponsors, followed by a review of the events, timing, and desired outcomes. The principal investigators, Drs. Ryoo and Winkelmann gave presentations to familiarize students with the transformative technologies in STEM education featured in the 2018 X-FILEs Workshop. Defnitions and the potential of each technology, as well as examples, were provided.

The "STEM Idea Harvest" involved students in a gallery walk activity with stations representing each of the fve aspects of teaching and learning. As the students

**Fig. 2** Jam participants engaging in the gallery walk activity at the assessment station. February 8, 2020

arrived at each station, they refected on the challenges, barriers, and opportunities presented by that aspect. They wrote their many ideas on sticky notes at each station. Students moved to all fve stations in 3- to 5-min increments (Fig. 2).

In order to form the day's teams, participants were asked to select one of the aspects based on their personal interests. The desire was to form teams with a mix of discipline interests and levels of undergraduate or graduate education. Participants organized into fve teams:


Team members introduced themselves and developed a "How Might We …" starting statement for their team-based on the generic example:

"How might we improve learning for students to enjoy, embrace, and apply STEM concepts using a variety of existing and future technologies?"

The day's agenda consisted of a series of activities designed to promote team formation, ideation, solution creation, and idea refnement. The beginning activity was the "Innovation Disruption Analysis." Teams randomly selected three cards from the Business Innovations Cards stack (Board of Innovation 2020). Each card provided an example of a case study from a company that generated a new product after recognizing a gap in the industry. The cards represented the company's responses to technological, market, customer, and regulation trends. Teams refected on why it was an inspiring case and how it became an innovative disruption. They later presented a short description on how these disruptions may infuence their idea-solution. The facilitators declared Team To-Do, focusing on the aspect of learner activities, winners after suggesting how the Duolingo app could be incorporated into each of the aspects of teaching and learning to improve the students' learning experience.

Participants continued with an "opposite thinking" exercise. Teams developed a list of assumptions for their aspect of teaching and learning. Based on these assumptions, the teams formulated the opposite response or representation of the assumption. For example, if one assumption of STEM education is that all students receive homework assignments using the same context, an opposite response may be that the context is personalized for each student based on their personal interest or area of study. Each team was provided writing materials and encouraged to record their ideas and the process as they developed their idea-solution plans.

To conceptualize the idea-solution, teams engaged in the "Amazing Idea-Solution Process." Inspired by the show *The Amazing Race*, the task involved teams independently completing certain steps to narrow their ideas before moving forward (Fig. 3). Five cards were created with instructions on how to complete each steps. The facili-

**Fig. 3** Participants from Team WhatsApp working on Card 2 of the "Amazing Idea-Solution Process." February 8, 2020

tators provided the next level card to teams after step completion. These cards for the idea-solution development were:


At the completion of the "Amazing Idea-Solution Process," teams presented their idea-solutions to the larger community in a 10-min presentation and a 2-min question and answer session. While the participants voted for the Student's Choice Award, fve judges evaluated the presentations using a criteria checklist to determine the winners in the remaining categories. Drs. Ryoo and Winkelmann presented the awards which would earn a gift card to each student in a team:


### **4 Outcomes**

### *4.1 Trends Generated from the STEM Idea Harvest*

The future of innovative learning environments within each aspect of teaching and learning presented challenges, barriers, or problems for students. Many of these were self-evident, such as the high cost of study materials or the lack of access to digital class notes. Others were more subtle and may only affect certain individuals or populations, such as students with physical or emotional disabilities and access to class materials while being deployed in the military. The following list represents a sampling of the recurring themes captured from the participants' notes.

**Content Presentation** Context Presented to Students: "When considering how the course content is managed and taught in a course, what issues, barriers, challenges, or problems do students have? For example, it may be the cost of materials, the timing of when you can get the materials or even the weight of the books! Include in this question the "teaching" of this course material. For example, too many lectures, not enough review time, lack of access to the professor, etc."

Common Themes of Participant Input:

• Time—Concerns were expressed around the impact of too little or too much time on a course. In general, when a set timeframe is established for all students, some will feel rushed through the materials, while others become impatient to move ahead. This perennial dilemma causes gaps in understanding or inhibits the ability to fully apply content knowledge within a real-life perspective.


**Interactions and Communications** Context Presented to Students: "Thinking about the interactions and communications surrounding a course, what issues, barriers, challenges, or problems do students have? For example, not having easy access to the course schedule, not getting regular updates on course changes, not having access to other students' contact info, etc."

Common Themes of Participant Input:


**Learner Activities** Context Presented to Students: "The student activities include course assignments or requirements that encourage students to apply, practice and further develop facility with the course objectives. What issues, barriers, challenges, or problems do students face in this aspect of the course? For example, lack of time to complete assignments, no support available when needed, etc.…"

Common Themes of Participant Input:


**Assessment** Context Presented to Students: "Courses always include some method(s) of assessing progress in the course. Other methods may be used such as midterms or fnals to determine the fnal grade. What issues, barriers, challenges, or problems do students have in regards to the assessment and evaluation methods used in a course?"

Common Themes of Participant Input:


**Co-curricular Activities** Context Presented to Students: "Many times courses will include materials or experiences beyond the scope of the actual course. These may be related to the course content while other times they are "nice to haves" but not critical. For example, attending a lecture or presentation given by a visiting scholar or joining the BioResearch Club. What issues, barriers, challenges, or problems do students have with co-curricular activities?"

Common Themes of Participant Input:


### *4.2 Amazing Idea-Solutions*

Five teams of three or four individuals were organized as an output of the aspects of teaching and learning Idea Harvest. Each team chose a single aspect to use as the context for their idea-solution. The participants' thoughts garnered on post-it notes through the gallery walk became the basis for the day's work. The remaining sequence of the day's events guided the teams through the ideation process in order to present a fnal idea-solution for the Jam competition.

**Content Presentation** Team Stuff designed a "Fitbit"-type device that could track a student's learning pace and could suggest "learning time" that an individual should devote to a course or task.

**Interactions and Communications** Team WhatsApp conceived an app that would connect students 24-hours a day with either live or AI tutors worldwide to provide assistance clarifying content or requiring help related to course materials.

**Learner Activities** Team To-Do addressed a modifcation to Learning Management Systems via a survey to customize student's learning activities to their hobbies, interests, or careers making them more meaningful and engaging. This idea was premised on the belief that students would be more interested and motivated with their learning when it directly interacted with their personal interests.

**Assessment** Team Measure's idea consisted of creating a "controlled virtual reality environment" for administering formative assessments to students requiring special accommodations (e.g. ambient music and relaxing atmosphere, tasks assessed virtually, track eye movement with Oculus to denote attention, etc.)

**Co-curricular Activities** Team Extra Extra formulated an app called "Mentor Tinder." This app would consist of a database capable of matching a student with appropriate mentors that shared their interests and expectations. These mentor relationships could be local, regional, or even international.

### **5 Recommendations**

The X-FILEs Jam was considered a success in gathering input from the end users, the students. There are features of the event that, slightly modifed, may have produced different outcomes. The research team makes the following recommendations seeking to improve the experience for students as well as more closely align with the original X-FILEs project goals.


**Fig. 4** X-FILEs Jam Organizers and Participants

### **6 Conclusion**

Gaining the insights and input of the end user of any system is a critical step in product design. Soliciting the student voice in describing and envisioning the future of STEM education through a Jam event proved to be an effective and educational experience for all. The success of the program was largely due to the willingness of the participants to address the challenge by generating creative and practical idea-solutions. Although the number of student participants was less than anticipated from preregistration data, they dedicated the day to working as a team to consider how to improve the state of STEM education through the use of the four technology categories. The quality of their presentations of idea-solutions validated the process and the efforts of each team.

Capturing the student output through the transcription of notes and drawings provided a glance into the ideation progress and outcomes from each team. The positive interactions among participants and their willingness to consult with the X-FILEs research team served as a valuable guiding force throughout the event. Final evaluations denoted a high sense of satisfaction by students, suggesting the participants enjoyed and found value in the day's activities. Evaluations encompassed everything from the accommodations to the facilitators and the overall view of the event.

As in every event offering, refection on the areas for improvement presents an opportunity to enhance and enrich the event design. The recommendations suggest minor adjustments to a Jam event in order to more closely align the designed activities to the larger program goals.

### **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# **Index**

#### **A**

Adaptive controls, 20 Adaptive learning, 109, 111 Amazing Idea-Solution Process, 125, 128, 129 *The Amazing Race*, 124 Analytics, 19, 20 Artifcial intelligence (AI), 2, 79 US Education, 80 Assessment and evaluation, 91 challenges personalized learning, 94 predicting performance, 93 reading/writing tasks, 94 ZPD, 94 implementation strategies, 94 opportunities personalized learning, 93 predicting performance, 92 reading/writing tasks, 92 research questions, 95 Assistive technology (AT), 41 Association of Public and Land-Grant Universities (APLU), 22, 26 Augmented reality (AR), 8, 56 Autograders, 81 Automated essay scoring (AES, 92 Autonomous education, 96

#### **C**

Chatbots, 81 Chunking, 19 Co-curricular, 2, 3 Co-curricular activities, 95 acceptance/availability, 97

assessment, 97 implementation strategies, 97 need, 97 opportunities, 96 research questions, 98 robustness, 97 Co-curricular education, 96 Cognitive information processing (CIP), 19 Collaboration, 58, 60, 62, 66, 70, 72, 74 Collaborative learning, 111 Collaborative virtual environments (CVEs), 57, 61 Communities of practice (CoP), 57 Computer-based simulations, 8 Computer-supported collaborative work (CSCW), 66 Constructivist learning, 19 Content presentation challenges content management, 83 feedback, 83 implementation strategies, 84 opportunities automated feedback, 83 multifaceted presentation, 82, 83 simulations, 83 research questions, 84 Conventional assessments, 67 Courseware in Context (CWiC) Guide, 23 Cross reality (XR) assessment and evaluation challenges, 68, 69 implementation strategies, 69 opportunities, 67, 68 research questions, 69, 70

© The Editor(s) (if applicable) and the Author(s) 2021 133 J. Ryoo, K. Winkelmann (eds.), *Innovative Learning Environments in STEM Higher Education*, SpringerBriefs in Statistics, https://doi.org/10.1007/978-3-030-58948-6

Cross reality (XR) (*cont*.) co-curricular challenges, 72 commercial XR applications, 70 experience of learners, 69 implementation strategies, 72, 73 opportunities, 70, 71 research questions, 73 content presentation challenges, 60 classroom, 59 faculty, 59 implementation strategies, 60 instructors, 59 opportunities, 59 research questions, 60 web-based instruction, 59 defnitions, 56, 57 interactions and discussions challenges, 62 implementation strategies, 62, 63 opportunities, 61 research questions, 63 key trends, 58 learner activities challenges, 65 implementation strategies, 65, 66 opportunities, 63–65 research questions, 66, 67 science classroom, 63 scientifc method, 63 virtual labs, 63

#### **D**

Deep learning, 81 Design, 59, 62, 66 Dialogic instruction, 43 Digital communications, 7

#### **E**

E-learning systems, 87 Embodiment, 57 Emergency remote teaching (ERT), 13 Emerging technologies, 20, 21 Empathy, 57, 61, 66 English-as-a-Second-Language (ES), 93 Evidence-centered design (ECD), 69 eXploring the Future of Innovative Learning Environments (X-FILEs), 107

### **F**

Feedback, 18, 21, 24, 26, 28–30 Flipped classrooms, 7 Formative assessments, 21, 27–29 Framing questions (FQ), 2

### **G**

Game-based learning (GBL), 8

### **H**

Higher education, 1, 2, 4, 6, 7, 12 Higher interactive virtual environments (HIVE), 60 Human-centered environments, 71 Human-Centered Smart Environments (HCSE), 71

### **I**

IEEE's IC Industry Consortium on Learning Engineering (ICICLE), 72 Inclusive pedagogy, 37 Industry, 66, 71, 72, 74 Innovation Disruption Analysis, 124 Innovative learning environments (ILEs), 1, 13, 79, 107 Integrated innovation hubs, 71 Intelligent systems applications, 86, 95 Intelligent tutoring systems (ITS), 29, 82 Intelligent tutors, 20 Intelligent virtual reality systems (IVRS), 9 Interactions and discussions challenges detecting/conveying, 86 integration, 86 classroom, 85 opportunities communications, 85 guided linear learning, 86 problem-solving, 86 research questions, 87 Interactive activities, 22 Interactive learning environments, 99 Internet of Things (IoT), 7 Intrusive technologies, 115

### **L**

Learner activities challenges access, 90 interactive, 90

#### Index

implementation strategies, 90, 91 intervention, 88 opportunities interactive, 89 supportive, 89 research questions, 91 Learner-centered activities, 7 Learner engagement, 37, 44 Learner experience, 56 Learning management systems (LMS), 79, 82, 98, 110 Learning paths, 18–21, 27

#### **M**

Machine learning (ML), 2, 19, 20, 79, 81, 90, 92, 93, 95, 98 Massive online open courses (MOOC), 7, 92 MicroMasters programs, 7 Mixed reality (MR), 8, 56 Multimodal instruction, 39 Multimodality, 36 Multimodal learning agency, 36 assessment and evaluation challenges, 48 course/classroom environment, 47 implementation strategies, 48, 49 opportunities, 48 research questions, 49 co-curricular challenges, 50 formal classes, 50 implementation strategies, 51 opportunities, 50 research questions, 51 cocurricular learning, 38, 39 content presentation challenges, 41, 42 opportunities, 40, 41 deep and sustained learning, 39 deep learning patterns, 36 digital technologies, 35 diverse students, 37 education landscape, 35 environment, 111 institutions, 39 instructional approaches, 36 instructional designs, 36 interactions and discussions challenges, 43, 44 dialogic interactions, 43 implementation strategies, 45 in-class lecture, 43

learners, 43 opportunities, 43 principles, 43 research questions, 45 learner activities challenges, 46 designer, 46 implementation strategies, 47 learning facilitator, 46 online and community resources, 46 opportunities, 46 research questions, 47 learning, 35, 36 literature assessment, 38 content presentation, 37 discussions, 38 evaluation, 38 interactions, 38 learner activities, 38 researchers, 36 research questions, 42 teaching, 35, 36 UDL, 36 well-researched pedagogy, 37 workshop participants, 39

#### **N**

2017 New Media Consortium (NMC), 20

#### **O**

Online learning, 7 Online learning design, 20 Online learning system, 82 Open educational resources (OER), 9

#### **P**

Personalized and adaptive learning assessment and evaluation challenges, 29 implementation strategies, 29 mastery learning, 27 opportunities, 28 research question, 30 challenges, 24 co-curricular challenges, 31 implementation strategies, 31 opportunities, 30 research questions, 31 communication, 23

Personalized and adaptive learning (*cont*.) component systems, 24 content presentation challenges, 22 implementation strategies, 22, 23 opportunities, 22 research questions, 23 defnition, 18, 19 emerging technologies, 20, 21 implementation strategies, 25 interactions, 23 learner activities activities, 26 assessments, 25 challenges, 26 implementation strategies, 26 opportunities, 26 research questions, 27 opportunities, 24 pedagogies, 19, 20 research question, 25 student learning, 23 Personalized Learning Consortium (PLC), 23

#### **R**

Real-world scenarios, 7 Recurring themes, 108 availability accessibility, 113 equity of access, 112 learner protection ethics, 114 safe/secure, 113, 114 learning community, 111, 112 student experience fexible/fuid/evolutionary, 109, 110 personalized learning, 109 student-engaged learning environment, 108, 109 Robust interactive learning content, 20

#### **S**

Scaffolding, 19, 20, 24, 26, 28, 29 Science, Technology, Engineering, and Math (STEM), 1, 107 Self-paced, 24, 29 Self-regulated learning, 22 Sensors, 7 Simulation, 73 STEM Idea Harvest, 123 Student-centered learning, 19

Student engagement, 22, 26–28, 30 Student learning, 20 Student's Choice Award, 125 Student Teams, 122, 129 Support, 58, 59, 65, 70, 72

#### **T**

Technology-enabled assessments, 28 The Interactive Material (TIM), 84 Transdisciplinarity, 57 Transmedia learning framework (TLF), 59

#### **U**

Universal Design for Learning (UDL), 36, 69 University of Central Florida (UCF), 29

#### **V**

Virtual change agents (VCAs), 25 Virtual environments, 8 Virtual labs, 98 Virtual Performance Assessment (VPA) Project, 68 Virtual reality (VR), 8, 56 Virtual worlds (VWs), 56

#### **W**

Wearable technology, 10

#### **X**

X-FILEs approach assessment, 3 categories, 2 cocurricular, 3 communications, 3 components, 1 computer-based simulations, 8 content presentation and instruction, 3 corporations and online programs, 7 digital badges, 7 digital publishing, 9 digital stories, 7 digital storytelling, 6 education technology, 6, 10, 11 ERT, 13 factors, 1 faculty participants, 13 fipped classrooms, 7 FQ ILE category matrix, 4

GBL, 8 human emotions, 9 ILE categories, 2 impacts, 12 interactions, 3 IoT, 7 learner activities, 3 learning, 2 limitations, 12 micro-credentials, 7 mobile devices, 9 MOOCs, 7 online learning, 7 pedagogies, 2 smartphones, 9 stealth assessment, 9 STEM education, 13 student-centered learning, 6 tablets, 9 teaching, 2 technologies, 2 universities, 7 virtual environments, 8 virtual lab experiments, 8 VR program, 8 workshop activities, 4, 6 workshop agenda, 1 X-FILEs Jam assessment, 129 challenges, 122

co-curricular activities, 129 content presentation, 129 design, 123 discussions, 129 events, 123, 125 face-to-face environment, 122 goals, 122 interactions, 129 learner activities, 129 learning framework, 122 outcomes assessment, 127, 128 co-curricular activities, 128 content presentation, 126 discussions, 126, 127 evaluation, 127, 128 innovative learning environments, 126 interactions, 126, 127 learner activities, 127 self-evident, 126 recommendations, 129, 130 stakeholders, 121 STEM education, 121 student-based event, 121 teaching, 122 technology, 122

#### **Z**

Zone of Proximal Development (ZPD), 93