TEACHING MILLENNIALS WITH AUGMENTED REALITY : CASES FROM THE U . S . EDUCATION SYSTEM

This paper describes the design, development, and implementation of two separate classroom experiments using mobile augmented reality (AR) for teaching abstract topics to undergraduate architecture and construction students in two U.S. universities. The presented work is motivated by the fact that the new generation of students (millennials) have a natural ability to use technology and digital information, and show a great affinity for integrating new tools and mobile devices in their daily activities. In contrast, many existing instructional delivery methods are still teacher-centered and based on lectures and memorization to pass knowledge and information onto the students. The first project described in this paper is a construction AR magic book in which an ordinary course textbook was enhanced by overlaying multimedia contents such as images, videos, 3D models, and other visuals on top of the printed figures and diagrams of the book. The second project in this paper, describes the development of an immersive AR application called Skope, which is designed to enhance students’ site visit experience. This application, superimposes the BIM model of an existing building on the real-world building, allowing students to interact with different parameters of the model such as orientation, sitting, direction of prevailing winds, construction processes, structural systems, connection details, and the heating, cooling and ventilation systems. Overall, the assessment of student performance metrics and opinions collected in each project indicates that using AR technology in classroom has a good potential to improve student learning and engagement in the subject matter, and help them better relate abstract topics to real-world problems.


Introduction
According to a recent study, the United States is suffering from low standards of education, especially in math and science, compared to many other developed countries.The low standings in education and innovation has placed a pressure on science, technology, engineering, and math (STEM) disciplines and careers that depend on them (NCES, 2012).Several engineering education researchers have cited the lack of motivation and engagement in the learning process as a major root cause of the relatively low enrollment in STEM programs nationwide especially among minority and female students (EDZIE, 2014).Figures published by the U.S. Department of Education suggest that many of the STEM entrants left STEM several years later by changing majors or leaving college without completing a degree or certificate.For example, between 2003 and 2009, a total of 48% of bachelor's degree students and 69% of associate's degree students who entered STEM fields had left these fields by spring 2009, with about one-half switching their major to a non-STEM field, and the rest leaving STEM fields by exiting college before earning a degree or certificate (NCES, 2014).
The new generation of students is technology savvy and are very comfortable working with digital information on a daily basis, connecting to each other via social media, performing several tasks simultaneously, and playing strategic and collaborative games on their mobile devices.Yet, many STEM instructors especially those teaching subjects such as statics, equipment, construction methods, and building design heavily rely on traditional pedagogical methods which include the use of chalkboard, handouts, and computer slides that are often filled with too many words and take advantage of too little technology.This cultural and technology gap is a major impediment to student engagement in classroom activities and takes away from the quality of learning and instruction.Consequently, many engineering students complain about the lack of interaction with the learning environment.In the absence of clear strategies and examples on how to enrich existing teaching methods with the latest and greatest technology advancements, long term problems such as decreasing enrollment trends and low student retention are inevitable.Instructors must train and educate students with the latest and greatest tools and methods to prepare them for future advanced careers that are becoming more and more technologydependent.
In this paper, the authors describe their recent work on the integration of augmented reality (AR) teaching modules in the curricula of construction and architecture programs in U.S. universities.Findings will be further reinforced by presenting evidence of enhanced student learning and engagement.The paper will conclude with key observations with respect to certain aspects of technology-mediated learning, followed by suggestions for future work in this domain.

Background survey
In order to gain a better picture of students' opinions and attitudes toward technology-mediated learning, a survey was conducted on a representative sample (85% male and 15% female) of undergraduate students enrolled in civil, environmental, and construction engineering in 2012-13.A total of 241 responses were collected and results indicated that 92% of students identified themselves as visual learners.In particular, this group agreed to the statement that "I learn better when the instructor uses 2D/3D visualization or multimedia to teach abstract engineering and scientific topics".Moreover, 54% claimed that they learn better while working in a "collaborative setting (e.g.working in a team) where they can play a role in the learning process" (DONG, et al., 2013).
Despite the evidence that points to the positive effect of using portable electronic devices (PED) (e.g.laptop, smartphone, or tablet computer) on student learning and engagement, not all academic organizations are financially capable of providing these high-tech devices and tools to students.Therefore, one major concern in this research was the issue of affordability.For this reason, survey respondents were also asked to indicate if they already own a technology-enabled device that can be readily used in the classroom; 93% declared that they own either a smartphone or a tablet device or both, and can easily use it in their daily activities.In a different study, it was stated that 89% of high school students and 50% of 3rd through 5th grade students in the U.S. had access to internet-connected smartphones.Moreover, the same study showed that 50% of high school student had access to tablet computers and 60% had laptops (RIEDEL, 2014).
The fact that most students identified themselves as visual learners coupled with the large population of students who have a mobile device in their possession, motivated the authors to pursue the use of mobile AR/VR visualization technology that can be effectively integrated into portable devices.Details of three such efforts in incorporating mobile AR/VR tools into STEM pedagogy will be explained in the following sections.

Project I: Construction AR Magic Book
In the first project conducted as part of this research in the University of Central Florida (UCF), the goal was to design, implement, and assess a context-aware mobile AR tool that was used as part of an undergraduate course in construction methods.The goal of this experiment was to bring technology into the classroom by enhancing the contents of an ordinary course textbook.In doing so, the instructor and textbook were not eliminated from the learning procedure; rather they were supplemented with a new technology-based pedagogical tool that enhanced the leaning quality.The overall experimental design of the developed framework is illustrated in Figure 4.The AR application was designed based on Junaio, an open-source web-based AR experience language (AREL) programming environment.Junaio offers a free, webbased application programming interface (API) which enables users to access the AREL content and create various AR applications.The AREL package includes three different components: (1) the static extensible markup language (XML) to define all the content and linkages, (2) the Javascript logic to define dynamic parts such as user interactions, and (3) the content itself which includes 3D objects, images, and other multimedia files.The source of the AREL is identified by a channel content uniform resource locator (URL).This URL delivers the AREL XML through the mobile application.Using this process, when a user scans a QR code corresponding to a specific channel, a hypertext transfer protocol (HTTP) request will be sent to the server.The server will then forward the request to the channel content URL and responds to the request with either a static or dynamic XML.This XML will then be forwarded to the user and enables the user to receive desired content such as 3D models, images, movies, or other multimedia.The sequence diagram of the user query process is shown in Figure 2.Each channel has its unique channel identification (ID).When the application accesses a channel, it passes the channel ID to the server, and then forwards the request to the channel's content URL.The content server URL (a.k.a.callback URL) is the HTTP address of where the channel XML is created.For AREL channels that deliver static XML, the callback URL will be a simple link to an XML file.Static XML files considered as the simplest and fastest channels since the server should only provide the file without interpreting any server code.However, the channel logic is implemented in Javascript.On the other hand, in dynamic channels that return dynamic XML based on the user input, the resulting XML has to be created dynamically.In dynamic channels, there can be a database that contains the required objects.Hence, based on the input, the Hypertext Preprocessor (PHP) code could perform a database query and return all point of interest (POIs) close to the user's position.Using the AREL PHP helper provided by Junaio, the developed PHP script can create AREL XML and return it to the user (SHIRAZI; BEHZADAN, 2015a).

Experiment Design
In this project, a sample chapter from a construction methods and management textbook was enhanced by augmenting different types of virtual information (e.g.3D models, videos, sound clips, and 2D images) on existing figures, tables, and diagrams of the book (used as AR tracking images).Prior to studying the contents of their textbooks, each student uses the built-in camera of his/her web-enabled handheld device to scan a quick response (QR) code.Then, as students move their handheld devices over the images of the book, 3D computer generated and other multimedia (e.g.videos, sounds, images) appear on top of the textbook images.Figure 3 shows a snapshot of a classroom experiment conducted in this project.The ability to work in groups enables students to collaborate with their peers.It also allows teachers to form teams of arbitrary number of students, and easily implement the tool in classroom by asking students to use their own mobile devices at no additional cost.

Assessment of Student Performance
In order to assess the effectiveness of the developed AR magic book, students were randomly divided into two groups (A and B) of 8 people.Group A was the control group and asked to attend the traditional lecture, while group B was the test group and attended the technologyincorporated lecture.The two lectures were identical in terms of learning objectives and learning material.Students in both groups were not told ahead of time what to expect, so to minimize any perception bias.Considering different aspects and limitations of available assessment techniques, nine different classroom assessment techniques (CATs) were ultimately selected and used to systematically evaluate the pedagogical value of using AR in classroom.These CATs included background knowledge probe, memory matrix, categorizing grid, defining feature matrix, approximate analogies, course-related self-confidence surveys, punctuated lectures, teacher-designed feedback forms, and group-work evaluations.Detailed descriptions of these CATs can be found in Shirazi (2014).
As shown in Table 1, in post-test and long-term test, the mean grade and the standard deviation of grades for both groups A and B are very similar.However, looking at the pre-test results, it is evident that Group A (control group) had a stronger background knowledge about the topic compared to Group B (test group).In this Table, each group had 8 participants and the grades were out of 18.Given the small sample size in each group, the Mann-Whitney test was used which is a non-parametric statistics test for small sample sizes and is commonly used to compare data points of two different samples (MENDENHALL;SINCICH, 1991).The null hypothesis in this test considers similarity of the two populations while the alternative hypothesis considers the other way, especially when one population tends to have larger values than the other.Results of the Mann-Whitney test comparing pre-and post-tests indicated that the null hypothesis "values in Group B are larger than values in Group A" cannot be rejected.Similarly, comparing preand long-term tests, the null hypothesis "values in Group B are larger than those in Group A" cannot be rejected.In short, this analysis indicates that the test group (B) experienced a statistically significant improvement in short-and long-term learning compared to the control group (A).
At the end of this experiment, test group students answered an evaluation questionnaire regarding their attitude towards using AR and its impact on their learning experience.It was found that students showed more interest and motivation when AR was used in classroom.Respondents stated that they liked the interactive learning environment over the traditional lecture-based approach.However, a few students mentioned that it was difficult to work with the AR application while trying to concentrate on the lecture.All in all, the majority of students in Group B were satisfied with the new AR learning tool.In addition, responses given to two 5-point Likert scale questions revealed that most students (75%) rated AR as an effective tool and would highly recommended it to other educators (SHIRAZI; BEHZADAN, 2015b).
Further analysis also indicated that students who used AR left fewer blank answers in post-lecture and long-term tests than they did in their pre-lecture test.In particular, as shown in Table 1, the total number of blank answers decreased by 66 in post-test and 68 in long-term test for Group B, almost twice as much as the same measure for Group A (35 for post-test and 27 for long-term test).It must be mentioned that while a non-blank answer is not necessarily a correct answer, given that Group A students started with a higher prior knowledge (less blank answers compared to Group B students), it is interesting that with time, Group B caught up and ended up leaving less blank answers in long-term.This means that Group B students gained more self-confidence and better technical knowledge after using AR in their learning experience.
In conclusion, findings from this project suggest that AR can provide better learning support capabilities for barrier removal between students and technology.In addition, it provides an interactive workspace and encouraged collaboration and interaction between students and the course contents by immersing participants in the learning environment.Results of performance data analysis show that although there is still plenty of room for improvement, AR-incorporated instructional modules can be effective pedagogical tools to supplement traditional instructional delivery methods.However, potential pitfalls of using technology in classroom must also be considered.For instance, Dede and Barab (2009) stated that while users found AR interactive, situated, collaborative, and highly engaging, they pointed to its technological, managerial, and cognitive challenges to teaching and learning.

Project II: Interdisciplinary Learning with AR
The goal of the second project conducted as a part of this research was to examine if AR could help interdisciplinary leaning of Architecture, Engineering and Construction (AEC).AEC students are often trained and educated in their own disciplinary silos and do not get an opportunity to interact and collaborate during their academic life.In contrast, it is very likely that upon graduation and once entering the workforce, they pursue careers or get involved in projects that require them to collaborate.
To facilitate collaboration among AEC students, this project aimed to develop and test an AR application, called Skope.The application was produced following a design-based research approach which is an iterative process based on a feedback loop from the users.The following sections of this paper describe the development process of this tool.The project implementation and testing is ongoing and data is being currently collected for the purpose of summative assessment.
Skope is designed to create a realistic and immersive experience for learning about building systems, which is a core topic of study for all AEC students.A total of three classes from Architecture, Construction and Mechanical engineering curricula were selected to participate in this project.While each class conducted their own curriculum activities, students were required to complete three coordinated assignments or "Technical Reports" in groups.Teams of interdisciplinary students (at least one from each discipline) were put together to work and compile reports which consisted of a series of questions and problems focused on an existing building on Florida International University (FIU) main campus.
The report asked for the analysis of various building systems and recommendations on improving their performance and functionalities.
Each team met as a group on site and used Skope to gather pertinent information for completing the Technical Reports.As if having an x-ray vision, holding a handheld device, students moved around the building and viewed through the building materials, looking at its various components including but not limited to the façade system, structure, and mechanical systems.The application provided access to a 3D annotated and location-sensitive model of the facility for a better visual understanding of the building.Student teams gathered critical information on the building site by combining photos of the actual building and the actual experience of the real building.The information was then used to discuss findings with teammates and complete the Technical Reports.

Application Development Process
The Skope application production began by obtaining a partial model of the building from the Revit software (BIM model), as accessing the complete BIM model had limitations due to security concerns.The model was then exported to 3D Max software, where many details and features were added, creating an almost realistic model of the building.Simultaneous to this process, the pedagogical content of the application was also produced.This included several annotated visuals, animations, and interactive lessons of the building, used to describe building conditions such as orientation, sitting, direction of prevailing winds, construction processes, structural systems, connection details, and the heating, cooling and ventilation systems.
Upon completion of this process, the building model and the pedagogical content was exported to Unity 3D game engine to build the AR application.This game engine offered flexibility to plug in information from different software platforms as well as to support several operating systems.The application uses the mobile device's compass and GPS sensors, as well as camera to determine the user's direction and location.It then links and overlay the digital content on the actual building.
Students could download the application from Google Play or Apple App Store into their mobile devices.

Skope User Interface
As shown in Figure 4, the Skope application has an intuitive user interface consisting of a side navigation menu, an icon linking to the project's website, and a live video feed of the camera's perspective.The side navigation menu has two tabs for structure and mechanical system.When selected, each tab will activate the correlating component of the building model, revealing visual and textual information and lessons.For example, when the mechanical systems tab is selected, it enables the student to visualize the components of the mechanical system, zoom in to examine duct sizes, enter the mechanical room, and run a video showing how the heat recovery system works or visualize the flow of air through the system (Figure 5).As previously stated, the Skope project was developed using a design-based research approach, which is based on iterative testing that is pragmatic, integrative, and contextual.The process involves the analysis of the use and performance of the product being developed to understand, explain, and improve its attributes for learning (WANG; HANNAFIN, 2005).This method uses progressive refinement by testing, analyzing, and refining the design from very early stages of development (COLLINS et al., 2004).As Collins et al. (2004) states, the design-based research approach is particularly important because unlike traditional laboratory experiments, pedagogical experiments are set in realworld learning environments where there are many variables that cannot be controlled.Instead, design researchers try to optimize as much of the design as possible and to observe carefully how the different elements are working.In this project, the application of the design-based approach allows observations and analysis of student engagement with the project and collaboration during project testing, while guiding decisions with regard to project developments.
In developing the Skope application, the testing began early on with preliminary versions.As shown in Figure 6, three consecutive prototypes were given to a group of volunteer students.During an exit interview, each group was asked a series of questions including various features of the application, the quality and method of presentation of lessons and information, and their learning experience.
Once the results of exit interviews were collected and examined, students' feedback was incorporated into the next prototype.Using this approach was extremely helpful to the development process, as it allowed the prototypes to be developed in a meaningful direction for the users.

Conclusions
This paper described two technology-mediated classroom experiments motivated by students' natural affinity for mobile technologies and the use of digital information in their daily lives.It was observed that the curricula in many architecture and engineering programs still do not fully take advantage of new course delivery and instruction methods, which could be a major contributor to problems such as relatively low enrollment and even lower retention rates in AEC disciplines, and an impediment to student engagement in classroom.
In this paper, the authors presented detailed descriptions and discussions of two recently developed AR educational tools namely the AR Magic Book, and Skope.Each application was used to enhance the existing pedagogical methods in construction and architecture programs in U.S. universities.Results indicated that using technology in classroom significantly improved student learning and engagement in the subject matter, and helped them better relate abstract topics to real-world problems.
Future work in this research will include incorporating the developed AR-enhanced teaching modules in more courses offered by multiple AEC programs in the U.S.This will enable the authors to better assess the strengths and weaknesses, as well as real pedagogical values of technology-mediated learning in the presence of large student populations of diverse backgrounds and with different learning styles.

Figure 1 -
Figure 1 -Experimental design used to combine traditional and technology-based learning methods

Figure 2 -
Figure 2 -Sequence diagram of the user query process in Junaio

Figure 3 -
Figure 3 -Two users simultaneously viewing virtual contents overlaid on two different pages of the textbook

Figure 5 -
Figure 5 -Screen capture of Skope showing the mechanical Duct work

Figure 6 -
Figure 6 -Iterative design-based process for developing Skope application

Table 1 .
Statistical analysis of results obtained from pre-test, post-test, and long-term test for control and test groups