Using a virtual reality sandbox as a teaching tool

This story originally appeared in the March 2023 Innovatus Issue of YFile. Written by Elaine Smith.

By the time students enter York’s Lassonde School of Engineering, they’re long past the age of playing in sandboxes – or so they believe. Mojgan Jadidi and her colleagues have turned that assumption on its head.

Jadidi, an associate professor in the Department of Earth & Space Science & Engineering, has created a virtual reality sandbox (XR Sandbox) teaching tool that builds on an augmented reality physical sandbox (AR Sandbox) devised by faculty at the University of California Davis (UCD).  

“I was thinking about the first-year LE/ESSE1012 Earth and Environment course that I was teaching to the engineering students from all civil, geomatics, mechanical and space engineering programs,” Jadidi said. “It’s a very dry and heavy theory course and I have always wanted to provide the students with something cool and fun to experience and learn.” 

Using the UCD AR sandbox students can sculpt terrain in a physical sandbox and, in real time, generate and project a topographical map onto it to replicate the landscape of a specific area. Since it is an open-source product, Jadidi built the system at Lassonde machine shop and tailored it as her own version.  

She discovered that there were additional features she wanted to include, such as adding artifacts (e.g., logo blocks), detecting man-made objects on the AR sandbox, exporting the 3D scene that students build as 3D mesh, and many more functionalities. In addition, Jadidi was eager to expand the use of AR Sandbox beyond the first-year classroom to all Lassonde programs. She reached out to colleagues Melanie Baljko, an associate professor of electrical and computer engineering; Usman Khan, an associate professor of civil engineering; and Matthew Perras, an associate professor of civil engineering, to join the project and received an Academic Innovation Fund (AIF) grant to improve the tool. Their goal was to add other functions so the tool was useful to students in a variety of courses and engineering disciplines, providing for real-time, 3D geo-mapping. 

Then came the pandemic. 

“Disruption equals opportunity,” Jadidi said. “We decided to re-create the sandbox in a virtual/gaming environment. Now, using the virtual game sandbox, the tool is accessible to all students via web browser. They can mimic water simulation and flooding, for example. I’ve also tried it using a virtual reality headset, and that gives us endless opportunities. We can go to different locales, such as the Grand Canyon to look at the layers of soil and rock.

“There are applications for civil engineering, but for other engineering disciplines as well. Electrical engineers can create a circuit network, for instance, play with the components and see their design in a more immersive way. Mechanical and space engineering students can assemble a drone and fly it – and there are many more options.” 

Jadidi has applied for another AIF grant to expand the project to all Lassonde programs and refers to it as the Augmented and Virtual Reality (XR) Sandbox. 

“The XR Sandbox is an inclusive, diverse learning environment that helps students to retain information,” she said. “I want students to be able to use all three versions of the XR sandbox: augmented, virtual game and virtual reality. The physical sandbox is ideal for learning tangibly; the gaming version is good for remote learning and for redoing an experiment without time constraints to allow students to learn from mistakes; and the virtual reality version gives more immersive information to students so they can experience things they can’t always access in the physical world.  

“For example, they can simulate flooding in the Toronto downtown core (an application is under development) or simulate an earthquake at the Grand Canyon and see how the different geological layers respond. It allows them to think about the future and see the implications of their designs or decisions.” 

Jadidi, whose own research focuses on 3D data integration, analytics and digital twins, has had success using the XR Sandbox in her courses. She has also created a 3D game to teach land surveying. She created it early in the pandemic in response to the need to avoid field-based class cancellations. 

“The surveying gaming environment helps students to be prepared for physical tasks while they were on the field for surveying,” Jadidi said. “This generation is comfortable with the technology, digital world and gaming environment, so we are talking the same language as they are.” 

Perras, too, has incorporated the XR Sandbox into his geological processes course, LE/CIVL 2160, taken by second-year civil engineering students. 

“Civil engineers need an understanding of the environment in which we build things, and we can’t always go out into the field to show them everything,” Perras said. “With virtual reality, there is an opportunity to create landscapes and project different geological features onto the terrain, which helps bring things together for the students.” 

His first opportunity to use the XR Sandbox in class came last term when Perras was able to use it to replace a problem his students tackled on paper prior to the pandemic. The problem required them to look at a site that needed to be excavated for a building and determine the type and volume of material involved.  

“It was hard for students to use a topographical map with geological observation points to visualize a three-dimensional site, but now, the sandbox allows them to do the problem in 3D,” Perras said. “The system helps quite a lot with complex problems, although there’s still a learning curve for both the students and me in using it.” 

Jadidi continues to share the XR Sandbox developments with the engineering world, presenting its innovations at national and international engineering education conferences. The XR Sandbox earned the best poster award at the 2022 American Society of Engineering Education Saint Lawrence Conference. This year, she will be speaking at the American Society for Engineering Education and the Canadian Association of Engineering Education’s annual conference, showcasing her recent developments.  

The XR Sandbox and associated applications are examples of how Lassonde is empowering its students by familiarizing them with creative learning tools.