FLAG - Flexible Learning Advisory Group

Augmented Reality for Kitchen Orientation and Safety Procedures

TAFE NSW North Coast Institute

Contents

Augmented Reality - Kitchen Orientation and Safety

Organisational overview

One of the ten Institutes of TAFE NSW, North Coast TAFE consists of seventeen campuses located along the coastline of northern New South Wales stretching from the Great Lakes region, north of Newcastle, through to the Queensland border. North Coast TAFE has a staff of approximately 3,100 and provides vocational education and training to over 45,000 students per annum. The Institute is a Registered Training Organisation (RTO) under the National Regulator ASQA and maintains International Standards Organisation (ISO) 9001:2000 quality accreditation. The Institute offers a broad range of qualification levels in most industry areas and, with an emphasis on providing innovative and flexible learning solutions, is responsive to community, business and industry workplace training requirements.

back to Contents

Trial overview

Up until now subjects like Kitchen OH&S have been taught in a classroom (sit down) environment. The amount of information retained, particularly during an emergency is low. It’s the aim of this project to make the learning more relevant and take the user directly to the safety item (for example the fire extinguisher), have them scan the object, and receive a tutorial on using the equipment in an emergency situation. It’s hoped that by doing this it will also help the student remember the procedure, due to the active learning process that he or she has been through. It’s also an objective of the trial to incorporate mobile technology as a delivery tool. By creating an Augmented Reality (AR) environment we address the two objectives:, active learning and mobile technology.

It has been a previous issue with AR that without Global Positioning System (GPS) reception the platform is unusable. With this project we will attempt to use the latest method of AR referencing, using objects instead of geo-location. This will enable the AR process even if we are under several floors, or there is poor GPS satellite reception. The main requirement will be access to the internet.

back to Contents

Trialing the technology

Research

Our technical expert has keenly observed Augmented Reality (AR) over several years however it’s since the inclusion of image recognition rather than the reliance on geo-location technology that the decision was made to investigate Augmented Reality in a more serious way.

The project team has since discovered that using AR leads to improved retention and comprehension of information.

“…this research explores the potential of AR to advance visualization tools in education and for the design and development of learning technologies. AR interfaces do not merely change the delivery mechanism of instructional content. They may fundamentally change the way that content is understood, through a unique combination of visual and sensory information that results in a powerful cognitive and learning experience
Using Augmented Reality for Teaching Earth-Sun Relationships to Undergraduate Geography Students

Brett E. Shelton
College of Education
Program for Educational Transformation Through Technology
University of Washington

Nicholas R. Hedley
Department of Geography
Human Interface Technology Laboratory
University of Washington

back to Contents

Implementation

The Augmented Reality content has been created using an online AR builder called buildAR which then supplies the information to Layar, the AR hosts. We have an account with both companies. BuildAR offers a graphical web based interface that does all the coding for the user. It’s a process of selecting a reference image and linking to the relevant content. Without using a third party app like buildAR, coding experience would be required.

There are essentially four main steps to creating an AR layer. They are:
  1. Create a clear reference image
  2. Plan and create the user interface (layout)
  3. Attribute resources to the AR item
  4. Test and publish

Create a clear reference image - It is important when creating the reference image that it contains good contrast, has no flash flare and the true real life dimensions are attributed to the image, that is, if the image is 10cm in real life then enter the size dimensions as 10cm. The reference image is then uploaded to buildAR who then send it onto Layar. A simple example of the process can be viewed here.

touch icon - target sighted, touch to viewPlan and create the user interface (layout) - Once loaded and scanned the next steps involve creating an AR banner or icon. This is what the user will see imposed over the object on the mobile device. For this project I have simply used a green hand without a background. The menu at the base of the screen indicates what the object is and that the user should “touch here” to commence the AR experience.

Attribute resources to the AR item - The next step involves attributing graphics to the toolbar, text colour and background colours. Then it’s onto linking the content to the image. The main content for this project has been video. Each “safety” item around the kitchen has had a Video created for, or one sourced to help demo that items use. Some of the videos have been created by North Coast TAFE, some by private providers and some by Futura training.

Test and publish - The final stage of creating the AR layer involves testing before publishing. This is possible using the AR App “Layar” on a mobile device. The developer logs into their account and all the features are made available for testing. Once the developer is happy, the Layer is requested for publishing or “going live”. This process takes approximately 3 working days.

Testing with students - With the created layers in place it’s now time for real world testing. The reference images are all identified within the app, so students will know what images to scan.

back to Contents

Images listed to scan

Images listed to scan

Reference image interface

Reference image interface

Video tutorial

Video tutorial

Once the image is scanned the reference interface appears (Reference image interface above). From here the user selects one of three options, Support site, Email developer or Watch this video. Video tutorial above shows the video option.

Safety areas supported are:

  1. Hand wash procedure
  2. Using a fire extinguisher
  3. Using a fire blanket
  4. Shutting down the gas supply
  5. Shutting down the electricity supply
  6. Making an emergency phone call (mobile phones only)

back to Contents

Classroom trial/s

Equipment used - We used 10 iPads, 2 Android Tablets and 2 iPhones for the student testing. We had both 3G enabled devices and campus Wi-Fi for internet access.

The trial - The classroom trials had two separate but demographically similar cohorts participate in the unit SITXOHS001B Follow Health, Safety and Security procedures. Group A participated in a traditional face to face lesson while Group B participated using the AR technology in the kitchen. The content for both classes was exactly the same however the technology used for the delivery was the variable factor.

A period of two weeks was maintained between demonstration and assessment. We anticipated the retention of information of Group B would be proven in the results of the assessment event. Unfortunately we believe the results were skewed due to poor bandwidth. In addition to the assessment we asked both groups to participate in a survey with questions based around levels of engagement and quality of content.

We anticipated the results from the survey would confirm our original research and provide us with evidence that using AR technology provided the student with a more engaging and transformative learning experience. The results however were too close to clearly state AR has a certain benefit to learning. As mentioned, I believe the lack of bandwidth greatly reduced the user experience, and as a result misrepresented the potential of the technology. The outstanding feedback from AR users was "great idea, shame it doesn't work...". this was in reference to the devices not accessing the content, being slowed down to dial up speed or - even worse - no connectivity at all.

back to Contents

Implementing the technology

Evaluating the technology

Were we satisfied with the technologies used? At the end of the trial, while not all testing ran according to plan, we still remain satisfied with the technology used. The technical difficulties that we experienced seemed to have more to do with the bandwidth issues brought about by the TAFE firewall and old network hardware than with the Layar app or the mobile technologies and devices used in the trial.

Did the technologies functions and features meet the requirements of our trial? Somewhat. While the app and devices were available we feel that they were severely limited by the TAFE wireless network restrictions.

Additional functions and features we would like the technology to have. We would like to have had the opportunity to use some green screen video so that we could have objects or even a host appear as an overlay to what the users are viewing. This would add another dimension to the use of this style of technology for instruction.

Will we use these technologies in the future? Abosolutely. We have plans to continue with Augmented Reality to build a database of 100 OH&S objects institute wide, and we would also like to use AR in a workplace learning app for mines and construction workers.

Addtional details about the technology. It is worth mentioning that mainly iPads were used in the trial as these were available to lend to students, however in reality the most effective device for using this technology is definitely a phone rather than a tablet. The individuals who did have a phone were able to access the AR Layars with a minimum of fuss; additionaly the phone was less cumbersome to use than the tablet.

Positives:

  • Students were engaged with the content
  • AR Layars displayed content on the spot and within context
  • Content could be tailored to the specific group of learners
  • Students became aware of the location of an object.
  • Mobile technologies and (importantly) BYO device technology has been utilised.

Negatives:

  • Students weren't prepared enough in advance of the trial as to how to use the technology
  • Current infrastructure couldn't support so many devices on the network at once.
  • Network firewall blocked a lot of content.
  • Firewall was the main impediment to content delivery.
  • Students had a negative view of the technology due to our poor bandwidth.
  • Some images were difficult to scan due to fluctuating lighting conditions.

back to Contents

Survey

The following questions and results were collected after the live trial.

Do you think Augmented Reality (AR) is an appropriate teaching tool for safety?

  • 40% Yes
  • 20% No
  • 40% Maybe

Maybe responses include…“I couldn’t access the site”, “the scan wouldn’t work”, “not easy to use”, “thought the visual learning easy to understand”

Do you think AR is a better medium than previous ways you have been taught safety?

  • 33% Yes
  • 33% No
  • 33% Maybe

Maybe responses include… “I don’t have a device”, “like being able to refer back later”, “not all ARs worked”

What other areas do you think AR would be a good teaching tool?

Responses were… “tools and machinery”, “cool room and dry store”, “a pre workplace safety training”.

Do you like the idea of mobile devices being used in teaching and assessing?

  • 33% Yes
  • 13% No
  • 53% Maybe

Maybe responses include… “if the devices were provided”, “Maybe not as an assessment”, “enjoy using iPhone”.

back to Contents

Assessment results

Students were assessed on their knowledge of Kitchen Safety two weeks after the AR experience. We also had a control group that had a traditional sit down in class delivery method used.

The findings were disappointing. Due to poor network capabilities, students became focused on AR's inability to work rather than discovering it as new alternative for learning.

The control group and the AR group had roughly the same unit completion rate.

Our current poor campus bandwidth is being addresses. It's anticipated that within one month of writing this report our Wi-Fi capabilities will be more than adequate to support technologies like AR. In fact it's a result of these trials that ICT are now looking at unauthenticated internet access for mobile devices. Authenticated internet access was a major hurdle given that it slowed connectivity (data had to go through a proxy filter) and restricted video file types for the AR user. Some video file extensions such as the universal mp4 couldn't get through our firewall.

Despite all of the issues, we are now pursuing AR in areas such as course content delivery, assessment and marketing.

back to Contents

Strategies to enable use of the technology

Strategies to support teachers, staff and students:

  • Working with teachers to identify key objects or areas for inclusion in AR projects
  • Providing teachers with templates to organise information for inclusion on AR
  • Providing teachers with simple help guides to allow them to clearly photograph objects in an appropriate way for use in AR
  • Looking at other ways to use the technology, particularly as an addition to currently produced written material. Scanning fluctuations would be overcome and the user would have an interactive book-like experience as shown in this video(Opens in new window).

Discoveries through the trial

  • The technology is new and constantly evolving. Between the time of commencing the project and now at conclusion there have been two major developments. The first is the ability to overlay green screen video and the second is to deliver AR in a magazine like format. Here you can see the two new developments in action.
  • Students love the potential of AR, and they're very keen to use their mobile device in the classroom as a learning device.
  • Not all objects are scannable. Objects that have fluctuating lighting, poor key features (one colour with little definition) or have multiple angles tend not to scan very well.
  • It works very well on the current 3G mobile network. I'm sure once the NBN and 4G networks are available, better content with larger file sizes will follow.
  • At the moment video sizes are held back to around 10mb each

As a result of the trial our current network firewall is under review. It's becoming loud and clear that emerging technologies like this need to be utilised in VET training, without the imposed barriers of an authentication firewall.

It's anticipated that the AR delivery will soon move from mobile devices to Smart Glasses. It's belived this will occure in the next 3 - 5 years.

View this example of Smart Glasses from Google.

back to Contents

Barriers to implementing the technology

Barriers to the project have been:

  1. DEC firewall - throughout the project this has been the main hurdle to consider. Due to the firewall requiring authenticated access, it's not always a smooth experience for the end user. We have found for example video content recorded in the recommended format (H.264 MP4) will not get through the firewall. However by chance we discovered video recorded in H.264 M4V did get through. It's examples like this that end up costing more money due to having to work out a solution.
  2. Older Android devices were unable to display the video content (those older than 2011 models). A great issue in supporting the hundreds of Android phones currently on the market is the problem of different hardware capabilities.
  3. Student familiarity using a mobile device in this way. It's a new app to use and therefore requires some initial training to become effective as an educational tool.

Additional considerations

Whilst the technology works on all mobile devices it's not been a comfortable experience when holding a large device like a tablet. The need to touch the screen and hold it up becomes tiring for the user. It's been noted that a smart phone seems to address this issue due to its smaller form and lighter weight.

back to Contents

Future directions

Augmented Reality is viewed as one of the top five technologies that will shape our learning environment over the next ten years. The evolution from geo-referenced AR to image-referenced seems to make way for a synergy of the two rather than one being better than the other. The Google glasses project looks set to incorporate these two AR systems as well as other web based services like email/calendar, video chat and mapping services.

So with that in mind, it's encouraging to extend the AR experience further. North Coast TAFE has embarked on incorporating AR into more areas of study. We are now looking at extending the AR program to include all our faculties covering Occupational Health and Safety. Each section is being shown the current examples and being asked "could you utilise this type of delivery"?

The use of AR in VET training has huge potential (refer BMW AR video). Students with mobile devices can use the technology regardless of whether they are on campus or at work. It will lead to a reduction in time that students sit down in class and listen to the teacher. Instead the learning experience will be active, involved and contextual, helping the learner retain the newly learnt information.

back to Contents

Additional Materials

Resources

BMW AR for Mechanical repairs

The future of education

Kids clothing with AR - body movement recognition

back to Contents

Video Diary

First test video

Firewall blockage - DEC

Project Trial

Demo Video AR

back to Contents

For More Information

Elizabeth Everingham Miller
Project Manager
p: 02 65862217
e: elizabeth.everingham-miller@tafensw.edu.au

Tyrone McGillick
Technical Expert
m: 0447468618
e: tyrone.mcgillick@tafensw.edu.au

back to Contents

Strategy contact information

For general enquiries about the National VET E-learning Strategy, please contact:
Secretariat of the Flexible Learning Advisory Group
+61 (0)3 9954 2700
Email: flag_enquiries@natese.gov.au
Website: flexiblelearning.net.au

New Generation Technologies for Learning incorporating E-standards for Training
Email: e-standards@flexiblelearning.net.au
Website: e-standards.flexiblelearning.net.au