EVALUATION OF HAPTIC AND AUDIO FEEDBACK NAVIGATION FOR PEOPLE WITH VISUAL IMPAIRMENTS

ABSTRACT

Assistive technologies are becoming prevalent among people with visual impairment (PVIs); by definition such supports can increase these individuals’  autonomy in their daily lives. Examples include Artificial Intelligence powered applications, applications using Remote Sighted Assistants, Braille Technology, Screen Magnifiers and Readers among many others. Among these technologies, Remote Sighted Assistant applications are garnering considerable attention; these use the novel paradigm of a conversational agent who guides the PVIs through a video-mediated conversation. My thesis project investigated novel augmentations for a Remote Sighted Assistance service called AIRA.  The specific goal was to support PVIs in navigation within the task context of grocery shopping (moving around a store but also acquiring target products). Working with a larger team, I investigated a wristband that provides haptic (vibration) feedback (with the help of the four attached vibration motors that are controlled by an Android application) to the participant for obstacle warning and hand navigation. The haptic information is combined with audio (spoken) feedback from the Remote Sighted Assistant who communicates through a video call; this enables the agent to see what is in front or around the participant and provide corresponding guidance. The prototype was tested on sighted, blindfolded participants to examine its strengths and weaknesses, and to gather suggestions for future device improvements. The valuable feedback received from the participants will be helpful for both our own future work and that of other researchers working on these kinds of system

 

Chapter 1 Introduction

In 2018, WHO (World Health Organization) (“Vision impairment and blindness”, 2018) reported that around 1.3 billion people have some sort of visual impairment; as these numbers increase, so does the need for assistive technology to help them in their daily lives. As most of the people are aware, PVIs usually go about their work using a cane or guide dogs. Over the last couple of years assistive technology have also been making waves amongst them.

It is no surprise that our sight plays the main part in our daily activities, especially in navigation and understanding direction that is essential in our day to day lives. And PVIs have to rely on many different aids such as dogs or canes to help them in this task. This becomes worse when PVIs have to visit unfamiliar places, or if there is an obstacle on the road such as pothole or branch of a tree or a rush hour traffic as they have to use their other senses as well to interpret any danger ahead (Giudice et al., 2008) as these aids do not always work perfectly.

Furthermore, humans can be used as aids as well but PVIs don’t want to burden their family members or friends all the time with their daily tasks and like to be independent. PVIs value their independence and don’t prefer relying on other or asking for help all the time. And sometimes it is possible that someone is not present to help them (Bigham et al., 2011).

These issues are why assistive technologies are becoming really popular such as applications using applications using AI, Remote Sighted Assistance (RSA) and haptic feedback. We focused on RSA due it garnering fame in the recent years as it gives the PVIs the independence to do what they want to do without relying on anyone else. It includes a remoted, sighted assistant or agent that describes the visual data via speech or text. RSAs can provide help in many different ways, such as locating objects in their house, reading, navigation, social interaction (Lee et al., 2018) etc. However, the issue with them is that these agents can only describe so much to the participants (Lee et al., 2018) in a limited amount of time and due to which RSAs experience difficulty in that department. This prompted us to find improvements in the current RSA systems to make better and improved prosthetics and led us to combine haptic feedback as haptic cues are proven to be faster than the conversational ones.

In this paper, we discuss the usability study of the prototype that uses remote sighted assistant or agent (RSA) paired with haptic feedback. Haptic feedback was added as sometimes just conversation is not direct or prompt mode of communication whereas haptic or vibration can achieve that. Our study is also an example of derived form of Wizard of Oz where the participant knows that another human is used as an RSA. And on the other hand, as the navigation system is not fully developed and is still in a prototype phase, so team of people are needed for assistance.

The main crux of the paper is to test the system on sighted, blindfolded participants while grocery shopping as there is a lot of unpredictability in super markets and many events can pose threat if not properly taken care of, and then getting their feedback in understanding if adding haptic cues to the existing RSA make it a better design and the prototype is actually helpful and clear in conveying the information to the participants.

The paper is organized in the following way: the next section will talk about related works using the organization of prosthetics, AI assistive technologies, Remote Sighted Assistance and haptic feedback devices and issues with them. The following sections will focus on the description of the device and the background work and their results. After that, the section on the testing the prototype on the participants and their evaluation. In the end, discussion and future work to discuss the improvements and suggestions to make the prototype better for the PVIs.

EVALUATION OF HAPTIC AND AUDIO FEEDBACK NAVIGATION FOR PEOPLE WITH VISUAL IMPAIRMENTS

Sharing is caring!