Introduction

While it is a common misconception that Visual Impairment (VI) is a single form of disability, there are, infact, several categories under this label. Drag the green sliders over the images below to get some idea of what it is like to have a type of visual impairment.

.

Cataract

Glaucoma and

Retinitis Pigmentosa

Macular

Degeneration

Diabetic

Retinopathy

Homonymous

Hemianopia
Simulations Taken From Ohio Lions Eye Research Foundation

Naturally, with one or more these conditions, visually impaired people face issues in many basic daily living activities like navigating around places, cooking and eating, locating their belongings and finding readable material, to name a few. A physical fitness routine is generally the last thing on their minds owing to added factors such as altered balance and fear of falling.

However, there is no doubt that maintaining physical fitness reaps many benefits such as keeping us happy and less depressed or anxious. Clinical depression is a common symptom amongst many visually impaired individuals and hence exercising is of paramount importance to them.

Studies suggest that visually impaired children who were physically inactive were unlikely to pick it up later in life. Thus, the childhood of a visually impaired individual is the best time for an intervention that would make maintaining physical fitness a lifelong habit for them.



Context:  Design project for the course of 'Human-Computer Interaction Foundations'

Instructor:  Dr. Bruce Walker

Duration:  August 2019 - December 2019

Team:  Hyun Tae Park, Jae Hyuk Kim, Ruchita Parmar, Aditya Kundu, Arpit Mathur

Tools:  Interviews, Sketching, Storyboarding, Paper Prototyping, Wireframing, P5JS, Unity, Usability Testing

Contributions:  Performed competitive analysis and background research, Conducted an interview with a visually impaired athlete, Created sketches, paper and cardboard prototypes, Conducted usability testing with a visually impaired school student, Recruited participants for expert and user testing

Research

Trying to catch a ball thrown at you may seem like an easy task to most of us but it involves considerable hand-eye coordination, quick visual responses and motor skills and exhibiting such skills may not be possible for many visually impaired children. Naturally, the thought of a Physical Education (P.E.) class can get the worst out of their anxieties.

We decided to work on improving existing methods to enable visually impaired school children in performing better at P.E. class activities in school.


Preliminary Research
Getting acquainted with disabilities, accessiblity and inclusive design


  • Research on Visual Impairments

    First, we tried our best to understand the differences between various eye conditions and what each of those entail.

  • Research on Adaptive PE

    We studied how activities are currently conducted in an adaptive P.E. class. We looked up different types of modifications that are made to common place objects such as a ball, bat or even a goal post to make them accessible.

  • Competitive Analysis

    We analysed existing assisstive technologies that have been designed to help visually impaired individuals in all walks of life be it reading, writing, navigating or playing.


User Research


Due to time constraints, we were unable to recruit visually impaired children for interviews. Instead, we conducted semi-structured interviews with the following participants.

  • ○   A life and skills public high school teacher with 19 years of experience who shepherds three visually-impaired students in an adaptive P.E. class.
  • ○   A general classroom teacher at an elementary school who has a visually impaired student in her class.
  • ○   A special education teacher who runs an adaptive P.E. class in a rural public high school.
  • ○   A completely blind international paralympics athlete who is also a subject matter expert in accessibility and usability engineering.


Interview Objectives:

  • ○     To get a sense of the school and P.E. class structure and number of students.
  • ○     To find out how P.E. class activities are conducted in their school.
  • ○     To know more about the role and involvement of teachers, paraeducators and adaptive P.E. specialists in designing and conducting a P.E. class.
  • ○     To learn about painpoints, issues and concerns of adaptive P.E. teachers.
  • ○     To discover the tools and technologies used in adaptive P.E. classes.
  • ○     To get an idea of the physical fitness routines of visually impaired individuals.

Analysis

Owing to the diverse nature of our interviewees, the data collected was especially tricky to analyse

Research Findings
The most important findings that we discovered, framed from a P.E. teacher's perspective.


  • ○     "I need assistance from many people to plan and execute my P.E. classes"
  • ○     "My students and I decide activites on-the-go based on available resources."
  • ○     "I want my P.E. classes to be flexible enough to accommodate students with and without disabilities."
  • ○     "My students do not want to be treated differently."
  • ○     "My students need a solution that is tailored for their individual needs."

Ideation

We conducted a 'Walk the Wall' session for brainstorming, followed by rating the shortlisted ideas with respect to creativity and practicality



Modular Sensory Blocks

  1. A toolkit consisting of blocks of various shapes and sizes embedded with different sensor technologies. One or more blocks can be attached to regular P.E. equipment to make these tools more accessible.
  2. A mobile application for teachers to acquire step-by-step set-up instructions and guidance on making the best use of the blocks.
  3. The app stores and visualises data collected by all the sensors which can be used to track health statistics of students and also share the data with their parents.
  4. The app also serves as a communication platform between teachers to discuss and share the activities conducted, blocks and tools used and even success stories of the students in an adaptive P.E. class.


A Voice User Interface with Haptic Feedback

  1. A physical device in the form of a friendly persona with a Voice User Interface (VUI) that acts as an instructor
  2. The device provides verbal cues to guide visually impaired students in exercising and it also detects and measures the correctness of their postures using motion tracking.
  3. A companion device is a watch that can deliver haptic and auditory feedback to the student wearing it and visual feedback in the form of a green or red light to the teacher.
  4. Students can also send responses to the VUI device using the tactile button on the watch.


Sketch Courtesy: Hyun Tae Park

Simulation of Sports Virtually

  1. A digitally simulated virtual sports game to be played on a TV screen by visually impaired students and their paraeducators using controllers, motion sensors and a running pad.
  2. The controller is a hand-held baton with haptic and audio output to simulate a hitting or throwing motion.
  3. A running pad is a mat that simulates an in-place running effect.
  4. Paraeducators can customise the visuals on the screen based on the needs and abilities of the student.


Sketch Courtesy: Jae Hyuk Kim

Final Design Selection

Based on the feedback received from our course instructors and peers at the poster session, we analyzed the strengths and weaknesses of the three designs with respect to the following criteria

  • ○     Real World Implications
  • ○    Confidence on Evidence
  • ○     Range of User Groups Addressed
  • ○    Technological Feasibility




Proposed Solution

"Tell me where the left goal post is?" *Beeeeeeep*
"Now tell me where the right goal post is?" *Boooooop*


Tablet Interface

This will allow the teacher to select from a list of sports activities. Once she chooses a sport, the interface will display a general description of the game rules followed by a set of instructions on where to attach the necessary sensors to set up the P.E. class environment for the chosen sport. After this, the tablet will calibrate the placed sensors and fetch the relative distance of the goal post from the penalty spot where the visually impaired student will be standing. This tablet is exclusively for the P.E. class and does not require the teacher to make any installations or changes to their own devices in order to calibrate the sensors or monitor the game.


Spatial Sensors

Three distance sensors one of each will be attached to the left and right goal post and the penalty kick spot. The placement of these sensors is inspired by how a penalty soccer game works at a national or international Paralympics event. A visually impaired player stands at the penalty kick spot while a coach or referee taps both the left and right goalposts at different heights, one after another. These acoustics help the player determine the distance of the goal post relative to the penalty spot. It is with these sound cues that the player is able to visualize the goal post in his head and can aim to score a goal. This ability, called 'Propioception' is known to be heightened in visually impaired individuals.


Headsets With A Voice User Interface

The tablet acts as a connecting link via which the output data from the sensors is sent to the voice user interface of the headset. A student could ask the headset to tell him where the left goalpost is and it would simulate a sound effect that would seem like it is coming from the left goal post. This would allow the student to map out the position of the goal region in his head. The teacher would have to calibrate the sensors just once for a game but the student could unhesitantly request the headset to play these sounds multiple times. Bone conduction headsets unlike regular ones, transmit sound through the bones of the skull and this leaves the eardrums free to sense other sounds thereby not compromising the student's existing sense of hearing.



Prototyping

As a proof of concept, we built low-fidelity prototypes of the sensor blocks and the voice user interface based sound simulation environment.

  • We used a VL53L0X Time of Flight sensors along with an Arduino Uno microcontroller which was connected to the laptop through a USB cable.
  • We taped the sensor set-up to one of the legs of a four-legged table as a make-shift arrangement to replicate placing a sensor on a goal post and using it for calibration.
  • The distance measured by the sensor was displayed on a laptop screen.

Prototype Courtesy: Arpit Mathur

  • We recreated the game field layout on P5JS and prototyped the interaction between the voice user interface and the user using a chatbot.
  • We used the P5JS speech library to enable a user to interact with the chatbot using an audio input command.
  • We used the P5JS sound library to provide the output of the chatbot in the form of a sound effect.
  • Finally, after designing initial mockups, we built a high-fidelity prototype of the tablet interface.






Wireframes Courtesy: Aditya Kundu

Testing

We conducted expert evaluations and user-based testing separately for both parts of the solution -
the tablet interface and the sensors-headset combination


Testing The Tablet Interface


Expert Evaluation:
We performed a Cognitive Walkthrough activity with 2 second year MS-HCI students from Georgia Tech who are experts in designing and evaluating interfaces. They were provided with a list of tasks such as 'Set up the system to hold a game of penalty-kick soccer among two visually impaired and three normally sighted students in P.E. class today.' They were asked to carry out the tasks while thinking out loud throughout.

User Testing:
We conducted task based user evaluations with 2 participants who had experience with teaching students in a high-school P.E. class setting or gymnasiums. Both the participants had experience in mentoring individuals with disabilities. We provided similar tasks for these sessions as well but the objective here was to gain feedback about the solution concept from an actual user’s perspective.


Testing The Voice User Interface


Expert Evaluation:
We met with an expert from the Center for Inclusive Design and Innovation on campus. He has more than 18 years of experience as an Assistive Technology Specialist, Vision Rehabilitation Therapist and Orientation & Mobility Specialist and possesses low vision himself. We demonstrated our solution through the sketches, storyboards and prototypes followed by a semi-structured interview.

User Testing:
Our participant was a 13 year old girl studying in eighth grade who has been completely blind since birth. The venue of the user-testing session was a local park where we had set up our goalposts. The girl was carrying a walking stick and was accompanied by her mother. At first, her mother guided her to the goal region so that she could accustomise herself with the goal posts and the net. We then instructed the participant to perform a few tasks such as navigate to the penalty spot, shoot the ball into the goal by hearing sounds coming from the left and right or center of the goalposts. This was followed by an informal discussion to collect her feedback.


Future Design Improvements
The most insightful learnings that we acquired through the testing phase.


  • ○    Continuous audio streaming with a pause option instead of the current request and response audio system in the voice user interface
  • ○    An option to locate the center of the goal along with the left and right ends in order to guide the students in aligning their shoulders, toes, and nose in the correct orientation to face the goal
  • ○    A feature to track the position and movements of the goal-keeper or some indication about where the openings on the goal were to shoot the ball
  • ○    A simpler to understand and less technical description of the sensor installation and calibration process on the tablet interface
  • ○    Inclusion of every visually impaired student's Individual Education Plan (IEP) in the tablet based system for designing individualised P.E. tasks and activities aligning with their IEP goals