Unheard - Haptic Driver Assistance

Using Haptic Interaction to optimize the driving experience for deaf drivers


  • Msc Design in IxD, Interaction Design Project 3
  • Academic Project
  • Role as IxD Designer


  • Research Dossier
  • Vibrotactile Feedback Graphs
  • Project Communication Video
  • GUI Prototype
  • Physical Prototype

Project Specifications


Set 2022 – Jan 2023


  • Figma

  • Photoshop

  • Illustrator

  • After Effects

  • Arduino

  • Blender


The Briefing

In the final stage of my Interaction Design master’s program, the concluding Interaction Project course centered on the notion that designers serve as more than just solvers of problems – designers are also societal analysts. Within this framework, the class embarked on a mission to identify facets within society where intervention wasn’t about solving problems, but rather about creating design interventions that add value.

Participants were encouraged to envision interaction design projects that would cater to a specific target audience, generating tangible benefits. The emphasis lay on free exploration of these design possibilities within the broader societal context, fostering a holistic understanding of design’s role as both a creator and enhancer of experiences.

The Proposal

I identified a unique intervention space at the intersection of mobility, specifically car interface design, and the principles of accessibility and universal design. During my research, I discovered a significant challenge faced by the deaf community while driving. Although they have the freedom to drive, they often contend with an elevated cognitive load due to the absence of auditory cues. Our hearing, being the second most immediate sense, is subconsciously employed for numerous tasks, including driving.

I set out to design a modular sensory substitution interface. This solution aims to empower deaf individuals to drive more securely and confidently, ultimately enriching their overall driving experience. By leveraging alternative sensory cues, this interface alleviates the cognitive burden associated with auditory deprivation.

Stage 1. Research

Literature Review


Conducting swift and effective research, I primarily relied on secondary sources. The literature review covered topics like Calm Technology, Assistive Technologies, Sensory Substitution, and Car UX. I also sought benchmarking examples, specifically focusing on products or interfaces designed for deaf drivers in cars. 

Although direct matches were limited, this process informed the development of my modular sensory substitution interface, ensuring a well-informed and innovative solution to enhance the driving experience for the deaf community.


  1. Despite the growing prevalence of implicit car technologies, accessibility has received limited attention.
  2. Minimizing cognitive load requires feedback that is easily perceived and understood.
  3. The interface must avoid being intrusive to ensure the vehicle remains drivable by others.

Stage 2. Product Definition

Project Statement

Make the driving experience for deaf and hard-of-hearing drivers safer, more comfortable and confident.


The interface’s core concept hinged on harnessing contemporary advancements in AI, object detection, categorization, and real-time processing. This innovative approach aimed to “translate” auditory environmental cues surrounding the vehicle into a streamlined vibrotactile syntax. This feedback was then strategically delivered at the points of contact between the driver and the car.

User flow

Interaction Model

This project revolves around the concept of implicit interactions, where the user engages not only with the interface but also with the surrounding environment. As a result, the system responds to these environmental interactions by providing tactile and visual feedback.

Stage 3. Materialization

External Stimuli

The ultimate objective of this project is to enhance the overall driving experience for individuals who are deaf by implementing sensory substitution technology. The initial phase of this endeavor involves identifying the key auditory and audio-visual cues that may be compromised or entirely absent for deaf drivers. To facilitate this, we have developed a taxonomy consisting of three levels of significance:

  1. Danger: This category encompasses the most critical stimuli, representing those that pose the highest level of danger if missed.
  2. Alert: The second tier focuses on stimuli that serve as important alerts, helping drivers stay informed and attentive to potential hazards.
  3. Assistance: At the third level, we address non-dangerous stimuli that offer assistance, enhancing the overall confidence and enjoyment of the driving experience when conveyed effectively.

By classifying stimuli into these categories, we aim to prioritize the development of sensory substitution solutions that address the unique needs and safety concerns of deaf drivers.


  • Horns (A)
  • Sirens (A)
  • Imminent Collision (AV)


  • Approaching Vehicles (AV)
  • Approaching the Kerb (A)
  • Pedestrians (AV)


  • Internal Sounds (A)
  • GPS (AV)
  • Parking (AV)

(A) – Auditive | (V) – Visual | (AV) Auditive and Visual

Properties of Vibrotactile Feedback

To encode various external stimuli into vibrotactile feedback, we employ the four fundamental vibration properties commonly employed in Design Systems like those of Apple and Microsoft. These properties encompass granularity, amplitude, timbre (including ease-in and ease-out characteristics), and the precise location of the vibration sensation.





Visual Representation of Vibrations (Audio-Tactile Skeumorphism)

The subsequent challenge involved encoding all the categorized threat stimuli into a vibrotactile pattern. Given the necessity for visual presentation, I devised a visualization method that incorporated the four key aspects of vibration. To ensure inclusivity for individuals who may have acquired deafness later in life and those who are not deaf but may use the interface, I meticulously crafted each vibration pattern to mimic specific characteristics of the stimuli it aimed to represent. A clear example of this can be observed in the ambulance graph, which replicates the sound of a siren.

Graphical User Interface


GUI Prototype

Communication Video

Physical Prototype

To create the physical prototype, a Tinkercad Arduino diagram was designed. This diagram facilitated the activation of six consecutive vibration patterns, originating from the user’s upper back and traveling down to their hand on the steering wheel. The setup utilized an Arduino Uno connected to a breadboard, which in turn triggered six micro vibration motors strategically positioned on a car seat cover and a steering wheel cover.

During the user testing, participants were instructed to simulate being in a car, sitting in the driver’s seat and holding a steering wheel. Next, on a tablet placed on the table, users interacted with a graphical user interface (GUI) prototype and activated the UNHEARD system. Simultaneously, on a computer screen, a first-person point-of-view (POV) video of a driver’s perspective while driving was displayed.

The critical scenario unfolded when, without any audible sound, an ambulance suddenly approached from the left, necessitating an immediate right turn by the driver. The prototype user was alerted to the presence of the ambulance solely through precisely calibrated vibrations, which commenced just moments before the ambulance came into view.

Final Considerations

this project stands out as one of the most captivating and multifaceted endeavors I’ve undertaken as a designer. I hold the belief that technology serves as a pivotal way to enhance accessibility, seamlessly integrating into our daily routines to empower us to not just do less, but to do better.

Given the opportunity, there are avenues for further refinement in this project, such as through user testing and the enhancement of existing prototypes. I aspire to continue working on projects of this nature in the future, driven by the aspiration to contribute to a more inclusive and technologically enriched world.

Scroll to Top