Stopping Autonomous Vehicles
with Gestures – A VR Experiment
Stopping Autonomous Vehicles
with Gestures – A VR Experiment
How intuitive gestures build trust among pedestrians toward automated vehicles
How intuitive gestures build trust among pedestrians toward automated vehicles
This study examines whether hand gestures help pedestrians safely prompt autonomous vehicles to stop. In a VR experiment, we analysed crossing behaviour in different scenarios, with and without additional communication options. The results show that gestures significantly increase trust, safety, and willingness to act, but they also introduce new challenges for interaction design.
This study examines whether hand gestures help pedestrians safely prompt autonomous vehicles to stop. In a VR experiment, we analysed crossing behaviour in different scenarios, with and without additional communication options. The results show that gestures significantly increase trust, safety, and willingness to act, but they also introduce new challenges for interaction design.
Problem:
The lack of clear interaction and communication concepts between autonomous vehicles and pedestrians led to uncertainty and made decision-making in road traffic more difficult.
Approach:
Conceptualization and prototypical development of interaction solutions that use visual cues and clear communication mechanisms to make the behavior of autonomous vehicles understandable.
Impact:
Increased trust and a better basis for decision-making for users through clearly understandable interactions between people and systems in critical traffic situations.
Problem:
The lack of clear interaction and communication concepts between autonomous vehicles and pedestrians led to uncertainty and made decision-making in road traffic more difficult.
Approach:
Conceptualization and prototypical development of interaction solutions that use visual cues and clear communication mechanisms to make the behavior of autonomous vehicles understandable.
Impact:
Increased trust and a better basis for decision-making for users through clearly understandable interactions between people and systems in critical traffic situations.
Problem:
The lack of clear interaction and communication concepts between autonomous vehicles and pedestrians led to uncertainty and made decision-making in road traffic more difficult.
Approach:
Conceptualization and prototypical development of interaction solutions that use visual cues and clear communication mechanisms to make the behavior of autonomous vehicles understandable.
Impact:
Increased trust and a better basis for decision-making for users through clearly understandable interactions between people and systems in critical traffic situations.
Problem:
The lack of clear interaction and communication concepts between autonomous vehicles and pedestrians led to uncertainty and made decision-making in road traffic more difficult.
Approach:
Conceptualization and prototypical development of interaction solutions that use visual cues and clear communication mechanisms to make the behavior of autonomous vehicles understandable.
Impact:
Increased trust and a better basis for decision-making for users through clearly understandable interactions between people and systems in critical traffic situations.
Problem:
The lack of clear interaction and communication concepts between autonomous vehicles and pedestrians led to uncertainty and made decision-making in road traffic more difficult.
Approach:
Conceptualization and prototypical development of interaction solutions that use visual cues and clear communication mechanisms to make the behavior of autonomous vehicles understandable.
Impact:
Increased trust and a better basis for decision-making for users through clearly understandable interactions between people and systems in critical traffic situations.
Problem:
The lack of clear interaction and communication concepts between autonomous vehicles and pedestrians led to uncertainty and made decision-making in road traffic more difficult.
Approach:
Conceptualization and prototypical development of interaction solutions that use visual cues and clear communication mechanisms to make the behavior of autonomous vehicles understandable.
Impact:
Increased trust and a better basis for decision-making for users through clearly understandable interactions between people and systems in critical traffic situations.
View study
Would you like to learn more about my study?
Here you can find my paper with all the key content.
Would you like to learn more about my study?
Here you can find my paper with all the key content.
View study
View study
View study
View study
Focus:
Focus:
UX Research, Trust & Interaction
UX Research, Trust & Interaction
Format:
Format:
Scientific Study / VR Experiment
Scientific Study / VR Experiment
Key questions:
Key questions:
Which gesture(s) can be used to stop automated vehicles?
Which gesture(s) can be used to stop automated vehicles?
Can a gesture like this strengthen pedestrians’ trust in automated vehicles?
Can a gesture like this strengthen pedestrians’ trust in automated vehicles?
My role
My role
Research
Research
Design of the test environment and questionnaires
Design of the test environment and questionnaires
Analyze
Analyze
Evaluation
Evaluation
Skills
Skills
Human–Vehicle Interaction Research
Human–Vehicle Interaction Research
Gesture-Based Interaction Design
Gesture-Based Interaction Design
Experimental UX Research
Experimental UX Research
Wizard-of-Oz Prototyping
Wizard-of-Oz Prototyping
Mixed-Methods Analysis
Mixed-Methods Analysis
Trust and Safety UX
Trust and Safety UX
Behavioral Pattern Analysis
Behavioral Pattern Analysis
Problem Statement
Problem statement
Problem Statement
In road traffic, pedestrians rely on implicit human signals such as eye contact or vehicle behavior. Autonomous vehicles disrupt this familiar communication — there is a lack of a clear, intuitive interface between humans and machines.
In road traffic, pedestrians rely on implicit human signals such as eye contact or vehicle behavior. Autonomous vehicles disrupt this familiar communication — there is a lack of a clear, intuitive interface between humans and machines.
Hypothesis
Hypothesis
Explicit hand gestures enable intuitive, always-available communication with autonomous vehicles—and thereby increase trust, safety, and willingness to cross.
Explicit hand gestures enable intuitive, always-available communication with autonomous vehicles—and thereby increase trust, safety, and willingness to cross.
01
01
Methodology
Methodology
Mixed-Methods Approach (Quantitative & Qualitative)
Mixed-Methods Approach (Quantitative & Qualitative)
Interviews conducted with participants (before and after the VR test) about trust and gesture selection
Interviews conducted with participants (before and after the VR test) about trust and gesture selection
VR Experiment (Wizard of Oz)
VR Experiment (Wizard of Oz)
Virtual test environment for participants
Virtual test environment for participants
Programming for manually stopping the vehicle at the touch of a button
Programming for manually stopping the vehicle at the touch of a button
21 participants
21 participants
Scenarios (Gesture Usage)
Scenarios (Gesture Usage)
Do not use a gesture (cross at your own risk)
Do not use a gesture (cross at your own risk)
Using a predefined gesture (palm raised and facing the vehicle)
Using a predefined gesture (palm raised and facing the vehicle)
Using a custom, freely chosen gesture
Using a custom, freely chosen gesture
Overview of the virtual test environment for participants (the red vehicle should be stopped).
Using a participant’s freely chosen gesture
Using a participant's predefined gesture
02
02
Key Findings
Key Findings
Gestures increase the willingness to cross from
~50 % to over 90 %
Gestures increase the willingness to cross from
~50 % to over 90 %
Self-selected gestures were the most effective
Self-selected gestures were the most effective
The open-palm gesture (“stop gesture”) was intuitively preferred
The open-palm gesture (“stop gesture”) was intuitively preferred
Younger participants showed higher trust than older participants.
Younger participants showed higher trust than older participants.
Most often, the hand facing the vehicle was used, regardless of whether the person was right- or left-handed.
Most often, the hand facing the vehicle was used, regardless of whether the person was right- or left-handed.
The chi-square test (χ²) examines whether there is a statistically significant relationship between two categorical variables — in this case, between the type of gesture and the vehicle’s response (stopping vs. not stopping). A high χ² value with a very small p-value (p < .001) indicates that the observed differences are not random, but that the gestures significantly influence the stopping behaviour of automated vehicles.
03
03
UX Insights
UX Insights
People seek explicit control in uncertain situations
People seek explicit control in uncertain situations
Trust is built through:
Trust is built through:
Visibility, Clarity & Predictability
Visibility, Clarity & Predictability
The human body itself becomes the interface
The human body itself becomes the interface
No additional hardware required → low barrier to entry
No additional hardware required → low barrier to entry
Challenges
Challenges
Potential misuse of gestures
Potential misuse of gestures
Traffic Flow Disruption
Traffic Flow Disruption
The Need for Clear System Rules and Feedback Mechanisms
The Need for Clear System Rules and Feedback Mechanisms
Differentiating Between Autonomous and Manual Vehicles
Differentiating Between Autonomous and Manual Vehicles
Gesture-based interaction is a promising approach to bridge the gap between people and autonomous vehicles.
It builds trust through simplicity — but requires clear rules, feedback, and systems thinking.
Gesture-based interaction is a promising approach to bridge the gap between people and autonomous vehicles.
It builds trust through simplicity — but requires clear rules, feedback, and systems thinking.
Research, where the future becomes reality.
Research, where the future becomes reality.
I investigate how people interact with new technologies — from VR to autonomous systems.
If you are looking for UX research that makes safety, trust, and behavior measurable
I investigate how people interact with new technologies — from VR to autonomous systems.
If you are looking for UX research that makes safety, trust, and behavior measurable
Interested? I’m always open to connecting, answering questions, or having a personal conversation.
Interested? I’m always open to connecting, answering questions, or having a personal conversation.
josefbley@web.de
josefbley@web.de
017661444919
017661444919
Josef Thomas Bley
Digital Product Designer
Engagement Architect
Josef Thomas Bley
Digital Product Designer
Engagement Architect
Josef Thomas Bley
Digital Product Designer
Engagement Architect
Josef Thomas Bley
Digital Product Designer
Engagement Architect
Josef Thomas Bley
Digital Product Designer
Engagement Architect
Josef Thomas Bley
Digital Product Designer
Engagement Architect
Josef Bley
Josef Bley
Josef Bley
Josef Bley