Observe interaction
Bring body movement, controller input and machine telemetry together on a shared timeline.

A human-understanding layer for intelligent systems.
We study how people and machines coordinate-and how that coordination changes.
See how it worksA person acts. The machine responds. That response shapes the person’s next action. Human-machine coupling describes how well the two coordinate through this continuous loop.
The Human State Layer brings movement, control inputs and machine feedback onto a shared timeline. It measures changes in coordination, then uses the task context to assess where a change may come from: the person, interface or machine.
The result is useful information for the machine’s existing decision logic: what is changing, how certain the estimate is, and when support may help. Assistance can adapt its timing or intensity while the person retains control.
Three views of the same moment.
How well do actions and responses fit together?
What changed-and what is its likely source?
Estimates include confidence and task context.
The machine uses this information to choose appropriate support.
Where human understanding can make a difference: four possible applications of the same interaction loop.

Coordinate under pressure.
A firefighter’s control inputs and the drone’s flight data reveal how well they are working together. Changes in coordination can guide clearer feedback and timely flight assistance.

Support at the person’s pace.
Movement, gestures and robot feedback help identify when an interaction slows or becomes difficult. The robot can pause, clarify its intention or adjust the pace of assistance.

Make handovers more informed.
Steering behavior, driver movement and vehicle responses provide a shared view of the interaction. This can guide the timing of prompts and transitions between driver and automated support.
An interaction that responds to you.
In therapy and training scenarios, movement and conversational timing can help a virtual character respond to the participant-adapting its pace, prompts and behavior as the interaction unfolds.
Our foundation is movement-dynamics research in VR. We are investigating how that principle extends to human-controller-machine systems.
Body keypoints, interaction signals and machine telemetry can provide complementary information. Meaning comes from how the signals relate to the person, the task and the system-not from a movement in isolation.
Explore Our ResearchWe’re interested in exploring how human understanding can improve the interaction between people and intelligent machines.
Get in touch