
The question
Most AR assistants are reactive: they wait for you to ask. A good human helper is proactive. They notice you’re looking for the coffee beans, or that you’re about to skip a step, and they speak up. To do that, an assistant needs a rough model of your mind: what you already know, what you’re trying to achieve, and what you’re about to do.
Today, proactive AR help is usually scripted by hand for each task, which doesn’t scale. Can an AI system infer these mental states on its own and decide when and how to help?
What we did
We first ran two formative studies with 12 AR and design experts to learn what proactive help should look like. We then built Satori on the Belief–Desire–Intention (BDI) model from psychology:
- Belief: what the user knows about the scene and the task, such as where an object is or how to use it.
- Desire: the user’s current goal, for example “make coffee.”
- Intention: the immediate next step, for example “wet the filter.”
Satori runs on a HoloLens. It combines the camera view, the conversation, and the interaction history with a multimodal LLM (GPT-4V) to infer belief, desire, and intention. It then predicts when to step in and what to show: text, a generated illustration, or both.
We tested it with 16 participants on four everyday tasks (arranging flowers, making coffee, cleaning a room, and connecting a Nintendo Switch). Each participant used both Satori and a Wizard-of-Oz baseline, in which AR designers had hand-crafted the guidance for each task.

What we found
1. It reads the task and the next step well
Satori identified the user’s task goal with 100% balanced accuracy on two datasets. Its scene understanding (belief) was close to a specialist model (66.5% vs. 69.5% on EgoTaskQA). It predicted the user’s next action with 78% precision. And it chose the same kind of help (text, image, or both) as expert designers 75% of the time.

2. Users rated it as good as designer-made help
On comprehensibility, helpfulness, timing, and learnability, Satori was statistically not worse than the hand-designed baseline (non-inferiority tests, p < .01). It scored better at reflecting users’ actual intentions and at placing guidance in the right spot (p < .05).

3. No extra mental effort
NASA-TLX workload scores showed no difference between Satori and the baseline on any of the six dimensions. Proactive help did not make the task feel harder.

My take
Links
BibTeX
@inproceedings{li2025satori,
title = {Satori: Towards Proactive AR Assistant with Belief-Desire-Intention User Modeling},
author = {Li, Chenyi and Wu, Guande and Chan, Gromit Yeuk-Yin and Turakhia, Dishita G. and
Castelo Quispe, Sonia and Li, Dong and Welch, Leslie and Silva, Cl{\'a}udio T. and
Qian, Jing},
booktitle = {Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems},
series = {CHI '25},
year = {2025},
publisher = {ACM},
doi = {10.1145/3706598.3714188}
}