Professor Taku Komura, chief scientist at Miro Dynamics, has received the Test-of-Time Award at SIGGRAPH 2026, the leading conference in computer graphics. The award recognizes papers published a decade earlier that still shape the field today. This year it went to "A Deep Learning Framework for Character Motion Synthesis and Editing", published by Taku and his co-authors in 2016.
That paper was the first to use deep learning to recover the underlying structure of human motion from large motion-capture corpora, and then generate natural character motion from high-level instructions. At a time when AI breakthroughs were concentrated in images, it applied representation learning systematically to complex 3D motion — taking AI from learning how to see to learning how to move.
Ten years later the work reaches well past character animation. How a machine learns from human motion and interaction, understands the physical world and acts in it, is exactly the question physical AI now has to answer.
Twenty years of making the body computable
Taku Komura has worked on that question for over twenty years. He taught at the University of Edinburgh for many years and joined the University of Hong Kong as a professor of computer science in 2020. He chaired SIGGRAPH Asia 2025, and AI 2000 ranks him among the 15 most influential scholars in the field worldwide.
His goal is to have machines learn the structure of human motion from data, and from there understand how movement changes with the scene, the object and the task. Starting from the 2016 paper, he and his team followed one line — the interaction between body, objects and environment — through a series of results.
The 2016 work used an autoencoder to distill a low-dimensional motion space out of large motion-capture data. It established the human motion prior, and let a system generate natural human movement.
The team then moved motion into complex scenes. Neural State Machine (2019) let digital characters sit down, carry objects and avoid obstacles based on scene geometry; Local Motion Phases (2020) handled physical interaction with multiple contact points and full-body coordination.
In 2022 the team won a SIGGRAPH Best Paper Award for DeepPhase, which uses a periodic autoencoder to extract a multi-dimensional phase space directly from raw, uncleaned data — solving how motion stays coherent over time.
That line of research became the open-source project AI4Animation, now past 8,000 GitHub stars and one of the most influential open ecosystems in motion generation.
Why a human interaction prior matters for embodied AI
Put end to end, that line is a complete human interaction prior: first learn to represent natural human motion in the real world, then learn how people interact with scenes, objects and tasks, then bring real contact, force and environmental change into the model.
The robotics industry is now proving out why this matters. Teleoperated capture on real robots is expensive, narrow in scene coverage and unnatural in motion; general capability is not going to come from it. If a machine is to act in the real world, the more workable source is how people already operate in theirs.
But human data is not usable the moment you record it. To capture enough of it, the hardware has to be cheap enough to spread — and cheap sensors are noisy and incomplete. The human interaction prior closes that gap: it knows which poses and which changes over time belong to natural human motion, so it can fill in what the signal is missing and constrain the result back to real human movement.
On that prior, the team now captures and reconstructs with consumer devices — an iPhone will do — and gets hands, objects and scene as 3D in one world frame. Capture cost drops to a fraction of what it was, and error on the public egocentric hand-reconstruction benchmark is down 60%.
The value here is more than cheaper capture. It points at a judgment call: the next opening in embodied AI is in consumer settings, not controlled ones. Tidying a room, working in a kitchen, handling everyday objects — enormous, scattered, near-zero marginal cost, and until now without a scalable source of quality data, because the motion is fine-grained and the contact is complicated.
This approach turns real everyday operation, shot on consumer hardware, into physically valid training data. Embodied AI and world models will scale out of the factory line and into ordinary life.
A human-native multimodal world model
Capturing the data is only the first step. Behavior cloning lets a machine imitate a demonstration, but acting in the real world also requires knowing what an action will cause, and the physics behind that outcome. What the team is building is a multimodal world model trained on human-native data.
Every current embodied data format — egocentric video, UMI, robot teleoperation — carries only vision and action. Yet two motions that look alike can involve entirely different contact and produce entirely different results. On vision and trajectory alone, a model has little to separate them by.
The team fills in those missing signals on top of human-native operation data. The capture rig is built to be worn: the egocentric camera records what the person sees, the reconstructed 3D state separates camera motion from real change in the environment, gaze records what the person is attending to, and sEMG records force and contact. The wearer just goes about their task, and the result is several modalities aligned in time.
That changes what the world model has to predict. Not the next frame of pixels, but how the physical state moves: the 3D state of an object, where contact happens, how much force is involved. What actually helps a robot is knowing whether a push will topple something, or whether a grip is firm enough to open a jar. That is the world state a machine needs — not a pile of pixels.
Human-centered data is not the end point. There is a limit to what people can demonstrate, and a robot will eventually face tasks nobody has done. We want robots that, having learned from human experience, keep building their own understanding of the world through acting and exploring — toward genuine self-directed learning. The human-centered paradigm we work in today is groundwork for that.
Physical AI is about how a robot adapts to an unfamiliar environment and learns new skills from limited interaction. Those questions are not settled by public datasets or one-off experiments. They take sustained capture, repeated testing on real hardware, and continual adjustment against real tasks — which is why this research needs to be done with industry, not beside it.
From understanding motion to understanding the world
After many years teaching at Edinburgh, Taku chose to continue his research in Hong Kong, and recently co-founded Miro Dynamics. He sees the case for doing this work in China: the academic field is active, and there are many places for it to land in industry, which gives physical AI research a lot to test itself against.
What he wants to push on next is the more fundamental goal of physical AI — robots that form an internal representation of the physical world, understand body, action and physics through real interaction, and carry behavior that transfers across robots, tasks and environments. Multimodal human-native operation data is the way in.
Time stops for no one, but it rewards those who understand it.