Basketball players need lots of practice before they master the dribble, and it turns out that’s true for computer-animated players as well. By using deep reinforcement learning, players in video basketball games can glean insights from motion capture data to sharpen their dribbling skills.

Researchers at Carnegie Mellon University and DeepMotion Inc., a California company that develops smart avatars, have for the first time developed a physics-based, real-time method for controlling animated characters that can learn dribbling skills from experience. In this case, the system learns from motion capture of the movements performed by people dribbling basketballs.

This trial-and-error learning process is time consuming, requiring millions of trials, but the results are arm movements that are closely coordinated with physically plausible ball movement. Players learn to dribble between their legs, dribble behind their backs and do crossover moves, as well as how to transition from one skill to another.

“Once the skills are learned, new motions can be simulated much faster than real-time,” said Jessica Hodgins, Carnegie Mellon professor of computer science and robotics.

“This research opens the door to simulating sports with skilled virtual avatars,” said Liu, the report’s first author. “The technology can be applied beyond sport simulation to create more interactive characters for gaming, animation, motion analysis, and in the future, robotics.”

Hodgins and Libin Liu, chief scientist at DeepMotion, will present the method at SIGGRAPH 2018, the Conference on Computer Graphics and Interactive Techniques, Aug. 12-18, in Vancouver.