Human motion prediction.

Abstract

Motion data has been extensively used in the computer animation industry, feature films, pedestrian tracking, and surveillance. The capacity to understand and predict humans’ future movements is much sought-after, for it would have a range of practical applications in fields such as autonomous vehicles and interactive robotics. The complicated constraints of the human body and its high-dimensional dynamics, however, mean that human motion prediction is extremely challenging. A growing body of approaches are being used to develop various models for motion prediction. Traditional methods, such as Boltzmann machine and Markov chains, proposed possibility models for predicting the sequences of human movement. Furthermore, deep-learning models are introduced and most of them treat motion prediction in similar ways to machine translation problems. However, recent deep learning approaches adopted RNN, CNN, or Fully Connected Networks to learn motion features that do not fully exploit the hierarchical structure of human anatomy. Existing models suffer from the mean pose problem, and oversmoothing problem. Mean pose problem is the models prone to produce the fix pose for a time period. The oversmoothing problem is spefic exist in the graph neural network models for motion prediction. It means that when the neural network goes deeper, the feature learned from network are not distinguished anymore. In this PhD research, these problems are pursued and two new models for motion prediction are proposed. These models are state-of-the-art: not only are they highly efficient in computation and memory, but they produce realistic motion visualizations too. The first model used the hierarchical structure of human body to reduce the parameter size significantly. The second model introduced a densely connected GCN model to reduce the oversmoothing problem. To be more specific, a convolutional hierarchical autoencoder model for motion prediction is proposed. Its novel encoder incorporates 1D convolutional layers and hierarchical topology. The new network is smaller and faster than existing deep-learning models. The qualitative and quantitative results show that these models outperform state-of-the-art methods in both short-term and long-term prediction. Following the recent success of graph neural network, an advanced GCN based framework for motion prediction is proposed which connects all the GCN blocks directly. Therefore, the output feature of each GCN block skips the middle layers and jumps to the final layers. Compared to the existing GCN model for motion prediction, this model requires almost the same level of parameters. But it significantly enlarged the feature maps utilization and increased the impact of earlier layers’ feature map. Moreover, this densely connected structure makes the model for motion prediction easier to train and able to go deeper. All these models are evaluated by conducting extensive comparison experiments on the standard benchmarks for motion prediction, which are Human3.6M and the CMU. The performance evaluated from both the angle and 3D position aspects, demonstrated these models’ superiority over the state-of-the-art works.