Developing Innovative Technologies for Creating Realistic Animation of Digital Characters for Real-time Environments.

Abstract

The digital entertainment industry has seen exponential growth in recent years, of which the developments have been paral- leled by the evolution of the Metaverse. The Metaverse repre- sents a convergence of virtual and real worlds, where users in- teract through digital characters, necessitating advanced and authentic digital character animations. Traditional methods for creating 3D digital character animations involve complex, labour-intensive processes that are not conducive to real-time applications or large-scale production due to their high com- putational costs and extensive data requirements. The rapid advancement of virtual technology, big data, and artificial intelligence, especially in generative AI, offers promising so- lutions to these challenges. However, key research questions in this topic still remain, including how to create authentic 3D models with small data sizes, how to generate model ani- mations with dynamic effects in real-time, and how to create the skeletal motion of digital humans efficiently. This thesis develops innovative technologies for creating real- istic and immersive animations of digital characters in real- time environments, focusing on three primary procedures: digital character modelling, skin deformation methods, and skeletal motion construction. The contributions we provided in this research are concisely listed as follows: • A survey summarizing the recent developments and ap- plications of position-based approaches is proposed. This survey includes the core idea of the position based dy- namics (PBD) method, the advancements made inside the algorithm, the applications in other fields, and some guidance on the research directions for future work. • An innovative modelling method is introduced, integrat- ing the governing equation of elastic beam deformation and Newton’s second law to reconstruct dynamic 3D models with high accuracy and reduced data volumes. This PDE-based approach offers a time-dependent so- lution for creating detailed deformable models, signifi- cantly improving upon other baseline surface reconstruc- tion techniques. • A novel method for facial blendshape generation is pro- posed, leveraging an ODE-based surface creation method and Newton’s second law to produce natural facial ani- mations. This method effectively reduces the data size while maintaining natural edge continuities and high ef- ficiency in creating interpolated facial animations. • A new neural network structure, the Video-to-Motion (VTM) framework, is developed to reconstruct skeletal motion from video sequences. By pre-learning motion priors and jointly training them with the model, this method ensures high-fidelity motion reconstruction with substantial computational efficiency. To evaluate the efficacy of our proposed methods, various comprehensive experiments are conducted. These experiments demonstrate that the developed technologies can significantly benefit the process of creating realistic and immersive anima- tions for digital characters in real-time interactive scenarios. We believe that the completely constructed digital charac- ter animation can alleviate the labour cost in the game and animation industries and can be integrated with other ad- vanced game technologies to lead more users to enter the Metaverse, resulting in expanding the market and generating more wealth.